Visual not updating when using with a GLFW window (ImGUI)

[BrunoB81HK]

Hi!

For my plugin, I needed to plot some data to help with the debugging and to control some variable too. To do this, I used ImGUI with glfw. I used this because it is very easy to setup and let me do exactly what I want.

In init(), I call the following method to initialize ImGUI:

void SofaTrexConnector::initTrexUI()
{
    glfwSetErrorCallback(glfw_error_callback);
    glfwInit();
    glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
    glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 0);

    // Create window with graphics context
    this->trex_window = glfwCreateWindow(WINDOW_INIT_SIZE, "TrexGui Controls", NULL, NULL);
    glfwMakeContextCurrent(trex_window);
    glfwSwapInterval(0); // remove the fps cap

    // Initialize OpenGL loader
    bool err = glewInit() != GLEW_OK;
    if (err)
    {
        msg_error("glew") << stderr << "Failed to initialize OpenGL loader!\n";
    }

    // Setup Dear ImGui context
    IMGUI_CHECKVERSION();
    ImGui::CreateContext();
    ImPlot::CreateContext();
    ImGuiIO& io = ImGui::GetIO(); (void)io;

    // Setup Dear ImGui style
    ImGui::StyleColorsDark();

    // Setup Platform/Renderer backends
    ImGui_ImplGlfw_InitForOpenGL(trex_window, true);
    ImGui_ImplOpenGL3_Init("#version 130");

    // Set the window flags
    window_flags |= ImGuiWindowFlags_NoScrollbar;
    window_flags |= ImGuiWindowFlags_NoCollapse;
    window_flags |= ImGuiWindowFlags_NoBackground;
}

Also, in the destructor of my component, I call the following method :

void SofaTrexConnector::clearTrexUI()
{
    // ImGUI cleanup
    ImGui_ImplOpenGL3_Shutdown();
    ImGui_ImplGlfw_Shutdown();
    ImPlot::DestroyContext();
    ImGui::DestroyContext();

    glfwDestroyWindow(trex_window);
    glfwTerminate();
}

To render the window, my code is the following :

void SofaTrexConnector::drawTrexUI()
{
    glfwMakeContextCurrent(trex_window);
    glfwPollEvents();

    // Start the Dear ImGui frame
    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();

    ImGui::Begin("Trex controls", NULL, window_flags);

    ... // Widgets definitions

    ImGui::End();  // Trex controls

    // Rendering
    ImGui::Render();
    int display_w, display_h;
    glfwGetFramebufferSize(trex_window, &display_w, &display_h);
    glViewport(0, 0, display_w, display_h);
    glClearColor(0, 0, 0, 0);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

    glfwSwapBuffers(trex_window);
}

My problem is the following :

* If I call drawTrexUI() in my component’s draw() method, my scene visuals models are rendered, but every other model aren’t. If I want to display the force field or the collision model, they won’t render. The scene still works as intended tho.

* If I call drawTrexUI() on the AnimationEndEvent or the AnimationBeginEvent, everything is rendered, but the visual models doesn’t move. It’s like they are frozen at their first frame. Apart from that, the scene works as intended.

I feel like this has to do with a rendering conflict between the windows, but I am not sure.

Thanks a lot!

Bruno

[Hugo]

Hi @brunob81hk

For your information, a plugin has recently been released by @froy : SOFA GLFW. This plugin now brings a simple GUI based on GLFW in SOFA. The PR #2062 integrates it within SOFA (subdir) so that you can run :
./runSofa -g glfw

Regarding your problem, I am sure @froy would be able to help. Sorry I am not myself an expert in rendering paths. I guess here the openGL path and GLFW path are not always compatible.

I hope this helps already a bit.
Best

Hugo

[BrunoB81HK]

Hi @hugo

Thanks for your answer. I’ve seen SOFA_GLFW but I haven’t been able to make it work. I’m not using the master branch so I guess it would been hard to make it work on my setup. I look very interesting and I think it could help me solve my problem.

When I try to configure with CMake, I get the following error :

Could not find a package configuration file provided by "Sofa.Config" with any of the following names:
    Sofa.ConfigConfig.cmake
    sofa.config-config.cmake

I added this with the external plugin repo.

Thanks a lot and have a nice day!

Bruno

[Hugo]

Hi @brunob81hk

Usually we do not try to make back-compatibility since the SOFA GLFW plugin was created recently and therefore compatible with only recent versions of SOFA.

However, since the plugin is quite simple, it should not be too hard to make it work.
Could you try replacing:

find_package(Sofa.Config REQUIRED)

by

find_package(SofaFramework REQUIRED)

Best wishes,

Hugo

[BrunoB81HK]

Hi @hugo

When I replace it, I get other component that aren’t found.

I guess I’ll wait until the next stable release of SOFA (21.06).

Thank you and see you then!

[Hugo]

Hi @brunob81hk

If there is no hurry indeed, it’s easier. It should come within a week or two. Stay tuned 😉
Best,

Hugo

[BrunoB81HK]

Hi @hugo & @froy

I’ve now switched to Sofa v21.06 and I’m trying to use SofaGLFW. I’ve been able to build it and run runSofaGLFW. When I’m trying to run my scenes with runSofaGLFW -f /my/scene.scn, I get a black window along with my custom window using ImGUI which work as intended.

I changed my code so that the initialization and termination of GLFW is handled by runSofaGLFW.

Called in my plugin’s init() method ;

void SofaTrexConnector::initTrexUI()
{
    /*glfwSetErrorCallback(glfw_error_callback);
    glfwInit();
    glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
    glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 0);*/

    // Create window with graphics context
    this->m_trexWindow = glfwCreateWindow(SOFATREX_CONFIG_WINDOW_INIT_SIZE, "TrexGui Controls", NULL, NULL);
    glfwMakeContextCurrent(this->m_trexWindow);
    glfwSwapInterval(0); // remove the fps cap

    // Initialize OpenGL loader
    /*bool err = glewInit() != GLEW_OK;
    if (err)
    {
        msg_error("glew") << stderr << "Failed to initialize OpenGL loader!\n";
    }*/

    // Setup Dear ImGui context
    IMGUI_CHECKVERSION();
    ImGui::CreateContext();
    ImPlot::CreateContext();
    ImGuiIO& io = ImGui::GetIO(); (void)io;

    // Setup Dear ImGui style
    ImGui::StyleColorsDark();

    // Setup Platform/Renderer backends
    ImGui_ImplGlfw_InitForOpenGL(this->m_trexWindow, true);
    ImGui_ImplOpenGL3_Init("#version 130");

    // Set the window flags
    this->m_trexWindowFlags |= ImGuiWindowFlags_NoScrollbar;
    this->m_trexWindowFlags |= ImGuiWindowFlags_NoCollapse;
    this->m_trexWindowFlags |= ImGuiWindowFlags_NoBackground;
    this->m_trexWindowFlags |= ImGuiWindowFlags_NoMove;
    this->m_trexWindowFlags |= ImGuiWindowFlags_NoResize;

Called in my plugin’s draw() method or in handleEvent with an AnimationEndEvent or an AnimationBeginEvent :

void SofaTrexConnector::drawTrexUI()
{
    glfwMakeContextCurrent(this->m_trexWindow);
    glfwPollEvents();

    // Start the Dear ImGui frame
    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();

    ImGui::SetNextWindowPos({SOFATREX_CONFIG_CONTROL_WINDOWS_POSITION}, ImGuiCond_Once);
    ImGui::SetNextWindowSize({SOFATREX_CONFIG_CONTROL_WINDOWS_SIZE}, ImGuiCond_Once);
    ImGui::Begin("Trex controls", NULL, this->m_trexWindowFlags);

    ... // Widgets definitions

    ImGui::End(); // Trex controls

    // Rendering
    ImGui::Render();
    int display_w, display_h;
    glfwGetFramebufferSize(this->m_trexWindow, &display_w, &display_h);
    glViewport(0, 0, display_w, display_h);
    glClearColor(0, 0, 0, 0);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

    glfwSwapBuffers(this->m_trexWindow);
}

Called in my plugin’s destructor :

void SofaTrexConnector::clearTrexUI()
{
    // ImGUI cleanup
    ImGui_ImplOpenGL3_Shutdown();
    ImGui_ImplGlfw_Shutdown();
    ImPlot::DestroyContext();
    ImGui::DestroyContext();

    glfwDestroyWindow(this->m_trexWindow);
    //glfwTerminate();
}

I don’t really know what to try next. What should I do?

Thanks a lot and have a nice day!

P.S. I don’t know if it’s related, but it seems like I can’t use runSofa -g glfw. I get : [ERROR] [GUIManager] GUI 'glfw' creation failed..

P.P.S Also, I don’t know if it’s related, but when using runSofaGLFW, I get every message twice in the console :

/home/busb2601/sofa/build/v21.06/bin/runSofaGLFW -a -f /home/busb2601/sofa/ext_plugin_repo/SofaTrex/scenes/Empty/PokeRefEmpty.scn
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaTrex.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaTrex.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaGraphComponent.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaGraphComponent.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaOpenglVisual.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaOpenglVisual.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaHaptics.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaHaptics.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaBoundaryCondition.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaBoundaryCondition.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaConstraint.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaConstraint.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaImplicitOdeSolver.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaImplicitOdeSolver.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaLoader.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaLoader.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaMeshCollision.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaMeshCollision.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaRigid.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaRigid.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaSimpleFem.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaSimpleFem.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaGeneralDeformable.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaGeneralDeformable.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaUserInteraction.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaUserInteraction.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaDeformable.so
[INFO]    [PluginManager] Loaded plugin: /home/busb2601/sofa/build/v21.06/lib/libSofaDeformable.so
[WARNING] [InteractiveCamera(baseCamera)] Too many missing parameters ; taking default ...
[WARNING] [InteractiveCamera(baseCamera)] Too many missing parameters ; taking default ...
[INFO]    [SofaTrexConnector(TrexDriver)] Cleaning Trex mess...
[INFO]    [SofaTrexConnector(TrexDriver)] Cleaning Trex mess...

Process finished with exit code 0

[BrunoB81HK]

Hi @hugo,

This topics was incorrectly flagged as solved. Do you have any idea on what I should do to solve this?

Thanks a lot,

Bruno

[Hugo]

Hey @brunob81hk

Sorry about this mis-solved flag!
Hey @froy could you take a look at it please?

Hugo

[Froy]

Hello @brunob81hk

– if runSofa reports that it cannot find the gui “glfw”, it would mean that the plugin has not been loaded. You may want to force loading it while running runSofa, with this command runSofa.exe -l SofaGLFW .....
– where do you call initTrexUI() ? This is important as the BaseObject::init() is not supposed to be enclosed in a OpenGL context. In BaseObject, only draw() has an OpenGL context, for debugging purpose usually.

I would advise you to make your component inheriting VisualModel, as this abstract class is more suited for “custom” draw commands. This comes with initVisual() (init things like vertex buffer, etc), updateVisual() (update buffer, vbo stuff like that) and drawVisual(calling draw commands). In your case, I would call initTrexUI() in initVisual() and drawTrexUI() in drawVisual()

[BrunoB81HK]

Hi @Froy

– Using runSofa -l SofaGLFW is working so that’s a good thing.

– In the next weeks, I’ll try what you’re proposing.

Thanks a lot for your input!

Bruno

[Froy]

Great @Brunob81hk,
Keep us in touch, it will be interesting to see your results 🤠

++
Fred

[BrunoB81HK]

Hi @Froy

I’ve tried what you proposed to me and here is the follow-up.

My plugin class now inherit VisualModel. I put my initTrexUI() in initVisual() and my drawTrexUI() in drawVisual() and I came up with some issues.

Just as a reminder:

* If I call drawTrexUI() in my component’s draw() method, my scene visuals models are rendered, but every other model aren’t. If I want to display the force field or the collision model, they won’t render. The scene still works as intended tho.

* If I call drawTrexUI() on the AnimationEndEvent or the AnimationBeginEvent, everything is rendered, but the visual models doesn’t move. It’s like they are frozen at their first frame. Apart from that, the scene works as intended.

* Also, I should add that on either of those solutions, the background needed to be set from runSofa to have a custom one. Those set from the .scn files were not rendering (the paths was in the Viewer/Image tab on runSofa, but I needed to select it and press enter to make it change).

With that in mind, with the changes done, I get my custom window to work flawlessly, but the main runSofa windows won’t show any models. Something that is also strange is that it displays the right background this time around. I tried to put the drawTrexUI() method inside updateVisual(), but I get the following error at runtime:

ERROR: ImGui_ImplOpenGL3_CreateDeviceObjects: failed to compile vertex shader!
0:2(1): error: <code>in' qualifier in declaration of</code>Position' only valid for function parameters in GLSL 1.10
0:3(1): error: <code>in' qualifier in declaration of</code>UV' only valid for function parameters in GLSL 1.10
0:4(1): error: <code>in' qualifier in declaration of</code>Color' only valid for function parameters in GLSL 1.10
0:5(1): error: <code>out' qualifier in declaration of</code>Frag_UV' only valid for function parameters in GLSL 1.10
0:6(1): error: <code>out' qualifier in declaration of</code>Frag_Color' only valid for function parameters in GLSL 1.10

ERROR: ImGui_ImplOpenGL3_CreateDeviceObjects: failed to compile fragment shader!
0:2(1): error: <code>in' qualifier in declaration of</code>Frag_UV' only valid for function parameters in GLSL 1.10
0:3(1): error: <code>in' qualifier in declaration of</code>Frag_Color' only valid for function parameters in GLSL 1.10
0:4(1): error: <code>out' qualifier in declaration of</code>Out_Color' only valid for function parameters in GLSL 1.10
0:7(27): error: no function with name 'texture'
0:7(14): error: operands to arithmetic operators must be numeric

ERROR: ImGui_ImplOpenGL3_CreateDeviceObjects: failed to link shader program! (with GLSL '')
error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader

Do you have any ideas what is causing all this? What should I try next?

Thanks a lot for your help!

Bruno

[Author]

Posts