[SOLVED] Adding Nodes at Runtime with SofaPython3 and changing Colors of Mesh

[Scheikl]

Hi,
first of all thanks for SOFA in general and the incredible community support in particular!

I am trying to create a scenario where a cauterization hook touches and cauterizes tissue. I do not really need the underlying mechanics to be correct, all I want is visual feedback of where the hook has already touched the tissue.

My first approach was as following:
1) Create a scene in python3 via the createScene() function
2) Run the simulation from within python3
3) Try to create a new node with the addChild() function of the root node

def Marker(rootNode, name, pose):

    marker = rootNode.addChild(name)
    marker.addObject(
        "MechanicalObject", name="MechanicalModel", template="Rigid3", position=pose
    )

    visual = marker.addChild("VisualModel")

    visual.loader = visual.addObject(
        "MeshObjLoader",
        handleSeams="1",
        name="VisualMeshLoader",
        scale=1,
        filename="mesh/ball.obj",
    )

    visual.addObject(
        "OglModel", src="@VisualMeshLoader", name="VisualModel", color=[1, 1, 1]
    )

    visual.addObject(
        "RigidMapping",
        input="@MechanicalModel",
        name="MM->VM mapping",
        output="@VisualModel",
    )
    print(f"Created {visual} with name {name}!")

    return marker

This runs without error, but the object is neither visible, nor is its position correctly set.

Created <Sofa.Core.Node object at 0x7f185357a6b0> with name test

print(marker.MechanicalObject.position.value)
[[0, 0, 0, 0, 0, 0, 1]]

Do you know whether it is possible to add new things to the simulation during runtime?

The second approach would be to locally change the color of the mesh.
Do you know whether it is possible to change the color of a mesh (a) locally (only specific nodes) and (b) at runtime from SofaPython3?

Or do you have a completely different idea I should try?

Looking forward to your answers,
Paul

[jnbrunet]

Hey Paul,

What you don’t see when you launch a python scene with runSofa is that the following things happen:

1. SOFA creates a root node
2. It calls the method createScene(root) of your script
3. And then, SOFA calls root.init() which will visit the root tree and call the init method on every components.

The last step is quite important as most SOFA components initialize their vectors/data fields during this “init” phase, and not during the creation of the object. There is ongoing work right now that aims at changing this behavior to better handle dynamic creation/modification of objects.

Fortunately, in SofaPython3 you are allowed to manually launch this “init” phase on any nodes you created:

def Marker(rootNode, name, pose):
    marker = rootNode.addChild(name)
    (...)
    marker.init() # recursively initialize components of the node tree
    print(marker.MechanicalObject.position.value)
    return marker

Hopefully this should solve your issue
J-N

[Scheikl]

Hi J-N,
thank you for your fast response and sorry for my late reply.

I added the marker.init() and it runs without error.

Sadly, the model is still not visible, and the position is also at a constant for:
print(marker.MechanicalObject.position.value)
[[0, 0, 0, 0, 0, 0, 1]]

Is there a possibility that I added the new node to a “wrong root”?
I pass a node to the function, that I previously assigned to a class with self._sofa_root = Core.Node("myroot").
I also use this node when I pass it to Simulation.init() of Sofa.Simulation

[jnbrunet]

Hey Paul,

Looks like the OglModel needs a special visual initialization: Sofa.Simulation.initVisual(marker). The following scene works for me, it adds a sphere at each time step.

import Sofa
import Sofa.Simulation

def Marker(rootNode, name, pose):

    marker = rootNode.addChild(name)
    marker.addObject(
        "MechanicalObject", name="MechanicalModel", template="Rigid3", position=pose
    )

    visual = marker.addChild("VisualModel")

    visual.loader = visual.addObject(
        "MeshObjLoader",
        handleSeams="1",
        name="VisualMeshLoader",
        scale=1,
        filename="mesh/ball.obj",
    )

    visual.addObject(
        "OglModel", src="@VisualMeshLoader", name="VisualModel", color=[1, 1, 1]
    )

    visual.addObject(
        "RigidMapping",
        input="@../MechanicalModel",
        name="MM->VM mapping",
        output="@VisualModel",
    )
    print(f"Created {visual} with name {name}!")
    
    marker.init()
    Sofa.Simulation.initVisual(marker)

    return marker
    
class Controller(Sofa.Core.Controller):

    def __init__(self):
        Sofa.Core.Controller.__init__(self)
        self.count = 0
        
    def onAnimateBeginEvent(self, e):
        Marker(self.getContext(), f'marker_{self.count}', [self.count*2, 0, 0, 0, 0, 0, 1])
        self.count+=1
    
def createScene(root):
    root.bbox = [[-1, -1, -1], [1, 1, 1]]
    root.addObject(Controller())

J-N

[Scheikl]

Hi J-N,
aaahh I see.

Unfortunately by SofaPython3 version is too old (it does not feature the initVisual function) and I ran into some dependency errors with Sofa.

I will try to get it working and give you a heads up, once that works.

Thanks a lot for your help!

Cheers,
Paul

[Scheikl]

Quick question on the side: Do you think the second approach is feasible as well?

[jnbrunet]

Hey Paul,

I never tried, by I don’t see anything that would prevent changing the color of an OglModel during runtime.

However, as far as I know, OglModel is using a texture map, so you would have to provide it with an image containing the different colors of your nodes, and assign the texture coordinates to each (or some) nodes during runtime. The texture image must be provided by a filename and is loaded during the init phase, so you probably won’t be able to change it during runtime.

In your example, you can have a look at the texture coordinates of your OglModel nodes with:

print(visual.VisualModel.texcoords.value)

and you can change the texture coordinates of a subset of nodes with, for example,

blue_nodes = [0, 5, 6, 7] # indices of nodes that must be blue
blue_coordinates = [0.7, 0.9] # coordinates of a blue pixel in the texture image
with visual.VisualModel.texcoords.writeableArray() as wa:
    wa[blue_nodes] = blue_coordinates    

J-N

Edit:
Don’t forget that the color of a triangle is interpolated from the color of its nodes.
If you want to change the color of triangles, I guess you could do something like

triangles = visual.VisualModel.triangles.value
blue_triangles = [0, 4, 9] # indices of triangles that must be blue
blue_nodes = triangles[blue_triangles]
blue_coordinates = [0.7, 0.9] # coordinates of a blue pixel in the texture image
with visual.VisualModel.texcoords.writeableArray() as wa:
    wa[blue_nodes] = blue_coordinates    

[Scheikl]

Hi J-N,
that looks promising, thank you very much! 🙂

[Scheikl]

Hi J-N,
I finally managed to get everything running. There were some weird version mismatches between pybind11, Sofa, SofaPython3, and *drumroll* Qt.

Your example works perfectly fine and a marker is created at each time step.

Thanks again for the help!

Cheers,
Paul

[Author]

Posts