rendering 3d elements on top

Hey, I’m creating a transformation tool that allows translating, rotating and scaling of polydata. To do this, I thought to create a structure within the 3d view that you can drag to transform the polydata like this:
Screenshot 2023-03-01 at 11.01.59
My issue with this is that I want to let the arrows appear on top of the polydata. Is there a way to do this?

I believe it should be possible to achieve this by having one pass where you draw all polydata, resetting the depth buffer and then drawing the controls. I’m not sure if it is possible to implement something like this in vtk.js?

1 Like

Hello !
A solution could be arrowActor.setForceTranslucent(true);.
It should force the actor to be rendered in an OrderIndependantTranslucentPass after the opaquePass, which will disable depth test and buffer writing.

1 Like

While arrowActor.setForceTranslucent(true); may work, it is more of a bug than what is expected from
The order independent pass shall NOT ignore the depth buffer (you do not want to have far translucent fragments rendered on top of near opaque fragments).

To me the best way is to render your actors in the final overlay pass.
Problem is, that overlay pass does not clear the depth buffer.
@sankhesh, don’t you think it could be the default behavior for the overlay pass, or at least an option for such pass ?

@nomis6432 , it would be great if you could contribute your widget/tool to VTK.js. It will have the benefit to then be supported by the community.

Hey, thanks for the responses.

So for the overlay pass I’d have to modify this code in the ForwardPass class?:

if (model.overlayActorCount > 0) {
    // reset depth buffer
    publicAPI.setCurrentOperation('overlayPass');
    renNode.traverse(publicAPI);
}

I was wondering if you know how to reset the depth buffer. I’m not familiar with webgl and couldn’t find where this is currently done.
One issue with this is that we’re still on version 17.0.0 which doesn’t have this overlay pass yet as far as I can tell. I could look into upgrading but that would require some work.

Maybe I could make a custom vtkForwardPass class with the added functionality? Would it be possible to replace the vtkForwardPass without modifying the VTK.js code?

I would love to contribute the widget to VTK.js. I can’t however promise this yet since I’d have to ask my boss for that and since we’re on an older version, it might not be compatible with the latest version.

Hey, so my current approach is overwriting the vtkForwardPass, vtkOpenglActor, vtkOpenglCamera, vtkOpenGLPolyDataMapper and making a seperate custom vtkActor (vtkOverlayActor3d) . The later is exatly the same as the normal actor except it has an attribute on the publicAPI to easily distinguish it from normal actors. I added a different passOverlay3d and traversePassOverlay3d to the other classes where needed. This is equivalent to the opaquePass function except that the opaquePass filters out normal actors and the traversePassOverlay3d handles the vtkOverlayActor3d.

I then add a step to the vtkForwardPass:

publicAPI.setCurrentOperation('cameraPass');
renNode.traverse(publicAPI);
if (model.opaqueActorCount > 0) {
    publicAPI.setCurrentOperation('opaquePass');
    renNode.traverse(publicAPI);
}
if (model.translucentActorCount > 0) {
    publicAPI.setCurrentOperation('translucentPass');
    renNode.traverse(publicAPI);
}
if (model.volumeCount > 0) {
    publicAPI.setCurrentOperation('volumePass');
    renNode.traverse(publicAPI);
}
if (model.overlay3dActorCount > 0) {
    const context = viewNode.getContext();
    context.clearDepth(1.0);
    publicAPI.setCurrentOperation('passOverlay3d');
    renNode.traverse(publicAPI);
}

These vtkOverlayActor3ds are visualized but they don’t yet ignore the depth buffer. I thus believe clearDepth doesn’t work? I was wondering if you happen to know how to reset the depth buffer

doing context.disable(context.DEPTH_TEST) does disable the depth buffer for just these overlay actors but that’s not the behaviour I’m looking for

I was wondering if you happen to know how to reset the depth buffer

gl.clear(gl.DEPTH_BUFFER_BIT)

To avoid overwriting classes in VTK.js, here is what you could do:
Create a PR in VTK.js with:

  • a new overlay boolean property in vtkActor and you conditionally call incrementOverlayActorCount in publicAPI.queryPass. You will also have to redirect publicAPI.translucentPass to publicAPI.opaquePass. (Or use vtkActor2D if that works instead)
  • a new clearDepthBeforeOverlayPass boolean property in vtkForwardPass and you then optionally call gl.clear(gl.DEPTH_BUFFER_BIT).

@sankhesh

Hey,

I still had to call gl.depthMask(true); before gl.clear(gl.DEPTH_BUFFER_BIT) for it to work but now It does.

I’m open to contribute some of my work once I’m done but I’ll have to see since I’m still on an older version.

Thanks a lot for all the info

current result:

Screenshot 2023-03-03 at 11.20.37

@nomis6432 Did you try using Actor2D/PolyDataMapper2D? That should do exactly what you want without having the manually clear depth.

1 Like

Hey, @sankhesh . I haven’t tried using PolyDataMapper2D yet since it was introduced after the version I’m using (17.0.0) but it does seem interesting. I’ll see if I can update. Thanks a lot

1 Like

I’ve upgraded to the latest vtk.js version and tried using Actor2D/PolyDataMapper2D. I was wondering if there is a simple way to make sure the axis always apear on top of the center of the 3d polyData I want to move. I’ve tried setting the origin of the axis on the polydata but that doesn’t seem to affect the position. The only way I seem to be able to control the position of the Actor2D is with setDisplayPosition which is the 2d pixel position

EDIT: for some reason the PolyDataMapper2D also doesn’t render vtkConcentricCylinderSource. I’ve tried passing it first through the vtkTriangleFilter since it has cells with 4 points but that didn’t fix it.

EDIT2: I’ve figured out I can set the position to the woorld coordinates by calling setCoordinateSystemToWorld() on the Actor2D but the orientation doesn’t update. The axis dont point to the woorld coordinates x, y and z direction but stay the same relative to the screen.

EDIT3: Property2D doesn’t have edgeVisibility. Is there a way to add this. If it’s not too much work I could make a PR for this

EDIT4: It also doesn’t seem like Actor2D ignores the depth buffer and writes everything on top of each other. This is not ideal for my case since I’d still want the arrows to be displayed correctly when they overlap. In the image below, the green square should be below the blue. Render order is red → green → blue so it just drew the blue on top of the green even though it’s further
Screenshot 2023-03-09 at 14.52.05

for some reason the PolyDataMapper2D also doesn’t render vtkConcentricCylinderSource . I’ve tried passing it first through the vtkTriangleFilter since it has cells with 4 points but that didn’t fix it.

Not sure why. Are the output point coordinates in world space?

I’ve figured out I can set the position to the woorld coordinates by calling setCoordinateSystemToWorld() on the Actor2D but the orientation doesn’t update. The axis dont point to the woorld coordinates x, y and z direction but stay the same relative to the screen.

How is the orientation being set?

Property2D doesn’t have edgeVisibility . Is there a way to add this. If it’s not too much work I could make a PR for this

Go for it :+1:

It also doesn’t seem like Actor2D ignores the depth buffer and writes everything on top of each other. This is not ideal for my case since I’d still want the arrows to be displayed correctly when they overlap. In the image below, the green square should be below the blue. Render order is red → green → blue so it just drew the blue on top of the green even though it’s further

That is indeed true. You can either order your axes in the order you want them rendered or use coincident topology resolution properties to push certain 2d props back in clip space.

Thanks for the response!

Not sure why. Are the output point coordinates in world space?

Just figured it out, was a bug on my end. I misconfigured a parameter making it too small to see.

How is the orientation being set?

like this:

const orientation = [1, 0, 0] // or [0, 1, 0] or [0, 0, 1]
const cylinder = vtkCylinderSource.newInstance();
cylinder.setDirection(orientation);
const cone = vtkConeSource.newInstance();
cone.setDirection(orientation);
const circle = vtkConcentricCylinderSource.newInstance();
circle.setDirection(orientation);

Go for it :+1:

Cool, I’ll look into it

That is indeed true. You can either order your axes in the order you want them rendered or use coincident topology resolution properties to push certain 2d props back in clip space.

Thansk for the tip. I’ve tried this using the 3D vtkMapper class but this doesn’t seem to affect vtkMapper2D. Is there a way to also do this for 2D mappers?

vtkMapper.setResolveCoincidentTopologyToPolygonOffset()

It could be that I’m not properly configuring setResolveCoincidentTopologyLineOffsetParameters, setResolveCoincidentTopologyPolygonOffsetFaces and setResolveCoincidentTopologyPointOffsetParameters but I’ve tried different values and didn’t see any changes

How is the orientation being set?

Figured it out. The issue was that I set the mapping to the world coordinates like this:

actor2D.getActualPositionCoordinate().setCoordinateSystemToWorld()

Which only takes care of the position. In stead I should do this:

const c = vtkCoordinate.newInstance();
c.setCoordinateSystemToWorld();
Mapper2D.setTransformCoordinate(c)

current result:

Hi @nomis6432,

Good job on getting it to work. Is there any chance you could contribute your widget into VTK.js ?

Thanks,
Julien.