Questions about VTK ray tracing roadmap - Anari, visRTX, OptiX

I recently built vtk with support for ray tracing - latest VTK : 9.6, ANARI SDK : 0.15, VisRTX : 0.13, NVIDIA OptiX : 8.1 on Windows and the system works well.

But there are several gaps between what IS supported in VTK and what COULD be supported since ANARI/VisRTX already has the support.

I’m particulalry interested in finding out if there are any designs available (or in work) about how these extended capabilities will be exposed through VTK. I’m even more intereted in how these would be exposed via the vtkPartitionedDataSetColleciton and associated vtkCompositePolyDataMapper and vtkCompositeDataDisplayAttributes - I use these heavily in my current work and they are somehwat at odds with the way VTK supports PBR mostly on the vtkActor? Idealy, I’d like ot be able to set ANY of these properties at the block level and have the mapper take care of getting the data to the right place….

Are there any plans to extend support for additional PBR properties? The tables below represnt my undestranding of the current support levels in each of the technolgies. I don’t really need all of these, but for sure the “transmission” poperty woudl be very useful.

If there aren’t any specific plans to enhance ghis capability, I will likely have a go at adding the transmission property myself - currently (based on very little reasearch) I would plan to do this by basically aping the way opacity is handled - and just extend the set of supported PBR properties? But if there is already a design, I’d rather align with that than just do it in isolation?

Scalar & Vector Parameters

Parameter vtkProperty VTK→ANARI Adapter ANARI KHR Spec VisRTX non-MDL VisRTX MDL
baseColor GetColor() :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
opacity GetOpacity() :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
metallic GetMetallic() :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
roughness GetRoughness() :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
ior GetBaseIOR() :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
specular GetSpecular() :white_check_mark: :white_check_mark: :white_check_mark:
specularColor GetSpecularColor() :white_check_mark: :white_check_mark: :white_check_mark:
clearcoat GetCoatStrength() :white_check_mark: :white_check_mark: :white_check_mark:
clearcoatRoughness GetCoatRoughness() :white_check_mark: :white_check_mark: :white_check_mark:
alphaMode Hardcoded "blend" :white_check_mark: (opaque/blend/mask) :white_check_mark: :white_check_mark:
alphaCutoff :cross_mark: :white_check_mark: (default 0.5) :white_check_mark: :white_check_mark:
transmission :cross_mark: :white_check_mark: :white_check_mark: :white_check_mark:
thickness :cross_mark: :white_check_mark: :white_check_mark:
attenuationDistance :cross_mark: :white_check_mark: :white_check_mark:
attenuationColor :cross_mark: :white_check_mark: :white_check_mark:
sheenColor :cross_mark: :white_check_mark: :white_check_mark:
sheenRoughness :cross_mark: :white_check_mark: :white_check_mark:
iridescence :cross_mark: :white_check_mark: :white_check_mark:
iridescenceIor :cross_mark: :white_check_mark: :white_check_mark:
iridescenceThickness :cross_mark: :white_check_mark: :white_check_mark:
anisotropy GetAnisotropy() :cross_mark: (not forwarded)
anisotropyRotation GetAnisotropyRotation() :cross_mark: (not forwarded)
coatColor GetCoatColor() :cross_mark: (not forwarded)
coatIOR GetCoatIOR() :cross_mark: (not forwarded)
coatNormalScale GetCoatNormalScale() :cross_mark: (not forwarded)
normalScale GetNormalScale() :cross_mark: (not forwarded)
occlusionStrength GetOcclusionStrength() :cross_mark: (not forwarded)
emissiveFactor GetEmissiveFactor() :cross_mark: (not forwarded)
edgeTint GetEdgeTint() :cross_mark: (not forwarded)

Texture Slots

Texture vtkProperty Slot VTK→ANARI ANARI KHR Spec VisRTX non-MDL VisRTX MDL
Base color / albedo "albedoTex" :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
ORM (O+R+M packed) "materialTex" :white_check_mark: (split to 3 samplers) :white_check_mark: (separate) :white_check_mark: :white_check_mark:
Normal "normalTex" :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
Emissive "emissiveTex" :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
Clearcoat normal "coatNormalTex" :warning: BUG (wrong param name) :white_check_mark: :white_check_mark:
Anisotropy "anisotropyTex" :cross_mark: (not forwarded)
Opacity (texture) :cross_mark: :white_check_mark: :white_check_mark: :white_check_mark:
Metallic (standalone) — (via ORM only) :white_check_mark: :white_check_mark: :white_check_mark:
Roughness (standalone) — (via ORM only) :white_check_mark: :white_check_mark: :white_check_mark:
Specular (texture) :cross_mark: :white_check_mark: :white_check_mark:
SpecularColor (texture) :cross_mark: :white_check_mark: :white_check_mark:
Clearcoat (texture) :cross_mark: :white_check_mark: :white_check_mark:
ClearcoatRoughness (tex) :cross_mark: :white_check_mark: :white_check_mark:
Transmission (texture) :cross_mark: :white_check_mark: :white_check_mark: :white_check_mark:
Thickness (texture) :cross_mark: :white_check_mark: :white_check_mark:
Sheen color (texture) :cross_mark: :white_check_mark: :white_check_mark:
Sheen roughness (texture) :cross_mark: :white_check_mark: :white_check_mark:
Iridescence (texture) :cross_mark: :white_check_mark: :white_check_mark:
IridescenceThickness (tex) :cross_mark: :white_check_mark: :white_check_mark:

Great work @scotsman60

Let me notify the interested people: @MathurinLemarie @lgivord @sankhesh @jaswantp @berk.geveci

1 Like

May also be of interest to @Michael

I figured I’d share my thoughts on what I’d really like to see in VTK related to PBR lighting propereties and whether or not VTK has a roadmap that includes defining PBR properties (scalar/vector/texture), and spatial transforms too, at the block level of composite data sets?

A related feature would be to associate shader replacements/compute shader access at the block level too. Currently these capabilities are only accessible via the Actor.

I do see that vtkCompositeDataDisplayAttributes supports a “Material” definition which makes me think that defining PBR lighting properties at the block level may be planned, however I can find no examples or other documentation on usage of this capability.

Here’s the way I currently understand the use of PBR lighting properties. Currently VTK puts PBR properties on the vtkProperty with the vtkProperty associated to the Actor. The pipelines need to be built so that each “object” that has a unique “rendering spec” has it’s own actor. For scenes that have many uniquely rendered objects we can end up with a LOT of actors, each with it’s own, mapper, renderer etc. and this can quickly bog down VTK rendering performance. Although there is some evidence that the WebGPU backend may not have the same issues with large Actor counts there are still significant problems wit this approach – not least the need to independently manage multiple geometry/mappers/renderer/actor pipelines

Models that use rich Part/Assembly hierarchies are easily implemented using the vtkPartitionedDataSetCollection/vtkDataAssembly/vtkCompositeDataDisplayAttributes classes. These classes work very well for CAD type models and at least some simulation systems. Control over the visibility, color, opacity, pickability and scalar displays of “Parts” is trivially easy. Render style (Shaded, shaded with edges, shaded with feature edges, wireframe etc.) are a little more complicated, but nothing too serious.

What I am really interested in is how I can efficiently, and performantly combine PBR materials and the composite data set objects. Ideally, I’d like to define the PBR lighting properties at the block level, via vtkCompositeDataDisplayAttributes, and avoid the requirement of splitting my multiblock data set into a multiplicity of Block/Mapper/Renderer/Actor pipelines which is the approach I use today.

One additional feature that I’m interested in, not specifically related to lighting, but still impacted by the composite dataset single Actor pattern is block transforms. The most performant way to apply time series translations/rotations per block is to define the transforms on the Actors – this enables the transform to be done on the GPU which is orders of magnitude faster than using a pipeline based transform.