# Image-based lighting (e.g., PBR) vs shadows
**URL:** https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835
**Category:** Development
**Created:** [December 4, 2023, 5:45pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835 "2023-12-04T17:45:11Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Sean\_Curtis](https://discourse.vtk.org/u/Sean_Curtis)
#### Post date: [December 4, 2023, 5:45pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/1 "2023-12-04T17:45:11Z")
I’ve recently started using the PBR rendering capabilities of VTK and I’ve hit a surprise, specifically, I seem unable to combine shadow maps with environment map lighting. In some ways, the two features seem orthogonal.
For example, I create a scene with objects that have both phong interpolation as well as pbr interpolation. Both get _illuminated_ by lights I put in the scene. However, if I have a shadow pass, only the phong-interpolated actors will receive shadows (with some very erratic behavior consisting of whether the PBR objects even cast shadows).
This behavior is independent of whether the render’s setting for image-based lighting is on or off (the only difference is that when it is off, the PBR materials reflect a black universe).
I would’ve expected that just as the lighting combines with the image based illumination, the shadows would as well. Have I missed something? Is this by design or does it constitute a defect?
---
### Author:  [@Sean\_Curtis](https://discourse.vtk.org/u/Sean_Curtis)
#### Post date: [December 4, 2023, 10:45pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/2 "2023-12-04T22:45:25Z")
I think my original problem statement is some combination of incomplete and/or incorrect. I started with the Shadows example and added in an environment map (well, I tried two). I find that whether or not I get shadows depends on the environment map. I’ll have to poke deeper. I’m currently operating under the hypothesis that it depends on either the range of the environment map. I’ll have to look at the exponents of the two images to see. Stay tuned.
---
### Author:  [@Sean\_Curtis](https://discourse.vtk.org/u/Sean_Curtis)
#### Post date: [December 5, 2023, 10:16pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/3 "2023-12-05T22:16:29Z")
Yep. I’ve spent an instructional day. For anyone else who trips over this:
1. The efficacy of the shadows will be affected by the _relative_ intensity of the lights with the radiant energy in the environment map. In my case, it was a massive ratio.
2. Tonemapping is your friend in this regard. As is documented in the [vtkToneMappingPass](https://vtk.org/doc/nightly/html/classvtkToneMappingPass.html#details):
> Advanced tone mapping like GenericFilmic, Reinhard or Exponential can be useful when several lights are added to the renderer.
While I tinkered with adding a “scale filter” to reduce the intensity of the environment map, I found simply reducing the exposure of the GenericFilmic tone map seems to make things sane again. This works even with the huge disparity between light intensity and map intensity.
---
### Author:  [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [December 11, 2023, 3:47pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/4 "2023-12-11T15:47:39Z")
FYI @lgivord @meak
---
### Author:  [@Sean\_Curtis](https://discourse.vtk.org/u/Sean_Curtis)
#### Post date: [December 15, 2023, 6:31pm UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/5 "2023-12-15T18:31:01Z")
After the initial confusion, followed by the subsequent realization, I continued digging and found actual defects (one might be up for debate, but I’m presenting it as a defect):
# Sometimes, actors with PBR interpolation will _not_ receive shadows.
`vtkOpenGLPolyDatamapper.cxx` and `vtkShadowMappass.cxx` work in conjunction to define the `radiance` value for when the interpolation is equal to `VTK_PBR`. They do this by having the mapper define the radiance values and the shadow pass subsequently modifies the radiance definition to include the “shadow factor”. For example:
- When the light is a headlight, [`vtkOpenGLPolyDataMapper` defines radiance as](https://github.com/Kitware/VTK/blob/master/Rendering/OpenGL2/vtkOpenGLPolyDataMapper.cxx#L1132):
**`radiance = lightColor0;`.**
In turn [`vtkShadowMapPass` substitues that string](https://github.com/Kitware/VTK/blob/master/Rendering/OpenGL2/vtkShadowMapPass.cxx#L342-L343) with the string
**`radiance = factor0.r * lightColor0;`**
It’s a _very explicit_ search and replace.
- When the light is a light kit, [`vtkOpenGLPolyDataMapper` likewise defines the radiance](https://github.com/Kitware/VTK/blob/master/Rendering/OpenGL2/vtkOpenGLPolyDataMapper.cxx#L1175-L1176) as **`radiance = lightColor#;`**. This string will likewise get replaced by the substitution linked to above.
- However, if the light complexity is “case 3” (tagged as “positional”), [`vtkOpenGLPolyDatamapper` defines radiance differently](https://github.com/Kitware/VTK/blob/master/Rendering/OpenGL2/vtkOpenGLPolyDataMapper.cxx#L1295-L1296):
**`radiance = lightColor# * attenuation;`**
There [is no substitution in `vtkShadowMapPass` to match this string](https://github.com/Kitware/VTK/blob/master/Rendering/OpenGL2/vtkShadowMapPass.cxx#L322-L346). Therefore, in this case, actors with PBR interpolation do not receive shadows.
The simplest solution would be to split the “case 3” generated shader code into two statements: **`radiance = lightColor#;`** and **`radiance *= attenuation;`**. This would allow the shadow map pass to still successfully find and replace the string using its current substitution logic. Admittedly, this still keeps the brittleness in place, where magical strings generated by one file have to match magical strings expected in another file. But that ship has probably sailed.
# “Steepness” constant in the exponential shadow map.
VTK implements [Exponential Shadow Maps](https://jankautz.com/publications/esm_gi08.pdf). Part of that is a constant that tunes the exponential function to more closely approximate the step function that defines the binary state of occlusion. The tuning is handled via a constant named “c” in the paper (and `depthC` in VTK’s implementation). As the paper says, if `c` is too low, “we will observe light leaking artifacts”. The paper goes on to discuss the fact that as this constant goes to infinity the approximation mathematically converges to the step function. However, it acknowledges that, due to numerical precision issues, there is a limit to how large it can become before other artifacts are generated. It states: “We empirically determined an optimal value of c = 80 for 32-bit floating point numbers.”
VTK has hard-coded c = 11 for some mysterious reason. In my applications, this led to clear “light leaking artifacts”. This is manifest by casting shadows onto a ground plane from above with numerous objects floating between light and ground at various heights. For a fixed bounding box on the scene, as light occluders get closer to the ground plane, the shadows get paler and paler (eventually disappearing completely). Visually, it looks less like shadows are being cast and more like the depth image is being projected by the light. This is because c = 11 relaxes the exponential function _significantly_ (see figure 2(a) in the linked paper). So, you can get clean, dark shadows from occluders near the light, but not those far from the light.
Perhaps the default value should be revisited? Failing that, making it settable by the user would be a boon. We’ve patched this to c = 80 in our fork and are considering adding a shadow map member that we can set.
---
### Author:  [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [December 18, 2023, 8:53am UTC](https://discourse.vtk.org/t/image-based-lighting-e-g-pbr-vs-shadows/12835/6 "2023-12-18T08:53:27Z")
Thanks for your deep research @Sean_Curtis
I agree that the vtkShadowPass should be improved and we welcome any such improvements! Feel free to open a MR with proposed changes.
Note: Looks like you are using the github mirror, please note contributions needs to be made in [https://gitlab.kitware.com](https://gitlab.kitware.com)
Best,