in an implementation of a viewing application which displays a slice of a medical image in 3D space (and using the vtkInteractorStyleTrackballCamera for rotating around its center in 3D), I have recently changed my integration of the rendering part into my application (now circumventing the vtkInteractorStyle and placing the camera manually on purpose).
For some reason (maybe related to my change above, not sure though) which I don’t know yet I have noticed that application turned slower: Besides the underlying render backend with the vtkOpenGLPolyDataMapper [requiring 0.4 ms] also UpdateWholeExtent() [requiring 22 ms] is called on every Render() call, even though the texture / resliced slice has not changed. In my earlier implementation, UpdateWholeExtent() was not called on every Render() call, leading to almost zero latency.
The source code suggests that in my slower implementation
…is returning true on every render call, since it keeps setting ResliceNeedUpdate to 1 (triggering a next UpdateWholeExtent()). What can be the reason for that behavior and how can I return to my old integration in which apparently this was not the case?
The refresh mechanism relies on “modified” time stamps (mtime). I believe the mapper consider them to decide whether it needs to refresh or not. You might want to check the GetNeedToRebuild... functions in the mapper.
thanks for the hints - unfortunately I have not found any GetNeedToRebuild... function in vtkImageResliceMapper. Based on your suggestion, I have inspected the following:
The “Update” function of vtkImageResliceMapper: When input, render size etc. does not change (which is the case for a rotating camera), no update is executed and therefore Modified() is not called (which would change mtime).
The “Render” function of vtkImageResliceMapper. As mentioned above,ResliceNeedUpdate is always true, therefore, each Render call Updates reslicing information and generates a new texture for rendering (which is not necessary for a camera rotating about the slice).
The “ProcessRequest” function of vtkImageResliceMapper, which is also invoked on every vtkRenderWindow::Render() call. Since the relevant request is asking for data (since REQUEST_DATA is set to true), as mentioned in my earlier e-mail, ResliceNeedUpdate is set to true in every invocation such that in the next Render call, the texture to be rendered is regenerated.
Since the mechanism in (3.) has been implemented by David Gobbi 12 years ago and has not changed since then, I would still think that the request having REQUEST_DATA set to true on every call is wrong.
Maybe @dgobbi has an idea what can cause REQUEST_DATA to be set for every render call and how to avoid this?
Apologies, indeed my question is related to a WASM implementation, but I guess the underlying problem is more related to the “support” group (C++). Any hint how to change the category? I was just searching for doing it without success, it seems that I cannot do it myself.
The “Update(int port)” method in vtkImageResliceMapper is the method that makes most decisions about whether an execution occurs. It does this by modifying the time stamp.
One big difference between using InteractorStyle vs. changing the camera yourself, is that the InteractorStyle calls SetDesiredUpdateRate() on the RenderWindow (to decrease quality and increase render speed during interaction)
The vtkImageResliceMapper is designed to automatically adjust quality when the UpdateRate is set, i.e. when used with an InteractorStyle (some of the code in “Update(int port)” is for changes in image quality)
It is possible that if you call AutoAdjustImageQualityOff() and ResampleToScreenPixelsOff(), the speed might be the same as before, when you used the InteractorStyle.
thanks for the hints! I must admit that I am still lost, even after having thought about your suggestions:
For debug purposes I have implemented an own Modified() function in vtkImageResliceMapper that prints log message and calls the base method. Debug output shows that Modified() gets called at the beginning, when the reslicing pipeline is set up, images are connected etc., as expected.
When rotating the camera around the center of the shown image, no Modified() call is logged, which is also expected, since the output of the reslice mapper (i.e. the generated texture) does not change.
However, during interaction, the demand-driven pipeline creates a new DataRequest including the REQUEST_DATA flag, which makes the vtkImageResliceMapper set ResliceNeedUpdate to 1 every time (for triggering another processing round in the next rendering call), even when not being Modified() in between.
Is there any way to change/fix this behavior? I am a bit puzzled since I am pretty sure that an earlier version (which I cannot easily rebuild) was working as expected (no UpdateWholeExtent() was called every time, only the texture has been correctly mapped to rotated 3D space).
Any hints, also where/how to continue debugging are still welcome.
For example it would be great to understand how the demand driven pipeline can detect that the vtkImageResliceMapper has not been Modified() and (in that case) skip the REQUEST_DATA request (which I guess should be the case not only for vtkImageResliceMapper, but for all algorithms)…
To dig deeper in your debugging, search vtk(Streaming)DemandDrivenPipeline.cxx for “NeedToExecuteData”. The pipeline is only supposed to forward REQUEST_DATA to the mapper when:
the mapper is modified
the mapper’s input data is modified
the new UpdateExtent is larger than the current data extent
Thanks, David, for pointing me to this function, indeed I have missed it!
It finally turns out that after each change in camera position, the mapper is modified!
More precisely, it’s modified timestamp (printed, when streaming the mapper to std::cout) is the one which corresponds to the last Modified() call to vtkOpenGLCamera while rotating it around the scene.
Isn’t this strange? In my understanding, rotating the camera should not modify the mapper, since the rendered texture should remain the same. And how would the camera’s modified timestamp propagate to the modified time of vtkImageResliceMapper? Again, I have made sure that the mapper’s Modified() function is not called.
However, I think I am close to fixing the issue - maybe with 1 or 2 more hints from you or someone else
If the ResampleToScreenPixels flag for the mapper is on, then rotation requires re-execution, since this flag causes the texture to be oriented with the screen, not with the data. Try ResampleToScreenPixelsOff().
Specifically, see the code in vtkImageResliceMapper::GetMTime() where it checks the camera MTime if ResampleToScreenPixels is on.
Getting back to my first reply on this thread, when you use an InteractorStyle, the “ResampleToScreenPixels” is automatically disabled during interaction (in order to increase the rendering speed).
That’s it! Awesome! Thanks also for the explanations, this is clear now. Problem solved I am not sure why my earlier version was already running in that mode, but whatever, this is a good day today.
I’ve tried to explain that. Your earlier version used an InteractorStyle, which calls SetDesiredUpdateRate() during interaction, and this causes the AllocatedRenderTime to change. In vtkImageResliceMapper there is a line that says
I am not sure why my earlier version was already running in that mode, but whatever, this is a good day today.