I need to volume render datasets with a wide range of volume sizes: 5-120GBs is common and 120GB+ are rare but still happen. This will mostly be for non-interactive scenarios (predefined animations or stills), so I can afford to wait for a high-quality result. My goal is to volume render at the highest quality possible with respect to volume size, available system memory, and available VRAM. These volumes are stored in 16-bit, grayscale TIFs (one per slice). I’ve been running into multiple questions and problems and was hoping someone could clarify these things for me. For my testing, I’m working on a 2017 iMac (64GB of RAM, Radeon Pro 575 4GB VRAM), macOS 10.14.4 and VTK 8.1.0.
I’m working in a project that uses C++14, so I started by modifying the GPURenderDemo.cxx example, swapping out the image reading sections with a call to vtkTIFFReader. Based on this test file, I also added a vtkMemoryLimitImageDataStreamer between the resampler and the volume mapper. This all works and I can render small volumes quite easily:
// Load the volume data
auto volReader = vtkSmartPointer<vtkTIFFReader>::New();
volReader->SetFileNames(slicePaths);
volReader->SetOrientationType(1);
volReader->ReleaseDataFlagOn();
// Create the renderer, render window and interactor
auto colors = vtkSmartPointer<vtkNamedColors>::New();
auto renderer = vtkSmartPointer<vtkRenderer>::New();
auto renWin = vtkSmartPointer<vtkRenderWindow>::New();
renWin->AddRenderer(renderer);
// Connect it all. Note that funny arithematic on the
// SetDesiredUpdateRate - the vtkRenderWindow divides it
// allocated time across all renderers, and the renderer
// divides it time across all props. If clip is
// true then there are two props
auto iren = vtkSmartPointer<vtkRenderWindowInteractor>::New();
iren->SetRenderWindow(renWin);
iren->SetDesiredUpdateRate(frameRate / (1 + clip));
iren->GetInteractorStyle()->SetDefaultRenderer(renderer);
auto resample = vtkSmartPointer<vtkImageResample>::New();
if (scaleFactor < 1.0) {
resample->SetInputConnection(volReader->GetOutputPort());
resample->SetAxisMagnificationFactor(0, scaleFactor);
resample->SetAxisMagnificationFactor(1, scaleFactor);
resample->SetAxisMagnificationFactor(2, scaleFactor);
}
// Volume Streamer
// memLimitBytes is precalculated or user-provided
auto streamer = vtkSmartPointer<vtkMemoryLimitImageDataStreamer>::New();
streamer->SetMemoryLimit(memLimitBytes / 1024);
if (scaleFactor < 1.0) {
streamer->SetInputConnection(resample->GetOutputPort());
} else {
streamer->SetInputConnection(volReader->GetOutputPort());
}
streamer->UpdateWholeExtent();
// Create our volume and mapper
auto volume = vtkSmartPointer<vtkVolume>::New();
auto mapper = vtkSmartPointer<vtkSmartVolumeMapper>::New();
mapper->SetInputConnection(streamer->GetOutputPort());
// Set the sample distance on the ray to be 1/2 the average spacing
double spacing[3];
if (scaleFactor < 1.0) {
resample->GetOutput()->GetSpacing(spacing);
} else {
volReader->GetOutput()->GetSpacing(spacing);
}
mapper->SetSampleDistance((spacing[0] + spacing[1] + spacing[2]) / 6.0);
// Rest of example code continues
...
My first problem is that my program uses more than the memory limit set on the streamer. As an example, if I load a 17GB dataset, set the scale factor to 1.0, and set the memory limit to 4GB, my program still uses ~18GBs of RAM. I know that the streamer is doing something, because if I set the scale factor to 0.6, the program only uses 4-7GBs of RAM (it varies as the volume loads). My working theory is that this is because the volume mapper must still hold all of the volume in memory and therefore only the reader and resampler pipeline are memory limited. This would make my total potential memory usage be the streamer memory limit plus the resampled volume size. Can someone confirm that this is the case? If so, is there a better way to limit the total RAM usage of my program?
My next problem is with the vtkSmartMapper GPU Ray Caster. I consistently get OpenGL Out of Memory Errors when I don’t manually scale the volume to fit within VRAM. My confusion here is that I have seen hints in other posts that seem to indicate that the GPU Ray Caster will do some sort of interpolation and downscaling in order to fit the volume in VRAM. Is this correct or have I misread? Is there a way to make the mapper fallback to the CPU renderer if the volume won’t fit in VRAM? This also wouldn’t be too big of an issue if I could programmatically determine how much VRAM is available and set the scale factor accordingly. However, I can’t seem to find the correct way to use vtkGPUInfoList for this purpose.
My final problem is with the CPU Ray Caster. When I use it, my first frame renders at full resolution, but later frames render at low resolution. This is very similar to this other thread, but sort of the opposite effect. This happens regardless of the volume size:
