Delete() doesn't release memory?

Support
1

Hi VTK experts,

I have a number of image stacks of which I need to crop a VOI out after I assemble them. I used vtkImageAppend* volumeAppend = vtkImageAppend::New() to assemble these stacks forming one single volume. Then I used vtkExtractVOI* voi = vtkExtractVOI::New() to get the VOI I wanted. Once I got the VOI, volumeAppend became no use anymore and I intended to release the memory held by it. However, calling volumeAppend->Delete() doesn’t seem to release the memory according to Visual Studio’s diagnostic tool. Moreover, I called volumeAppend->GetOutput()->GetActualMemorySize() and the return value was the same as it was before volumeAppend->Delete(). Eventually I found that I could use volumeAppend->ReleaseDataFlagOn() to free up the memory.

But it left me confused: Why Delete() doesn’t seem to do the job? Could someone enlighten me a little about some details behind it?

Thanks so much!

PS. To provide a little more context, I used vtkTIFFReader to read in those stacks, and called volumeAppend->AddInputConnection() to take vtkTIFFReader’s GetOutputPort() to assemble the stacks.

2

Hi @MKRangers

Please share actual code, it is hard to follow in its current form.

In any case, the data handled by the filter will pass through the pipeline and will only be released once it is not needed by anyone using reference counting.

But, the following is NOT valid code:

vtkImageAppend* volumeAppend = vtkImageAppend::New()
volumeAppend->Delete()
volumeAppend->AnyMethod()

After the delete, volumeAppend has been deleted, even if some of the data it handled have been passed to another class.

3

Note that Delete would be better named ReleaseReference. And yes, calling methods through a pointer that you’ve released the reference of is “getting lucky” when it works (you’re firmly in UB territory).

2 Likes
4

Hi @mwestphal

Thank you for your quick reply. Indeed, calling a method after Delete() and still got a value returned is another puzzling thing for me. Seeing the memory still being held after Delete() is what prompted me to call GetActualMemorySize() because I suspected Delete() wasn’t doing what I thought it was doing. The following is my actual code:

Note, ArboretaVolumeSP is a std::shared_ptr of ArboretaVolume class in which a vtkImageAppend* member mVolumeAppender resides. The method ArboretaVolumeProcessor::teraflyVolumeAssemble() is where stack assembling happens:

ArboretaVolumeSP ArboretaVolumeProcessor::teraflyVolumeAssemble(const vector<vector<vector<string>>>& teraflyStackNamesRange)
{
	ArboretaVolumeSP outputArbVolume(new ArboretaVolume);
	
	outputArbVolume->mVolumeAppender = vtkImageAppend::New();
	outputArbVolume->mVolumeAppender->SetAppendAxis(1);
	for (vector<vector<vector<string>>>::const_iterator yDirIt = teraflyStackNamesRange.cbegin(); yDirIt != teraflyStackNamesRange.cend(); ++yDirIt)
	{
		vtkImageAppend* xzAppend = vtkImageAppend::New();
		xzAppend->SetAppendAxis(0);
		for (vector<vector<string>>::const_iterator xDirIt = yDirIt->cbegin(); xDirIt != yDirIt->cend(); ++xDirIt)
		{
			vtkImageAppend* zAppend = vtkImageAppend::New();
			zAppend->SetAppendAxis(2);
			for (vector<string>::const_iterator zFilenameIt = xDirIt->cbegin(); zFilenameIt != xDirIt->cend(); ++zFilenameIt)
			{
				vtkTIFFReader* inputReader = vtkTIFFReader::New();
				inputReader->SetFileName((*zFilenameIt).c_str());
				inputReader->SetDataSpacing(1, 1, 1);
				inputReader->SpacingSpecifiedFlagOn();
				inputReader->Update();

				zAppend->AddInputConnection(inputReader->GetOutputPort());
				inputReader->ReleaseDataFlagOn(); // No memory reduction observed without it.
				inputReader->Update();
				inputReader->Delete(); 
			}
			zAppend->Update();

			xzAppend->AddInputConnection(zAppend->GetOutputPort());
			zAppend->ReleaseDataFlagOn();
			zAppend->Update();
			zAppend->Delete();
		}
		xzAppend->Update();

		outputArbVolume->mVolumeAppender->AddInputConnection(xzAppend->GetOutputPort());
		xzAppend->ReleaseDataFlagOn();
		xzAppend->Update();
		xzAppend->Delete();
	}
	outputArbVolume->mVolumeAppender->Update();

	return outputArbVolume;
}

As you can see, I called ReleaseDataFlagOn() in every layer of for loop in order to see memory usage reduction. There was no such phenomenon observed if only Delete() was called without ReleaseDataFlagOn().

Here’s the caller of the method above:

ArboretaVolumeSP wholeBrainVolume = ArboretaVolumeSP(new ArboretaVolume);
if (mFormat == volumeFileFormat::tif || mFormat == volumeFileFormat::tiff)
{
	wholeBrainVolume->mVolumeAppender = vtkImageAppend::New();

	wholeBrainVolume = this->mTeraflyVolumeProcessor.teraflyVolumeAssemble(lowestResFilenameTree);
    cout << wholeBrainVolume->mVolumeAppender->GetOutput()->GetActualMemorySize() << endl; // returns value xyz

	wholeBrainVolume->rescaleDynamicRange();
	wholeBrainVolume->mVolumeAppender->ReleaseDataFlagOn();
	wholeBrainVolume->mVolumeAppender->Update();
	wholeBrainVolume->mVolumeAppender->Delete();	
	cout << wholeBrainVolume->mVolumeAppender->GetOutput()->GetActualMemorySize() << endl; // still returns value xyz!?
}

In the above section, rescaleDynamicRange() is a method I implemented to convert the 16 bit volume to 8 bit by using vtkImageShiftScale with SetInputConnection(mVolumeAppender->GetOutputPort()). Again, I needed to call ReleaseDataFlagOn() to see memory use reduction, and even after I called Delete(), I still got to see same value returned by GetActualMemorySize() before and after Delete().

5

If you run under valgrind or some other memory checker, does it complain about use-after-free? Again, I think you’re just getting lucky and code is using garbage memory that happens to still have useful values in it.

1 Like
6

Hi @MKRangers,

As Ben mentioned, Delete() simply releases a reference to the object, it doesn’t always delete the object. Modern VTK programming style prefers smart pointers (vtkNew<> and vtkSmartPointer<>) to automatically release the reference, rather than using Delete() to release the reference.

Also, append->AddInputConnection(reader->GetOutputPort()) causes append to reference reader and, for memory efficiency, inter-object references should be avoided unless they are needed. So you can re-write your code like this:

  {
    // vtkNew<> is like std::unique_ptr<>
    vtkNew<vtkTIFFReader> inputReader;
    inputReader->SetFileName((*zFilenameIt).c_str());
    inputReader->SetDataSpacing(1, 1, 1);
    inputReader->SpacingSpecifiedFlagOn();
    inputReader->Update();

    // get the reader's output (we don't want to create a connection)
    zAppend->AddInputData(inputReader->GetOutput());
  }

But the zAppend is probably not needed at all, because one vtkTIFFReader is able to read a stack of images and generate a volume all by itself. Just use reader->SetFileNames(vtkStringArray*) instead of reader->SetFileName(const char*).

1 Like
7

Does wholeBrainVolume add additional references to vtkImageAppend inside the setter method?

8

Even if it does today, there’s no guarantee of that. You should not use pointers to which you do not have a definitive reference to (i.e., post-Delete()).

9

I totally agree. Dereferencing an object and then trying to invoke its methods is asking for trouble. I was encouraging @MKRangers to investigate the reason for the anomaly in this case.

10

Hi @dgobbi ,

Thank you for your advice. I checked VTK document, vtkTIFFReader->SetFileNames(vtkStringArray*) can only take image slices, but each file I’m working with is a 3D stack already, so I guess I still need to use vtkImageAppend.

To be honest, I’m a VTK newbie here and I don’t quite understand the difference between the pairs of AddInputConnection() - GetOutputPort() and AddInputData() - GetOutput(). Could you please explain a bit?

I did a little experiment comparing the memory usage between these 2 methods, no difference found by the end of teraflyVolumeAssemble(). However, when the method returns back to its caller, using AddInputConnection() - GetOutputPort() shows substantially less memory occupied than AddInputData() - GetOutput() (1 GB vs 1.6 GB). After rendering it, it becomes 1.5 GB and 2.1 GB, respectively.

11

Hi @ben.boeckel and @toddy,

Actually, wholeBrainVolume doesn’t add additional reference. But the fact that Delete() only releases the reference makes sense to me now. As I mentioned in 1 of my previous replies, I used a vtkImageShiftScale to SetInputConnection() from wholeBrainVolume’s vtkImageAppend member. This is where the additional reference takes place, and that’s why I got the anomaly. I did get read access violation if I skipped this step.

But again, the reason I called some method after Delete() is because I suspected Delete() didn’t destroy the object and I wanted to see if I still could get something after I called it. Now I got my answer, thank you very much!

Now another VTK newbie question for you if you don’t mind. When I checked the reference count, vtkImageAppend->GetReferenceCount() already returned 2 when it was created while vtkImageAppend->GetOutput()->GetReferenceCount() returned 1. What’s the difference and what is that?

Thank you again!

12

To understand the difference between SetInputConnection() and SetInputData(), it’s necessary to understand the VTK pipeline executive. Every “Algorithm” object in VTK is paired with an “Executive” object. When you call algorithm->Update(), it simply does algorithm->Executive->Update(), and then the Executive manages the actual execution of the Algorithm. Confusing, right? The idea is that the Executive is in control of the Algorithm, even though the Executive is mostly invisible to the user.

One important thing that the Executive does, is that it manages connections between Algorithms. When you do this

   append->AddInputConnection(reader->GetOutputPort());

the result is that the Executive of “append” holds not only a reference to “append” itself, but also a reference to “reader”. And if you call append->Update(), the Executive will automatically update “reader” as well.

In comparison, if you do this

append->AddInputData(reader->GetOutput());

then the Executive doesn’t know anything about “reader”, all it gets is the reader’s output (which must already be up-to-date before “append” is updated). More details can be found on the Kitware blog.

In general, if you only plan to update an algorithm once, you should call Update() and then GetOutput(), but if you plan to update an algorithm over and over as part of a pipeline, then you should call GetOutputConnection() and let the executive handle the updates.

I don’t know why you are seeing more memory usage with GetOutput() than with GetOutputPort(), it doesn’t make sense to me. I’d have to see the exact code that you used to test the memory usage.

With respect to ReleaseDataFlag, it just controls whether the algorithm releases its reference to the data object, so it doesn’t guarantee that the memory is released. The data is not “owned” by just one algorithm, since it exists both as the output of one algorithm, and as the input of other algorithms, all of which hold references to it via their executives.

(If it wasn’t already obvious, the VTK pipeline architecture can be crazy complicated).

1 Like
13

The class documentation VTK: vtkImageAlgorithm Class Reference shows that vtkImageAppend->GetOutput() returns vtkImageData; a completely different object.

14

I understand. I guess my real question is why vtkImageAppend->GetReferenceCount() returns 2 even there is obviously only one using count? It’s like the first instanced object already takes 2.

15

What version of VTK are you using? I did a bit of experimenting with the latest release, and found that the ‘executive’ object (which I mentioned in my post above) is responsible for the additional reference count. In Python:

>>> import vtkmodules
>>> vtkmodules.__version__
'9.2.2'
>>> from vtkmodules.vtkFiltersCore import vtkImageAppend
>>> a = vtkImageAppend()
>>> a.GetReferenceCount()
1
>>> # when the filter creates an executive, the reference count increases
>>> e = a.GetExecutive()
>>> a.GetReferenceCount()
2
>>> # likewise, the executive has a reference count of 2
>>> e.GetReferenceCount()
2
>>> # add an observer so we know when the filter is deleted
>>> a.AddObserver('DeleteEvent', print)
>>> # release our references to both objects, and the filter is deallocated
>>> del a
>>> del e
None DeleteEvent

So for VTK 9.2.2, at least, the extra reference count doesn’t appear until something causes the algorithm’s ‘executive’ to be created.

If I do the same experiment, but don’t hold the reference to the executive after calling GetExecutive(), then deleting the reference to the algorithm successfully deallocates the algorithm, even though it had a reference count of two. This works because VTK’s garbage collection can handle reference cycles.

>>> from vtkmodules.vtkFiltersCore import vtkImageAppend
>>> a = vtkImageAppend()
>>> a.GetExecutive()
<vtkmodules.vtkCommonExecutionModel.vtkCompositeDataPipeline(0x7ff9b3425f20) at 0x7ff9a84b3588>
>>> a.GetReferenceCount()
2
>>> # add an observer
>>> a.AddObserver('DeleteEvent', print)
>>> # release our reference
>>> del a
None DeleteEvent

Edit: I checked, and VTK has behaved this way all the way back to VTK 5, so I guess the version of VTK that you are using is a moot point.

An algorithm has a reference count of 1 when first created, and the reference count increases to 2 as soon as the executive is created. The executive is created when you call SetInputData(), SetInputConnection(), GetOutput(), GetOutputPort(), Update(), GetExecutive(), or any of the many other algorithm methods that require an executive.

1 Like
16

My VTK version is 9.1. Anyway, I know it’s been late, but I shall at least reply to show my appreciation of your time and informative posts. Once again, thank you very much. I learned a lot here.