VTK 9.4 debug build fails with MSVC due to "image size ... exceeds maximum allowable size"

Development
1

Debug build of VTK-9.4 fail on Windows if VTK_ENABLE_KITS is enabled, with this error:

CommonCore-objects.dir\Debug\CommonCore-objects.lib : fatal error LNK1248: image size (10098384C) exceeds maximum allowable size (FFFFFFFF) [C:\D\VTK-bin\Common\Core\CommonCore-objects.vcxproj]

This is a problem, because we can no longer build 3D Slicer in debug mode on Windows.

Full build command:

cmake c:\D\VTK -G "Visual Studio 17 2022" -T "v143" -DVTK_ENABLE_KITS:BOOL=ON
cmake --build . --config Debug -- /m

VTK 9.2 was built successfully with the same options. The issue seems to be that CommonCore is actually huge now. Size of VTK-bin\Common\Core\CommonCore.dir\Debug is 7.9GB in VTK-9.4, while it was 3.1GB in VTK-9.2.

The difference is about 230 extra files in VTK-bin\Common\Core\CommonCore.dir\Debug folder:

vtkAffineArrayInstantiate_<type>.obj
vtkCompositeArrayInstantiate_<type>.obj
vtkConstantArrayInstantiate_<type>.obj
vtkIndexedArrayInstantiate_<type>.obj
vtkStdFunctionArrayInstantiate_<type>.obj
vtkStructuredPointArrayInstantiate_<type>.obj
vtkTypedDataArrayInstantiate_<type>.obj
vtkAffine<type>Array.obj
vtkComposite<type>Array.obj
vtkConstant<type>Array.obj
vtkIndexed<type>Array.obj
vtkAffineImplicitBackendInstantiate_<type>.obj
vtkCompositeImplicitBackendInstantiate_<type>.obj
vtkConstantImplicitBackendInstantiate_<type>.obj
vtkIndexedImplicitBackendInstantiate_<type>.obj
vtkStructuredPointBackendInstantiate_<type>.obj
vtkDataArray<operation>.obj

I’ve tried to add -DVTK_DISPATCH_AFFINE_ARRAYS:BOOL=OFF -DVTK_DISPATCH_CONSTANT_ARRAYS:BOOL=OFF -DVTK_DISPATCH_STD_FUNCTION_ARRAYS:BOOL=OFF -DVTK_DISPATCH_STRUCTURED_POINT_ARRAYS:BOOL=OFF but it resulted in many build errors similar to this:

35>C:\D\S4D\VTK\Common\Core\vtkDataArray_GetTuples_ids.cxx(24,33): error C2039: 'DataArrayTupleRange': is not a member of 'vtk'
35>    C:\D\S4D\VTK\Common\Core\vtkAOSDataArrayTemplate.h(29,11):
35>    see declaration of 'vtk'
35>    C:\D\S4D\VTK\Common\Core\vtkDataArray_GetTuples_ids.cxx(24,33):
35>    the template instantiation context (the oldest one first) is
35>        C:\D\S4D\VTK\Common\Core\vtkDataArray_GetTuples_ids.cxx(66,5):
35>        see reference to function template instantiation 'void `anonymous-namespace'::GetTuplesFromListWorker::operator ()<vtkDataArray,vtkDataArray>(Array1T *,Array2T *) const' being compiled
35>        with
35>        [
35>            Array1T=vtkDataArray,
35>            Array2T=vtkDataArray
35>        ]

Would it be possible to move back implicit arrays into their own CommonImplicitArrays component as it was before?

If that was impractical, could someone provide CMake flags that allow building VTK without implicit arrays?

@spyridon97 @ben.boeckel

2

By default, all the implicit arrays are not dispatched with the exception of vtkStructuredPointsArray which is used internally in vtkImageData, vtkRectilinearGrid.

They can’t be deactivated from compiling since they are part of common core.

We moved it into common core, because it created a lot of extra includes and spagheti code, if this does not exist, use the other, and such. Plus all arrays should belong to the same module. I tried to create a CommonArrays module, but it did not work because of internal cyclic dependencies with vtkVariant which in turn depends on vtkInformationKey (or something like that, it’s been 2 years since i worked on this).

I personally see 2 potential solutions.

  1. Actually, implement the CommonArray module, but i don’t know if that will affect the kits compilation.
  2. Understand why windows does not work and Linux does (i hope). Is there something that we can pass to the compiler to reduce the size, or something that linux passed which reduces the size?
3

Hello,

Perhaps enabling some optimization in debug mode, by setting CMAKE_CXX_FLAGS_DEBUG to /O2 when configuring the build with CMake. Reference: /O options (Optimize code) | Microsoft Learn. Try to find one that reduces the executable size to just under the limit while minimizing impact on debugging capabilities or profiling accuracy.

best,

PC

4

A quick way to test your idea is to try RelWithDebInfo.

5

RelWithDebInfo and other optimized builds are not replacements for a Debug build, as they severely limit debugging (methods are removed, lines are skipped, code is not executed in order, variables are inaccessible, etc.).

By default, all the implicit arrays are not dispatched with the exception of vtkStructuredPointsArray which is used internally in vtkImageData, vtkRectilinearGrid.

I see that most of the VTK_DISPATCH_..._ARRAYS CMake variables are set to OFF, but I’m not sure if they are effective.

As I wrote above, when I built with default options, .obj files were created not just vtkStructuredPointArray (447MB of .obj files), but also for vtkAffineArray (413MB), vtkCompositeArray (692MB), vtkConstantArray (241MB), vtkIndexedArray (717MB), vtkStdFunctionArray (241MB), vtkTypedDataArray (233MB).

Considering their enormous size, would it be possible to add CMake options to remove instantiations of these rarely used types?

6

We will discuss how to resolve this issue internally, and let you know.

1 Like
7

That limit smells of a 32bit build. It looks like it should be, but could you please confirm that this is indeed a 64-bit build.

8

Yes, it is a 64-bit build. See the full build command in the first post.