Legacy PolyData File Compatibility

Support
1

Hi there!

I’m trying to use VTK’s Python wrapping to resample an ITK mesh stored in a legacy .vtk file with VTK Data Format 2.0. It appears that vtkPolyDataReader is able to successfully import from this format, but writes back out with Data Format 5.1, which supports reading in ITK mesh objects from Data Format 2.0 with itk.meshread.

PolyData point elements expressed in the output file differ from the documented format for 4.2 and below with both different column arrangements and a different number of polygons. Based on Issue 17989 it seems that documentation for Data Format 5.1 is not available, so it’s difficult for me to verify whether the output file is correct or contains errors.

First, is the writer correctly writing out the PolyData object for .vtk format 5.1? What are some troubleshooting actions I can take here?

Second, is there a better way to write out PolyData for later use in ITK, whether in .vtk format 2.0, .vtp, or another format? It doesn’t look like this is covered under ITKVtkGlue yet.

Thank you!

============================================

Input:

# vtk DataFile Version 2.0
File written by itkPolyDataMeshIO
ASCII
DATASET POLYDATA
POINTS 634182 float
4.0343256 0.5738715 0.85776365
4.0411196 0.5772415 0.8592521
…
POLYGONS 1268360 5073440
3 0 4 6
3 0 6 3
3 0 1 5
3 0 5 4
…

Output:

# vtk DataFile Version 5.1
vtk output
ASCII
DATASET POLYDATA
POINTS 634182 float
4.02881 0.465097 0.673449 4.02881 0.465097 0.673449 4.02881 0.465097 0.673449
4.02881 0.465097 0.673449 4.02881 0.465097 0.673449 4.02881 0.465097 0.673449
…
POLYGONS 1268361 3805080
OFFSETS vtktypeint64
0 3 6 9 12 15 18 21 24
27 30 33 36 39 42 45 48 51
…

Relevant code is as follows:

    template_reader = vtk.vtkPolyDataReader()
    template_reader.SetFileName('input.vtk')
    template_reader.Update()

    test_writer = vtk.vtkPolyDataWriter()
    test_writer.SetFileName('output.vtk')
    test_writer.SetInputConnection(template_reader.GetOutput())
    test_writer.Update()
1 Like
2

VTK master has a recent change so that you can write out either VTK 4.2 or 5.1 formats:
https://discourse.vtk.org/t/can-we-write-out-the-old-vtk-4-2-file-format-with-vtk-9/5066

2 Likes
3

Perfect, I’ll rebuild against master and take a look. Thanks!

4

We have upgraded to VTK9 in Slicer and with that came this .vtk file format change. It caused so many and so serious problems in just two weeks (previous Slicer version cannot read .vtk files saved with current version of Slicer; a clinical workflow that has been working flawlessly for years suddenly broke down because CARTO electrophysiology guidance system uses .vtk files and of course did not recognize the new file format, etc.) that we will hardcode usage of 4.2 file format in Slicer.

Simply giving more time to developers to upgrade their .vtk readers will not work, because all those software that use .vtk file format but not the VTK library reimplement their own reader/writer. They have no other choice because there is no small standalone library for .vtk file IO and entire VTK is too big and complex as a dependency just to read a mesh file format.

At this point I think it impossible to make such fundamental changes to the .vtk file format so abruptly. VTK should revert to writing 4.2 format by default.

I see two options moving forward:

  • Option A: When writing 5.1 file format, use a different file extension, such as .vtk5 (similarly to what hdf did) to make it very clear that the file does not work with software that is designed for .vtk files.
  • Option B: Create a small standalone library in C++ (and provide Python-wrapped or native Python implementation on PyPI) that reads/writes both the old and new .vtk files. The library (and VTK) should keep writing .vtk files in 4.2 format for 5-10 years (when the library gets widely adopted for reading/writing .vtk files) and then you can flip the switch and start writing 5.1 file version by default.
3 Likes
5

I 100% agree with @lassoan, this backwards compatibility break needs to be reverted ASAP.

I can absolutely attest that CARTO is not the only system in routine clinical use that could stop working for the same reason (I didn’t write that system but I know it exists). For those who don’t know EP means putting electric wires in someone’s heart to burn tissue so we’re not joking around here. I’ll bet some neurosurgery workflows are also vulnerable to this.

One could correctly argue that such critical systems shouldn’t have these vulnerabilities, but in reality they do.

6

IMO in the short term we should just admit that a mistake was made and by default read/write version 4.2. We can also output a “deprecation” message, and follow the usual VTK timeline for deprecating features. If someone needs the one feature that v5.1 provides, namely representing cell array data as 32bit vs 64bit, then they can still make this happen.

In the long term: VTK has a deprecation process for API changes and so on, but file formats are changed so infrequently that a rigorous deprecation process was not followed. This needs fixing.

I can make the change next week, but if anyone else is so inclined, please don’t wait for me (and let me know you are doing it).

5 Likes
7

Thanks so much @will.schroeder !

8

As an aside, is there a timeline on when v5.1 documentation will be introduced if at all?

9

Let me provide you with a smart a** answer to start: it’ll be available when you (or a generous community member) writes it :slight_smile:

There are actually just a few changes regarding cell arrays so most of the previous documentation can be reused. I am happy to help someone write the documentation; otherwise there are no definitive plans that I know of, but I will add it to my list and we’ll make this happen.

2 Likes
10

Thank you @will.schroeder, next week is perfectly fine (we have set the file version flag at application level already, so it is for the benefit of all other VTK-based tools that are built with default options).

11

Commenting for future use.

writer = vtk.vtkPolyDataWriter()
writer.SetFileVersion(42)
...

This will write in older format.

2 Likes