We are running into an issue upgrading ShapeWorks to VTK 9. We have an export mesh capability that writes out a polydata to a .vtk file. Our users import this into other tools such as CloudCompare. When we upgrade to VTK 9, the new 5.1 format written by our tools is not readable by these other tools. Is there a way that the older, well established, .vtk format can be written by VTK 9 and higher? For now, we are just sticking with VTK 8 so that we can provide this file format compatibility.
What are the differences between VTK 9 and VTK 8 when saving a polydata vtk file? Can you do a diff between the two files?
Thanks,
Dan
I’m sure it will depend on the polydata, but for a simple sphere example:
The older 4.2 uses this style of specifying polygons:
POLYGONS 30136 120544
3 0 46 34
3 0 36 1
…
While the 5.1 uses:
POLYGONS 30137 90408
OFFSETS vtktypeint64
0 3 6 9 12 15 18 21 24
27 30 33 36 39 42 45 48 51
…
I still haven’t seen any documentation for the 5.1 format.
This is likely the change that caused the bump in version number and incompatibility.
https://discourse.vtk.org/t/upcoming-changes-to-vtkcellarray/2066
Yes, that seems like it.
What suggestion do you have for tools using VTK that would like to write the older, more generally compatible format? I suppose I can write my own file writer, but I am guessing that others will be running into the same problem if and when they upgrade to VTK 9.
Can you file an enhancement request to CloudCompare issue tracking, asking for updating their VTK reader? Or CloudCompare could implement support for VTP file format (VTP is XML-based, which can be enhanced in backward-compatible way).
VTK supports many other mesh formats, so you may also be able to switch to a different format.
I don’t think CloudCompare is the only software used by our users, it was just one example. Is there documentation or specification for the new format? The old one was well specified.
I think you’re right that we will probably have drop support for VTK export if we upgrade to VTK 9.0.
I don’t think these changes made it into the documentation yet.
Maybe the new VTK Examples site provides a more dynamic and easier to update place.
I don’t think CloudCompare is the only software used by our users
Indeed, many software uses VTK legacy file format (especially the ASCII version), but probably most of them can handle the latest file format variant by simply ignoring unknown fields. This latest change from POLYGONS to POLYGONS+OFFSETS may be a substantial change, though (not just an extra field).
It would be useful to define a clear strategy for .vtk and XML-based .vt* file formats. My understanding is that a decade ago the plan was to declare text-based format “legacy” and slowly phase it out, and encourage everyone to use the XML-based formats instead. However, since then a few things happened:
- XML-based VTK formats did not really take off anywhere outside VTK. No third-party software (that I know of) bothered to implement support for these formats, but they still use .vtk (mostly the ascii version only)
- Not even VTK really let go the “legacy” .vtk format (see above for a good example - the format is still actively being worked on)
- XML is not that popular anymore, but instead the default human readable data exchange format is now json
A solution could be to recognize .vtk file format as a first-class citizen and treat it accordingly (properly document, pay attention to forward and backward compatibility, etc.), deprecate XML-based file formats (do not spend any time into improving them to reduce maintenance efforts) and use glTF instead (not as an export/import format but as a lossless VTK data object read/write format).
Or CloudCompare could implement support for VTP file format (VTP is XML-based, which can be enhanced in backward-compatible way).
IMO supporting reading of vtp,vtu,… formats without using vtk libraries feels like a bottomless pit. You have numerous things that could be thrown at it
- inline ascii, inline with base64 encoding
- append formats, raw or base64 encoding
- possible compression
- possible endian swapping
- and of course, an xml parser.
Fortunately, I’ve limited myself to just generating the files, but even there I found that creating the append formats (which I would actually prefer to have) was simply untenable for me. My geometry source can be multi-piece (one piece per processor), which I stream via the master process. At a later point I add CellData fields. The only way to generate the append format is if I know a priori which fields will be written, and than can either generate the header appropriately or (as per vtk internals) generate a header with additional spacing so that I can rewind and overwrite the offsets.
I fully agree that it does not make sense for non-VTK-based software to implement reader/writer for VTK’s XML-based file formats. It’s not just us: I know many software that supports .vtk files but I don’t know any that supports the XML-based formats (unless the application uses VTK).
It would be probably better to just let these XML-based file formats slowly die and move on to a capable standard file format, such as glTF. VTK already has nice glTF importer/export classes, but I haven’t checked recently if they can already save/load all data arrays, support multiblock, etc. glTF is a very complicated file format (much more complicated than VTK’s XML-based formats), but at least it is a widely supported standard, so the time investment can be better justified.
Simple text-based .vtk format would be still useful for exchanging data with small and simple software tools.
+1 for glTF, it can store extra channels of information. For the tracko format we convert point and cell data between vtp/vtk and the glTF files.
I think it would be great of the vtk writers could have a flag to produce earlier version of the formats. Probably it should even be the default to avoid this kind of breaking change (old Bill would not have liked the breaking change).
It has been a while since I last looked. Have they improved gltf? Last I saw it didn’t even support quads much less all the other common 2d, 3d cell types simulation and visualization routinely use. I know you can stuff other things in there as a user or vendor extension, but quads is pretty basic.
It would be great if this could be added to a minor/patch release of VTK in the near future. It would make updating projects to use VTK 9 much more feasible.
Khronos group intends to add support for storing quads (see [1] and [2]), but probably Kitware would need to specify vendor extensions for storing everything that VTK needs. Even if data in vendor extensions are not readily usable by generic glTF viewers, using glTF as a container is useful, because it makes it easier for third-party software to read/write VTK files (because low-level infrastructure for file access, encoding, compression, etc. are taken care of) and allows storing all metadata in a single file.
FYI, I’ll be reworking the legacy readers to handle the old formats. It’ll take a couple of weeks with my current work load.
I’ve got a first draft of VTK 4.2 support. Please take a look and comment as necessary.
2 Likes
Hi Will, does something similar exists under python? I installed vtk using anaconda (it was impossible for me to force the installation of 8.x version). I need to write vtk files in the same format vtk 8.x does.
Thanks,
Cesare
If you are using “recent” versions of VTK it should be available. (Because of the way the automated Python wrapping is performed, most anything you can do in C++ can be done in Python.) Look at the test script VTK/IO/Legacy/Testing/Python/TestVTKLegacy.py to see how it’s done.
writer = vtk.vtkPolyDataWriter()
writer.SetFileVersion(42)
...
This will write in older format.
1 Like