# Adding support for opening \*any\* archived single file

**URL:** https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254
**Category:** Development
**Created:** [January 26, 2026, 2:33pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254 "2026-01-26T14:33:58Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [January 26, 2026, 2:33pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/1 "2026-01-26T14:33:58Z")

</div>

**Design:**

Some time ago, I shared a potential design to read archived file in [ParaView discourse](https://discourse.paraview.org/t/adding-support-for-opening-any-archived-single-file/13365).

Well, this idea never went very far as it required a complex implementation in a third party that only partially existed at the time.

But since this is a topic that comes up [often](https://discourse.vtk.org/t/read-a-vtk-file-inside-a-tar-archive-without-extracting-the-files/2554).

I wanted to find a potential solution, and there is actually something that changes in the last few years, [streaming support has been added in VTK](https://www.kitware.com/faster-data-loading-in-vtk/) and implemented in many readers:

- [https://gitlab.kitware.com/vtk/vtk/-/blob/release/Documentation/release/9.6/add-image-reader2-stream-support.md?ref\_type=heads](https://gitlab.kitware.com/vtk/vtk/-/blob/release/Documentation/release/9.6/add-image-reader2-stream-support.md?ref_type=heads)
- [https://gitlab.kitware.com/vtk/vtk/-/blob/release/Documentation/release/9.6/add-stream-support.md?ref\_type=heads](https://gitlab.kitware.com/vtk/vtk/-/blob/release/Documentation/release/9.6/add-stream-support.md?ref_type=heads)
- [https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/release/dev/add-stream-support.md?ref\_type=heads](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/release/dev/add-stream-support.md?ref_type=heads)

While streaming support is in itself a good thing and let VTK be used in many other context than reading data from disk, it also improve a lot what we can do in terms of interoperability with other software and within VTK itself.

Indeed, we could leverage stream support and provide other kind of streams, from different, such as compressed data stream, thus we could imagine doing this:

Single GZip file:

```auto
vtkNew<vtkGZipCompressedResourceStream> stream;
stream->Open("/path/to/file.obj.gz");

vtkNew<vtkOBJReader> reader;
reader->SetStream(stream):
reader->Read()

```

Tarbal:

```auto
vtkNew<vtkTGZCompressedResourceStream> stream;
stream->Open("/path/to/archive.tgz", "file.obj");

vtkNew<vtkOBJReader> reader;
reader->SetStream(stream):
reader->Read()

```

ZIP archive:

```auto
vtkNew<vtkZIPCompressedResourceStream> stream;
stream->Open("/path/to/archive.zip", "file.obj");

vtkNew<vtkOBJReader> reader;
reader->SetStream(stream):
reader->Read()

```

For ZIP and TGZ, the second arg let us select a file from the archive.

**Details:**

As always, the devil is in the details.

1. Reading the whole file in memory

Some readers stream implementation require the whole data to be available in momery (as in `void*, size_t`. Unless such implementation is rewritten to support reading from a proper stream, then the while file will necessarly be copied uncompressed in memory before actually being read.

It will work, but will definitely be memory extensive and show limitations for large files.

1. Seek support

Most readers implementing stream support use the Seek method to move around the stream, which can be very useful, and, at the moment, this is supported by all types of streams. However, it is possible that certains compression algorithm won’t let us properly implement Seek, which will require to ensure all readers support a Seek-less version, which may not be trivial to implement.

1. Slice reading

Most readers, as we expect, read the file part by part, we can expect that some compression may not support that and may require to decompress the whole file before being able to read any data

**Implementation:**

The using a third party to read the compressed file is a must obviously. [zlib](https://www.zlib.net/) would be the classic choice but [libzip](https://libzip.org/) seems to be well placed because it supports many decompression algorithm and allows to [read a slice](https://libzip.org/documentation/zip_source_read.html) and to [seek](https://libzip.org/documentation/zip_file_is_seekable.html).

Of course, other more custom compression may require other implementations and third parties.

What are your thoughts on this ?

---

<div class="post-metadata">

### Author: ![alexy.pellegrini](https://discourse.vtk.org/user_avatar/discourse.vtk.org/alexy.pellegrini/32/5887_2.png) [@alexy.pellegrini](https://discourse.vtk.org/u/alexy.pellegrini)
#### Post date: [January 26, 2026, 3:49pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/2 "2026-01-26T15:49:14Z")

</div>

For archives, I think it would be good to support them in the vtkURILoader, as this would allow resolving and loading file indirectly, that could be useful for importers, (I don’t know if any importer supports the vtkURILoader at this time) and readers like the vtkGLTFReader:

```cpp
vtkNew<vtkURILoader> loader;
loader->SetBaseFileName("gltf.zip");

auto stream = loader->Load("dataset.json"); // load main file manually from zip

vtkNew<vtkGLTFReader> reader;
reader->SetURILoader(loader); // other relative files will be resolved and loaded from the zip
reader->SetStream(stream);

```

To support them in the URI loader there are different approach, either the base file name must be a zip (last component), or do something higher level by resolving zip in full paths, but that would be tricky.

* * *

About Seek support, note that the libzip indicates that compressed archives do not support seeking, and uncompressed ZIP are uncommon.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [January 26, 2026, 3:51pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/3 "2026-01-26T15:51:41Z")

</div>

> [@alexy.pellegrini](#):
>
> I think it would be good to support them in the vtkURILoader,

Indeed, that would be interesting in the context of glTF and we should add URI loader support to other readers/importers as well.

> the libzip indicates that compressed archives do not support seeking

Unfortunate, where did you find that info ?

---

<div class="post-metadata">

### Author: ![alexy.pellegrini](https://discourse.vtk.org/user_avatar/discourse.vtk.org/alexy.pellegrini/32/5887_2.png) [@alexy.pellegrini](https://discourse.vtk.org/u/alexy.pellegrini)
#### Post date: [January 26, 2026, 3:55pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/4 "2026-01-26T15:55:20Z")

</div>

In the link you gave:

> The [`zip_fseek`](https://libzip.org/documentation/zip_file_is_seekable.html#zip_fseek)() function seeks to the specified offset relative to whence, just like [fseek(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/fseek.html).  
> `zip_fseek` only works on uncompressed (stored), unencrypted data. When called on compressed or encrypted data it will return an error.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [January 26, 2026, 3:57pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/5 "2026-01-26T15:57:08Z")

</div>

Next time I will try reading 🙂

Anyway, it indeed shows that the Seek issue I highlight may be critical, unless we find another third party that can indeed Seek.

---

<div class="post-metadata">

### Author: ![toddy](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/t/edb3f5/32.png) [@toddy](https://discourse.vtk.org/u/toddy)
#### Post date: [February 3, 2026, 11:27pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/6 "2026-02-03T23:27:55Z")

</div>

Why not just read the compressed stream into an uncompressed (buffered) one then Seek() on it.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 4, 2026, 7:55am UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/7 "2026-02-04T07:55:30Z")

</div>

This is indeed a possible fallback, but not ideal when dealing with larg files.

---

<div class="post-metadata">

### Author: ![toddy](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/t/edb3f5/32.png) [@toddy](https://discourse.vtk.org/u/toddy)
#### Post date: [February 5, 2026, 3:24am UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/8 "2026-02-05T03:24:34Z")

</div>

I suppose that depends on whether you need to seek to the end of the file or just in chunks no larger than the decompressed buffer.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 9, 2026, 1:57pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/9 "2026-02-09T13:57:57Z")

</div>

With no further feedback, I’ll design a proper solution.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 9, 2026, 4:37pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/10 "2026-02-09T16:37:22Z")

</div>

Here is a quick analysis of potential 3rd parties:

| | zlib | minizip(zlib) | libzip | libarchive |
| --- | --- | --- | --- | --- |
| .gz | ✅ | ✅ | ✅ | ✅ |
| .xz | ⛔ | ⛔ | ✅ | ✅ |
| .tar.gz | ✅ | ✅ | ✅ | ✅ |
| .tar.xz | ⛔ | ⛔ | ✅ | ✅ |
| .zip | ⛔ | ✅ | ✅ | ✅ |
| .rar | ⛔ | ⛔ | ⛔ | ✅ |
| .7z | ⛔ | ⛔ | ⛔ | ✅ |
| popular | ✅ | ✅ | ⚠ | ✅ |
| maintained | ✅ | ✅ | ⚠ | ✅ |
| support stream | ✅ | ✅ | ⛔ | ✅ |
| support seek | ⛔ | ⛔ | ⛔ | ⛔ |
| in VTK/PV/PVSB | ✅ | ⚠ | ⛔ | ⛔ |

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 9, 2026, 5:02pm UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/11 "2026-02-09T17:02:18Z")

</div>

**libarchive** is the clear winner here! Seek support is just not possible with most compression algorithm so lets consider it not possible. A pure zlib backend would be nice but this doesnt seem necessary after all.

I’ll design assuming libarchive usage.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 10, 2026, 10:58am UTC](https://discourse.vtk.org/t/adding-support-for-opening-any-archived-single-file/16254/12 "2026-02-10T10:58:54Z")

</div>

Here is a tentative API:

```auto
​vtkCompressedResourceStream : public vtkResourceStream
{
  SupportSeek () { return false; };
  Read (void *buffer, std::size_t bytes); // impl using libarchive
  EndOfStream(); // impl using libarchive
  Tell(); // impl using libarchive
}

​vtkCompressedFileResourceStream : public vtkCompressedResourceStream
{
  Open(const std::string& archive, const std::string& file) // impl using libarchive
}

​vtkCompressedMemoryResourceStream : public vtkCompressedResourceStream
{
  SetBuffer(const void *buffer, std::size_t size) // impl using libarchive
}

```

Contrary to vtkMemoryResourceStream, the suport of copying the memory into its own buffer seems not needed for this specialized stream, but it could be added in the future.  
The URI loader support suggested by @alexy.pellegrini above is considered outside of the scope at the moment.
