# How to understand binary DataArray in xml vtk output

**URL:** https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489
**Category:** Support
**Tags:** file-formats
**Created:** [October 26, 2020, 12:11pm UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489 "2020-10-26T12:11:07Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![qzxuhui](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/q/d78d45/32.png) [@qzxuhui](https://discourse.vtk.org/u/qzxuhui)
#### Post date: [October 26, 2020, 12:11pm UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489/1 "2020-10-26T12:11:07Z")

</div>

Hello everyone, I have a problem with how to understand the binary `DataArray`.

The manual said if the format of `DataArray` is `binary`,

```auto
The data are encoded in base64 and listed contiguously inside the
DataArray element. Data may also be compressed before encoding in base64. The byte-
order of the data matches that specified by the byte_order attribute of the VTKFile ele-
ment.

```

I can not fully understand that, so I have obtain ascii file and binary file for same model.

# ASCII file

```auto
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian" header_type="UInt32" compressor="vtkZLibDataCompressor">
  <UnstructuredGrid>
    <Piece NumberOfPoints="4" NumberOfCells="1">
      <PointData>
      </PointData>
      <CellData>
      </CellData>
      <Points>
        <DataArray type="Float32" Name="Points" NumberOfComponents="3" format="ascii" RangeMin="0" RangeMax="1.4142135624">
          0 0 0 1 0 0
          1 1 0 0 1 1
        </DataArray>
      </Points>
      <Cells>
        <DataArray type="Int64" Name="connectivity" format="ascii" RangeMin="0" RangeMax="3">
          0 1 2 3
        </DataArray>
        <DataArray type="Int64" Name="offsets" format="ascii" RangeMin="4" RangeMax="4">
          4
        </DataArray>
        <DataArray type="UInt8" Name="types" format="ascii" RangeMin="10" RangeMax="10">
          10
        </DataArray>
      </Cells>
    </Piece>
  </UnstructuredGrid>
</VTKFile>

```

# Binary file

```auto
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian" header_type="UInt32" compressor="vtkZLibDataCompressor">
  <UnstructuredGrid>
    <Piece NumberOfPoints="4" NumberOfCells="1">
      <PointData>
      </PointData>
      <CellData>
      </CellData>
      <Points>
        <DataArray type="Float32" Name="Points" NumberOfComponents="3" format="binary" RangeMin="0" RangeMax="1.4142135624">
          AQAAAACAAAAwAAAAEQAAAA==eJxjYEAGDfaobEw+ADwjA7w=
        </DataArray>
      </Points>
      <Cells>
        <DataArray type="Int64" Name="connectivity" format="binary" RangeMin="0" RangeMax="3">
          AQAAAACAAAAgAAAAEwAAAA==eJxjYIAARijNBKWZoTQAAHAABw==
        </DataArray>
        <DataArray type="Int64" Name="offsets" format="binary" RangeMin="4" RangeMax="4">
          AQAAAACAAAAIAAAACwAAAA==eJxjYYAAAAAoAAU=
        </DataArray>
        <DataArray type="UInt8" Name="types" format="binary" RangeMin="10" RangeMax="10">
          AQAAAACAAAABAAAACQAAAA==eJzjAgAACwAL
        </DataArray>
      </Cells>
    </Piece>
  </UnstructuredGrid>
</VTKFile>

```

When I looked at the the DataArray, using the last one as an example, I can not create the relationship between `AQAAAACAAAABAAAACQAAAA==eJzjAgAACwAL` and `10`.

My understanding can be expressed using follow code, but it obtain `CggAAA==`.

```auto
#include "base64.h" // https://github.com/superwills/NibbleAndAHalf/blob/master/NibbleAndAHalf/base64.h
#include <iostream>
int main()
{
    int x = 10;
    int len;

    // first arg: binary buffer
    // second arg: length of binary buffer
    // third arg: length of ascii buffer
    char *ascii = base64((char *)&x, sizeof(int), &len);
    
    std::cout << ascii << std::endl;
    std::cout << len << std::endl;
    free(ascii);
    return 0;
}

```

Can someone give me an explanation of how to convert?  
Another relate topic can be see in

- [error when writing binary VTK files](https://discourse.vtk.org/t/error-when-writing-binary-vtk-files/4487/7)

Thanks for your time.

---

<div class="post-metadata">

### Author: ![Yohann\_Bearzi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/yohann_bearzi/32/671_2.png) [@Yohann\_Bearzi](https://discourse.vtk.org/u/Yohann_Bearzi)
#### Post date: [October 26, 2020, 10:00pm UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489/2 "2020-10-26T22:00:21Z")

</div>

You will find your answer in `IO/Core/vtkBase64Utilities.cxx`. Bytes are encoded by triplets, pairs, or singletons. Whenever you find one isolated `=`, it is encoded by pair. When you find the pattern `==`, it is encoded by singleton. Everywhere else, it is encoded by triplet.

I suppose that the reason for this custom mapping was to be able to read more easily binary data when you open your file in a text editor, with a set of 64 human readable symbols, and no occurrence of `Enter`, `Tab`, or this kind of symbols that could pop up.

The file I pointed you to allows you to encode and decode data.

---

<div class="post-metadata">

### Author: ![Yohann\_Bearzi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/yohann_bearzi/32/671_2.png) [@Yohann\_Bearzi](https://discourse.vtk.org/u/Yohann_Bearzi)
#### Post date: [October 26, 2020, 10:18pm UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489/3 "2020-10-26T22:18:31Z")

</div>

I suspect that you have a lot of what seems like extra data because you are using a compressor. When you do so, there is a need to write a compressor header at the beginning of each data array you are writing. If you do `SetCompressor(nullptr)` prior to writing your file, you should have raw binary data, following the encoding rule of `vtkBase64Utilities.cxx`.

---

<div class="post-metadata">

### Author: ![qzxuhui](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/q/d78d45/32.png) [@qzxuhui](https://discourse.vtk.org/u/qzxuhui)
#### Post date: [October 27, 2020, 12:06am UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489/4 "2020-10-27T00:06:50Z")

</div>

Thanks for sharing sir.

I now can have a better understanding now: grinning: .

It is a complicated process. I think I should directly call the vtk API in my code other than writing the my own base64 encoder, compressor et al.

---

<div class="post-metadata">

### Author: ![Yohann\_Bearzi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/yohann_bearzi/32/671_2.png) [@Yohann\_Bearzi](https://discourse.vtk.org/u/Yohann_Bearzi)
#### Post date: [October 27, 2020, 5:36pm UTC](https://discourse.vtk.org/t/how-to-understand-binary-dataarray-in-xml-vtk-output/4489/5 "2020-10-27T17:36:35Z")

</div>

@qzxuhui If you want to create a custom XML reader / writer, I highly recommend writing classes inheriting from `vtkXMLReader` and `vtkXMLWriter`. XML elements are easy to handle, and there are ready-to-use methods that write / read arrays from an XML file, with the possibility to read an array given an input segment of indices.

Edit: Plus, all this encoding is hidden under the rug. You don’t have to worry about compressed / uncompressed data. The only tedious thing is that if you write your data in `Appended` mode or in `Binary` / `ASCII` mode (which are inlined), you have to handle it explicitly in the writer.
