Has anyone hooked up VTK to something like Apache Arrow

Development
1

Has anyone tried combining VTK with Apache Arrow for large scale, out-of-core visualization? We are expecting the data sizes that we currently process to become much larger than the memory on our workstations and the Apache Arrow project looks promising to be able to efficiently process those data sets. But of course the next thing we want to do would be to visualize various hyper slabs of data from the data set. We are currently using VTK as our visualization back end. Was curious if anyone else has tried this combination. Or any other combination that is working.

Thanks
Mike J

2

Mike, you may also want to loop in the ParaView list. VTK / ParaView is used on large supercomputers using MPI etc. with similar decomposition challenges. I know little about this, but maybe the folks here or on the PV list may be able to help you.