How to speed up about loading CSV file?

Support
1

Hi all!

I have a question about loading the CSV file.

I’m using the ‘vtkDelimitedTextReader’ class to read the CSV model file.

The code is as below:

vtkNew<vtkDelimitedTextReader> m_vtkCSVReader;

m_vtkCSVReader->SetFileName(path);
m_vtkCSVReader->DetectNumericColumnsOn();
m_vtkCSVReader->SetFieldDelimiterCharacters(",");
m_vtkCSVReader->Update();

vtkSmartPointer<vtkTable> table = m_vtkCSVReader->GetOutput();

vtkNew<vtkPoints> points;

int row = table->GetNumberOfRows();
int col = table->GetNumberOfColumns();

for (vtkIdType i = 0; i < table->GetNumberOfRows(); i++)
{
	points->InsertNextPoint((table->GetValue(i, 0)).ToDouble(),
		(table->GetValue(i, 1)).ToDouble(),
		(table->GetValue(i, 2)).ToDouble());
}

vtkNew<vtkPolyData> polydata;
polydata->SetPoints(points);

vtkNew<vtkVertexGlyphFilter> glyphFilter;
glyphFilter->SetInputData(polydata);
glyphFilter->Update();

vtkNew<vtkLookupTable> lut;
double minz = 0.0f, maxz = 0.0f;
GetDataBounds(polydata, &minz, &maxz);

int numPoints = points->GetNumberOfPoints();
CreateLookupTable(minz, maxz, numPoints, &lut);

vtkNew<vtkUnsignedCharArray> colors;
CreateScalars(points, lut, &colors);

polydata->GetPointData()->SetScalars(colors);

vtkNew<vtkPointGaussianMapper> mapper;
mapper->SetInputConnection(glyphFilter->GetOutputPort());
mapper->EmissiveOff();
mapper->SetScaleFactor(0);

vtkNew<vtkActor> actor;
actor->SetMapper(mapper);
actor->GetProperty()->SetOpacity(1.0);

vtkNew<vtkRenderer> renderer;
renderer->AddActor(actor);
renderer->SetBackground(.1, .2, .3);
renderer->ResetCamera();

vtkNew<vtkRenderWindowInteractor> renderWindowInteractor;
renderWindowInteractor->SetRenderWindow(m_vtkRenderWindow);
renderWindowInteractor->Initialize();

vtkNew<CCustomMouseInteractorStyle> style;
style->SetDefaultRenderer(renderer);
style->SetParent(this);
renderWindowInteractor->SetInteractorStyle(style);
renderWindowInteractor->Initialize();

m_vtkRenderWindow->AddRenderer(renderer);
m_vtkRenderWindow->Render();

It works well but the speed is very slow.
Is there any fast way of reading the CSV model file?

Could you please advise me? :sob:

Thank you :slight_smile:

2

I’d recommend timing individual components using vtkTimerLog. Is the m_vtkCSVReader->Update() the slow part or is the filling up of vtkPoints?

If it’s the filling up of vtkPoints, here are ways to improve it:

  1. use Allocate to avoid resizing array on each InsertNextPoint. e.g. points->Allocate(table->GetNumberOfRows()) before the for loop.
  2. Don’t use vtkTable::GetValue is has unncessary conversions to vtkVariant. Directly access the column array and use them. e.g.
auto xArray = vtkDoubleArray::SafeDownCast(table->GetColumn(0));
auto yArray = vtkDoubleArray::SafeDownCast(table->GetColumn(1));
auto zArray = vtkDoubleArray::SafeDownCast(table->GetColumn(2));
for (...)
{
   points->InsertNextPoint(xArray->GetTypedComponent(i, 0), 
                           yArray->GetTypedComponent(i, 0), 
                           zArray->GetTypedComponent(i, 0));

}
3

Thank you for your reply!

I found that the m_vtkCSVReader->Update() is the slow part.