Unit Testing Approach VTK TDD

Development
1

I have current set up a unit testing framework with CMake and Google Tests with vtk and qt.

I cant find much documentation or examples of how people are developing VTK applications with TDD.

Problems I am finding:

  1. Its hard to test an output, as i have been currently verifying my algorithms with the rendered visuals in the render windows. Therefore a way i have tried to get round this is to test my vtk objects in my pipline to see if they have input and output connections, and if polydata->GetNumberOfPoints() > 0. Is this the way to correct way to test, or is this overkill?

  2. Should I test every vtk object in my pipeline?

  3. How granular should my functions be? in theory i could create a function for each stage of the pipeline (eg Create_mapper(), Create_render_add_mapper(Create_mapper())) which is clearly the wrong why to do it.

  4. i’m finding it difficult to distinguish if I am doing TDD or just putting unnecessary checks in my pipeline.

Myself, and i’m sure many others are looking to move over to TDD/unit testing in order to produce more robust code with a lower bug rate. If there is anyone with any experience with this, or has any advice, I would be very appreciative, as I would love to understand your VTK TDD approach.

If you had a simple example, that would be fantastic!. For example: drawing a line (Shown Below) . How would anyone go about testing this. https://vtk.org/Wiki/VTK/Examples/Cxx/GeometricObjects/Line

Many thanks in advance,

Joe

2

Current example for sphere

TEST.CXX

#include <gtest/gtest.h>
#include “sphere.h”

#include <vtkSphereSource.h>
#include <vtkPolyData.h>
#include <vtkSmartPointer.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkNamedColors.h>
#include <vtkProperty.h>

//Sphere_class* SC_PTR = get_sphere_obj();

Sphere_class SC;
bool test_setup = false;

void setup_test()
{
if (!test_setup)
{
test_setup = true;
SC.draw_sphere();
}
}

TEST(create_sphere_T1, sphere_pipeline_object_exists)
{
setup_test();
int sum_of_working_objects = 0;
int expected_num_working_objs = 6;

if (SC.source != NULL) { sum_of_working_objects++; }
if (SC.mapper != NULL) { sum_of_working_objects++; }
if (SC.renderer != NULL) { sum_of_working_objects++; }
if (SC.render_window != NULL) { sum_of_working_objects++; }
if (SC.interactor != NULL) { sum_of_working_objects++; }
if (SC.actor != NULL) { sum_of_working_objects++; }

ASSERT_EQ(expected_num_working_objs, sum_of_working_objects);

}

TEST(create_sphere_T2, sphere_center_at_origin)
{
setup_test();
bool expected_bool = false;
bool actual_bool = false;

double expected_COM_array[3] = { 0 };
double get_actual_array[3];

if (SC.centre_of_sphere[0] == expected_COM_array[0] && SC.centre_of_sphere[1] == expected_COM_array[1] && SC.centre_of_sphere[2] == expected_COM_array[2])
{
	actual_bool = true;
}

ASSERT_EQ(expected_bool, actual_bool) << "expected_COM_array:  " << expected_COM_array[0] << "  " << expected_COM_array[1] << "  " << expected_COM_array[2] << " actual " << SC.centre_of_sphere[0] << "  " << SC.centre_of_sphere[1] << " " << SC.centre_of_sphere[2];

}

TEST(create_sphere_T3, radius_is_5) {
setup_test();
double expected_COM = 5.0;
ASSERT_EQ(expected_COM, SC.source->GetRadius());
}

TEST(create_sphere_T4, sphere_output_has_poly) {
setup_test();
bool expected = true;
bool actual = false;
if (SC.source->GetOutput()->GetNumberOfPoints() > 0 )
{
actual = true;
}

ASSERT_EQ(expected, actual) << "num points : " << SC.source->GetOutput()->GetNumberOfPoints();

}

TEST(create_sphere_T5, mapper_is_connected) {
setup_test();
int expected = 1;
ASSERT_EQ(expected, SC.mapper->GetTotalNumberOfInputConnections());
}

TEST(create_sphere_T6, actor_has_mapper) {
setup_test();
int expected = 1;
ASSERT_EQ(expected, SC.actor->GetMapper()->GetTotalNumberOfInputConnections());
}

TEST(create_sphere_T6,renderer_has_actor) {
setup_test();
int expected_num_items = 1;
ASSERT_EQ(expected_num_items, SC.renderer->GetActors()->GetNumberOfItems());
}

TEST(create_sphere_T7,renderwindow_has_renderer) {
setup_test();
int expected = 1;
ASSERT_EQ(expected, SC.render_window->HasRenderer(SC.renderer)) << "has renderer " << SC.render_window->HasRenderer(SC.renderer) << endl;
}

TEST(create_sphere_T7,renderwindow_has_interactor) {
setup_test();
bool expected = true;
bool actual = false;

if (SC.render_window->GetInteractor() != NULL)
{
	actual = true;
}

ASSERT_EQ(expected, actual);

}

CXX

#include <vtkSphereSource.h>
#include <vtkPolyData.h>
#include <vtkSmartPointer.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkNamedColors.h>
#include <vtkProperty.h>
#include <gtest/gtest.h>
#include “sphere.h”
#include <vtkConnectivityFilter.h>
#include <gmock/gmock.h >

int main(int argc, char*argv[]) //int, char *[])
{
testing::InitGoogleTest(&argc, argv);
RUN_ALL_TESTS();
}

void Sphere_class::draw_sphere()
{

source->SetCenter(0, 0, 1);
source->Modified();
source->Update();
source->GetCenter(centre_of_sphere);

source->SetRadius(5.0);
source->Modified();
source->Update();

mapper->SetInputConnection(source->GetOutputPort());
mapper->Update();

actor->SetMapper(mapper);

renderer->AddActor(actor);

render_window->AddRenderer(renderer);
render_window->Render();

render_window->SetInteractor(interactor);
render_window->GetInteractor()->Start();

}

SPHERE.H

#ifndef H_SPHERE
#define H_SPHERE

#include <vtkSmartPointer.h>
#include <vtkSphereSource.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>

class Sphere_class
{
public:

vtkSmartPointer<vtkSphereSource> source = vtkSmartPointer<vtkSphereSource>::New();
vtkSmartPointer<vtkPolyDataMapper>mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
vtkSmartPointer<vtkRenderWindow> render_window = vtkSmartPointer<vtkRenderWindow>::New();
vtkSmartPointer<vtkRenderWindowInteractor> interactor = vtkSmartPointer<vtkRenderWindowInteractor>::New();
vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();

double centre_of_sphere[3];

void draw_sphere();

};

#endif // !H_SPHERE

3

The 2000+ tests in VTK should answer your questions. If you need guidance on application level testing, you might have a look at the 1000+ test of ParaView or 600+ tests of 3D Slicer, which include testing of GUI widgets, complex workflows, event recording/replay based tests, etc.

In general, it is indeed very hard to verify that information displayed on the GUI is correct, so application testing often happens at lower levels: for example, you query GUI widgets about what they display instead of trying to do OCR or image comparison.

Since large part of VTK is about rendering, results must be checked visually. VTK does this pretty well, with its image-based regression testing infrastructure.

Probably not. If you spend too much time with developing and maintaining automatic tests then you will not have enough time for everything else.

I usually determine the minimum level of testing by asking myself: Would I be embarrassed if this was broken? If yes, then I add an automatic test for it.

2 Likes
4

thank you for you response, it is very useful. Would It be possible to ask to be more specific?

would it be possible to receive some brief advise on how you might go about testing the sphere example. I would love to know via an example:

  • How many tests are you using for a small function such as draw_sphere?
  • Which parts of it you are focusing on testing.
  • Are you testing connection of VTK objects in your unit tests. Or would this come under integration tests. If so are these intergration tests included at the same time.
  • Are you only testing logic.
  • In order to test effectively would it be advised to split up the one large code in the sphere example, into smaller linked nested functions, in order to more effectively test, or is this necessary.
  • Do you test intermediate local variables with the function, or just the return output of that function.
  • If so, how do you make these locally define variables accessible to your tests class.

Many thanks in advance

5

These are all good questions that you need to answer yourself, based on your project’s requirements and constraints. The goal is not to write perfect software but that is sufficiently good for its intended use.

1 Like