How to use ProgrammableGlyphFilter?

Hi,
I’m trying to create glyphs which each have a unique color, transformation and cellIDs so I can pick them individually. I found this post which suggested a ProgrammableGlyphFilter and I managed to get the picker to work with it. But I’m not quite sure how to use the ProgrammableGlyphFilter since calling glyphFilter->GetSource() inside CalcGlyph will return the modified glyph from the call before (I got this from the tutorial but it might not be the right way here). So what’s the proper way to get the unmodified source data here or set the output and how can I handle skipping a glyph (for masking glyphs)?
Also the colors do not match the ones I specified in the point’s scalars. Instead of getting red,green (as specified in the code below) I get blue,red as glyph colors.
I also experimented with vtkMultiBlockDataSet and vtkCompositePolyDataMapper2 which has the nice property that the picker returns the block index. However mapper->SetBlockColor() did not change any color in the output.

void CalcGlyph(void* arg) {
  vtkProgrammableGlyphFilter* glyphFilter = (vtkProgrammableGlyphFilter*)arg;
  if (!glyphFilter) {
    return;
  }
  double pointCoords[3];
  glyphFilter->GetPoint(pointCoords);

  vtkNew<vtkTransform> trafo;
  trafo->Identity();
  trafo->Scale(2,1,1);
  trafo->Translate(pointCoords);

  vtkNew<vtkIdFilter> idFilter;
  idFilter->CellIdsOn();
  idFilter->SetInputData(glyphFilter->GetSource());

  vtkNew<vtkTransformFilter> trafoFilter;
  trafoFilter->SetTransform(trafo);
  trafoFilter->SetInputConnection(idFilter->GetOutputPort());

  glyphFilter->SetSourceConnection(trafoFilter->GetOutputPort());
}

vtkSmartPointer<vtkActor> createGlyphActor(vtkSmartPointer<vtkPolyData> mesh, vtkSmartPointer<vtkPoints> positions) {
  vtkIdType n = positions->GetNumberOfPoints();

  vtkNew<vtkUnsignedCharArray> colors;
  colors->SetName("Colors");
  colors->SetNumberOfComponents(3);

  for (vtkIdType i = 0; i < n; ++i) {
    if (i == 0) {
      colors->InsertNextTuple3(255, 0, 0);
    } else {
      colors->InsertNextTuple3(0, 255, 0);
    }
  }

  vtkNew<vtkPolyData> polydata;
  polydata->SetPoints(positions);
  polydata->GetPointData()->SetScalars(colors);

  vtkNew<vtkProgrammableGlyphFilter> glyphFilter;
  glyphFilter->SetInputData(polydata);
  glyphFilter->SetSourceData(mesh);
  glyphFilter->SetGlyphMethod(CalcGlyph, glyphFilter);
  glyphFilter->Update();

  vtkNew<vtkPolyDataMapper> mapper;
  mapper->SetInputConnection(glyphFilter->GetOutputPort());

  vtkNew<vtkActor> actor;
  actor->SetMapper(mapper);

  return actor;
}

I now managed to get everything working with MultiBlockData as described in this post.