vtkrenderedareaselector with one vtkAssembly for the renderer

I have an application that I am porting an earlier version of VTK (5.2) to use the latest supported version. The legacy app has a single renderer with a single vtkAssembly that builds a nested structure of actors. Picking is implemented through the vtkRenderedAreaPicker, and with the legacy version, I could use the GetProp3Ds from an area pick to select the child props that were picked from that single vtkAssembly.

However, when I migrated to the newest version which now uses the vtkHardwareSelector, the top level vtkAssembly is returned, not the lowest level. Am I missing something?
Is there a way to get at the actual child prop that was picked without adding all the children as props to the renderer?