I have some colors corresponding to points. How to render point clouds and set colors for each point

vtkPolyData does have setPoints(), see here (accessors is provided thanks to this).

You also need to set vertices for your point cloud (i.e. polyData.setVerts(...)).
And to assign a color at each point:

const colors = new Uint8Array(points.getNumberOfPoints());
// ...fill colors here...
const colorsDataArray = vtkDataArray.newInstance({name: colors, values: colors});
polyData.getPointData().setScalars(colorsDataArray);