The skates themselves. 400 grit sandpaper is extremely fine....it won't hurt them....but it might give you a more consistent surface and make it slide nicer.
Try different things....experiment.
Here's an example....just to give you an idea about experimenting with things. I have a Razer Turret keyboard and mouse and the surface that the mouse sits on gets polished and becomes very shiny over time....and then the mouse doesn't work (because the light from the mouse doesn't reflect right)....so I take some 400 grit sandpaper and sand the surface the mouse sits on. This gets rid of the shine and the mouse works again for about 6 months or so until it gets shiny again. I just figured this out by trying it.