It indicates that the connector is plugged correctly and provides power as intended. There is no reason to worry about it.
If you switch off PSU it should turn off because no power is provided.
Some manufacturers also have the same led turning a different color (usually red) if something is wrong with the connector.