Not really. You don't even know what criteria they are using for grey to grey to get that 1 ms number. At any rate there is still a lot of added input lag from the GPU to the monitor and processing in the monitor until it gets to the point where the pixel starts switching.
It could be that the 4ms monitor has an input lag of 11ms and the 1ms monitor could have a 19ms input lag. Giving the 4ms monitor overrall better response times. You won't know that unless you can find a review of monitor you are interested in. One which includes input lag.
Really as long as it is under 5ms. I'd be looking at user reviews for warning flags like ghosting or lag. Also backlight bleed for IPS.