I don't exactly know why but if I have to take a guess, if responded yes, you will be enabling the website operator to better serve you location-based services. I'd imagine that if you didn't allow it then sites like Google Maps (or others alike) will not know your nearest proximity. IP address these days are PII stuffs (Personally Identifiable Information) in the world of web security along with email address, name, age, passwords, address, bank account, etc.
If you are concern, then always respond in the negative, or just disable the "Ask for location" in Chrome. Or, just VPN.