When you increase graphical detail in games, games generally load higher resolution textures, and more of them. In essence, you're loading more DATA for your graphics card to work with, so in seeing your load times increase, you are experiencing normal behavior. This isn't something to worry or concern over, if you were, but it can be obnoxious, if you're used to faster loading times.
The only cure is to either return to the settings you used previously, or improve the speed at which your computer can feed DATA to your graphics card. Using an SSD as your boot drive and installed location of your games comes in handy for this, as it dramatically increases the speed at which games load and OS libraries can be accessed. It's not necessarily a cheap option, but it's probably the only acceptable option, as turning the graphics back down rather nullifies the benefits of upgrading your graphics card.
Also, increasing RAM can help, but only in the case that you had an insufficient amount to begin with. A minimum of 2 GB should run the Sims adequately, without increasing your load times, but other games such as Crysis would benefit from 4 - 8 GB, as they have much larger footprints in RAM.