Oblivion fps fluctuation

I have a 9800GTX with 1280x1024 resolution, so I expected to max out all the settings.

However, I have everything turned up, my fps fluctuates from a low of 20fps to a high of 110fps, with about 50-60 on average.

Is this normal? I didn't expect to get a barely playable 20fps (which happens a lot during combat) with my GPU and resolution.

 
Depends....Oblivion is very intensive outside. The frame rates going from inside to outside can vary a lot, yes as much as what you are experiencing.
How much memory do you have? How fast is your hard drive? What CPU are you running? Oblivion also kind of favors ATI cards, the game just seemed more optimized for ATI drivers. Also, at that resolution, you are depending on your CPU as much as your video card, practically.

The game goes to the hard drive a lot to load textures. There are several sites you can google for, that give you a list of things to change in the Oblivion .ini file, that can make the game run considerably better on higher end systems. When the game first came out, it was really, really tough on systems at the time, and some of the default ini settings can be increased quite a bit for todays hardware. For instance there is a setting that tells the game engine how much memory can be set aside for textures, and if you have the extra memory, you can increase this setting to decrease trips to the hard drive to load new areas and scenery. The game was originally written during Pentium D and early Athlon 64 days, when a gig of memory in a system was considered the norm.
Anyway, google is your friend, and there are lot of tips and tricks out there that can make the game run a lot smoother.
 
This wasn't even outside. It was actually in the city, at night, fighting some guards. It dropped down to 20fps for almost a full minute.

When I go outside in broad daylight or at night, I get like 50+ fps.

E8400 3.0GHz Core2Duo
4GB memory
7200rpm Harddrive

Considering I run Crysis on high and almost never ever drop below 30fps, it's strange that I drop all the way down to 20 with Oblivion.
 
Take my advice and check out some of the web sites for optimizing the game, because the game will literally bring the best systems to their knees when you try to max everything out. For a long, long time running Oblivion smoothly was every gamers ultimate goal. Cyrsis is tougher on hardware in a lot of ways, but it is mainly because the game engine is just poorly written. Oblivion is equally tough on hardware, but it is because of the huge amount of stuff going on the game. It was literally years before any systems were able to run Oblivion with most settings on max, let alone all of them.
Make sure you don't have a lot of background programs running, check to see if your anti-virus software is set for real time file monitoring, that can slow things down some.
Check some of the sites, that is all I can tell you, there are a lot tips about this very thing.