That's odd you should be able to get way better speeds than just 54Mbps. Enter the router config and go to the wireless setup page. Check the settings there, you should have the max Tx Rate set to 300Mbps and the Mode should be Mixed. Actually you can also try setting the Mode to N (only) if possible to see if it gets better.
Have a look at those settings and see how it goes. Otherwise you may need o check the settings of your adapter.
Have you checked the speed on any other devices connected to the router?