Yes there's a noticeable difference between USB 2.0 and USB 3.0, for eg if you have a flash memory connected to USB 2.0, read and write speed may not exceed 10 MB/s, where USB 3.0 speed will be in the range of 90-100 MB/s or faster depending on the drive.
Check if you have a PCIE X1 slot, most old mobos have PCIE X1 slots, if you don't have then you need a new board.