In an older HP computer, I found that I had to go into the BIOS, with the USB plugged in, then say for primary boot to use the "HDD Group" - and in the HDD Group, I could then select the USB drive.
If the USB drive wasn't plugged in when I was in the BIOS, though, then that option wouldn't show up under the HDD Group.
It was, in my opinion, a pretty bizarre way of doing things.