Ok first off you are going WAAAY over kill. Things you need to understand first.
150Mbps is WAAAY faster than you need for ANY KIND OF GAMING. Unless you are hosting a server where 64 other people are connecting to YOU (In which that 150 MAY still be enough depending on the game) you will NOT see ANY difference in gaming with anything faster than 50Mbps and a low ping of 20ms or lower.
This isn't the day of old DSL where having a 120-200ms ping was fast. These days, even with DSL, most pings are 40ms and below that is what counts NOT the speed of the internet.
A CAT6a Shielded network cable is NOT NEEDED. Shielded cabling is almost not needed period unless you have like 50+ cables all running together or in a place where there could be a lot of outside interference. I have to come across these and I manage the PC's and networks of 100 business or so.
Also a Modem is a Modem. As long as it is a DOCIS 3.0 which is the faster they got for faster speeds that is fine but if it is FIBER more than likely you HAVE to use their modem depending on the company. Either way you don't get the fiber directly to your router. It has to go to a ONT (Onsite Network Terminal) that is their equiptment that you can't change out. It then converts to Copper CAT5/6 from there.
Also 10Gpbs NIC cards will do you absolutely nothing. Nothing at all. Now I have a server, a few PC's, wireless devices, It is all Gigabit (1000) and Wireless AC. I have 25/25 FiOS. Now me getting CAT6a cables WILL NOT do anything to make my online gaming any faster. Now for me a 10Gbps in my server and my Main PC would be good as I do a LOT of video converting and editing and getting faster than 100MBps would be nice since both of them can go faster but my gigabit network is the bottleneck there but games don't come anywhere near to transfer that.
Now the ASUS RT-AC5300 is a monster. Devices like these are for enviroments where WiFi is dominant. A place where you litlery have to have 50+ Wireless devices to make any sense of getting this. This does not have any better range than most any other Wireless AC Router either! The only thing about this is it can handle the many many Wireless devices to connect to it. As far as LAN goes its just a gigabit lan nothing special. I have the ASUS RT-AC66U and love it. I also have a RT-N66U. The difference? Once has AC while one has only N. Now do i see a difference? Nope. Do i see any better coverage? Nope. Is one really better than the other? Nope. One can transfer over wifi faster than the other but again it is designed to have a lot of wifi devices, 4k streaming, etc.
Trust me you are over thinking it. You will be find with unshielded Cat5e/Cat6a. having Faster internet, unless your ping is shit, will get you any better gaming. Getting a better modem than a ISP modem most than likely won't help you any either. If you go to speedtest.net and your ping is under 40ms you are golden for any kind of gaming.
Also the only Routers i found with a 10Gbps port on them are NOT the internet port but a LAN port to connect a device to, this again would not help at all even if you connected a PC.
You internet and setup is more than likely more than enough for what you need. Maybe getting a better router for Wifi or if it wasn't Gigabit, which i'm sure it is, would be smart but the need to get a $200+ router for your everyday consumer is beyond me of why you would need it. There isn't anything better about them except maybe some features. Most high end routers you are thinking of, are enterprise class which in the enterprise world, has a lot of fiber.