It's not a problem with your computers.
You might see this symbol a few secounds when you connect to your wifi, which is a normal thing while it's connecting to the wifi. Also if you've just switched the router on, it might also take a minute or two for it to connect.
If the icon never goes away, you can't go online and you're experiencing the same with two different devices, then it's probably something with your router.
The icon tells you that you have a offline connection to your wifi (so you can still communicate with the other devices connected, but you can't go on the internet). If the icon sticks around, then it might be your router that is disconnected from your internet provider.
You should check if there's ANY other devices that will go online when this happens, (etc. check with your phone or another computer), if you can't connect on that device either, then it's probably your router.
You could also check the different cables, if one of them is a little loose (or not properly inserted) it might be that too.
I hope this helped you a little 😉