Yes you can. It will not harm your board.
But
More sticks means more stress on memory controller. Most four slot boards have two MC and four sticks puts slightly more stress than two would.
This isn't a big downside though. If you want four then go for four.