I'm the same in some ways. I previously had an AMD FX6300. Basically technology from 2011. It was ok for what it was, but I'm telling you compared to that, the Ryzen clobbers it.
That said, I just put my ryzen system together 3-4 months ago. It will overclock a little, and I'm happy with it. I think I'm running 3.7ghz on the stock cooler.
What it all boils down to is that for bang for your buck, ryzen is hard to beat right now. Coffee lake is good, but as you are seeing, you are going to spend more on the board and the cpu up front. So basically you will spend more now, or spend it later. In other words, if you stay with the Ryzen, plan on upgrading in a couple of years. If you get the intel, you might get 3-5 years.
The good thing with Ryzen, AMD is promising to support the same socket until 2020. Intel has a habit of releasing new chips that require new motherboards every 2-3 years. So in say 3 years, you may want to upgrade your intel processor, but may be stuck buying another new board.
With AMD, I'm hoping, because they've been decent about this in the past, if they keep the same socket, that say in 2019 or 2020 when say a Ryzen version 3 that is a nice jump over my 1600 is released, that I'll be able to update the bios on my motherboard, and install a shiny Ryzen version 3000 series chip or whatever they call it at that time, and jump back to whatever performance level is there at the current time. I can't promise they won't require new boards later, but my past experiences with them have been positive in that regard. Plus they seem to hit the wallet less when you initially purchase. The only reason I think intel is pricing so competively is due to Ryzen. I think they didn't know what to really expect with Ryzen, and that it kind of shocked them, and that is why the quick rollout of Coffee lake.
As I think matt said above too, don't buy into Coffee lake only because you think you'll be set for a long time. An article I just saw is talking about Cannon Lake already in late 2018. So they will be bumping that up again.
Here's a link to AMD's proposed roadmap.
http://hexus.net/tech/news/cpu/105772-amd-unveils-roadmap-ryzen-threadripper-ryzen-3-apus/
From what they show Ryzen was the start. I saw another article that is talking about a new Ryzen in February 2018, which may be the 14nm+ shown on their map. It looks from that like late next year, expect Ryzen 2, then after that sometime in 2019 or 2020 for Ryzen 3.
Here's another forum where someone was asking the question how long will a Ryzen 1600 and a 1080ti last him. A lot of the answers say 5 years essentially.
https://linustechtips.com/main/topic/816357-how-long-will-ryzen-5-1600-and-gtx-1080ti-last-me-estimate/
But that said, as you can see by the roadmap from AMD, it appears as though if you do for whatever decide you want to upgrade, you may be able to go to Ryzen 2 or Ryzen 3 later. Personally, as I said, I know if I want current performance, I tend to upgrade every 2-3 years incrementally. So for me, after I've had a cpu a couple of years, I'll evaluate if I see any new cpus worth upgrading to or not. Same with video cards as well.
But if you do decide to upgrade, lets say a ryzen 2600 6 core 12 thread or whatever they have out at the time costs 200 dollars like now, you'd probably be able to sell your old one for 75 or 100 dollars and make up part of the cost.
Ultimately though, you'll either spend more right now for better performance that hopefully lasts longer, or get a system that's still a stout system for what you want to do that should let you have an upgrade path for when you want to add to it or change it later. I would just get what you can afford. But if you do decide to stay with the AMD bulid, I don't think you will be dissapointed. If you look at a lot of the guys on youtube etc, some of them said they decided to change to ryzen from intel(at least before coffee lake), due to the fact they could have decent gaming performance, but still stream etc at the same time.