One big difference worth pointing out here is that 8 cores vs 6 cores equals 33.3% more cores, whereas 18 cores vs 16 cores only equals 12.5% more cores, so we're talking about a much smaller relative difference in core count.
Also, while I don't think it's known for certain what the multi-core boost clocks of the Ryzen 3950X will be when all cores are active, the i9-9980XE only boosts to 3.8GHz under those conditions. With it's greater efficiency, it's very possible that the 16-core Ryzen might be able to maintain higher boost clocks, as it will likely be drawing less power and in turn producing less heat than the i9. It also seems like Zen 2's IPC might be a bit ahead of Intel's, and even AMD's existing processors have been shown to get a bit more performance out of their implementation of SMT than Intel's get out of Hyperthreading.
So sure, I could definitely see AMD's $750 processor pulling ahead of Intel's $2000 processor in many heavily-threaded workloads, and since its announced single core boost is higher than what the 9980XE has to offer, it will likely outperform it in many lightly-threaded workloads as well. Of course, this could certainly vary depending on the task at hand, as I could see some things about the i9's architecture and its access to quad-channel memory allowing it to retain an edge in some software. But again, that's a huge price difference any way you look at it. It also seems likely that it won't be long before new Threadripper processors are offering even greater multi-core performance.
5Ghz overclocks seem pretty unlikely. If Zen 2 could do 5Ghz, I'm pretty sure AMD would have pushed at least their single-core boosts closer to that level. At E3, they had some overclockers push the 16-core 3950X to an all-core 5Ghz overclock, but that was on liquid nitrogen. Perhaps there were some power limitations or something at play due to the high core count, but I find it unlikely that an 8-core 3700X is going to hit 5Ghz without exotic cooling. Of course, if the IPC manages to be a bit better than Coffee Lake, that might not matter, since you may not need to hit the same clocks for a similar level of performance.