I don't think that's actually accurate. There were initially a lot of people buying RX 480/580 cards for gaming, and they tended to be highly recommended by tech sites for mid-range gaming systems. Performance was similar to Nvidia's 1060 cards at a somewhat lower price point, which made them a fairly attractive option for those building systems to a budget, where spending several hundred dollars or more for something faster wasn't an option. The only 500-series card that I would say was poorly received at launch was the RX 590, which was overpriced for being essentially another refresh of a 2.5 year old card, but that didn't come out until after the crypto-market had already crashed, and at a time when Nvidia had already started releasing some of their higher-end 20-series cards (which were also poorly received, but at least promised some new features down the line).
What I think actually happened was that since AMD's cards were largely out of stock due to the mining market for months prior to the supply of Nvidia's cards being significantly affected, it became really hard to recommend them at that point. Even after the supply of Nvidia's cards also dried up, resellers were charging more for AMD's cards, since they knew miners would pay a premium for them. So, for roughly a year or so there, AMD's cards didn't really make sense for anyone but miners to buy, since they lost the value advantage they had been offering over the competition, and actually cost substantially more for a given level of gaming performance. Vega was likewise DOA for the gaming market from launch, since it was released during the height of the mining rush, making the cards nearly impossible to buy for a reasonable price.
So, once the mining market did crash, a disproportionately large portion of AMD cards got dropped back onto the secondhand market, since they had been nearly all going to miners for the prior year. And since only miners had been buying the cards since their prices skyrocketed, AMD likely lost some momentum in the gaming market during that time. And of course, at that point, people were already wanting to see what the next generation of cards would offer. Combined, all that made it harder to sell cards at retail, hence the relatively large price cuts and incentives that needed to be thrown in to help clear inventory.