There is one area which tends to put many average users off of linux. I have tried to get some users who constantly get their PC loaded with malware, to use a linux distro (tried ubuntu, and linux mint), and it always failed because even though they were aware they were ruining their windows install very quickly, their needs weren't so basic that they could get by with just a web browser and an office application.
Distros such as ubuntu have made good progress in making things more user friendly, but the moment you venture outside of the walled garden of their app store, the complexity skyrockets. It is not like with windows where there are hundreds of layers of complexity where the user can gradually go to a level that is comfortable for them, in the major linux distros, you are in the basic and user friendly app store, or deep into CLI.
The CLI reliance is where things fall apart. The moment something outside of the app store, where there is no installer available, the user is stuck with a frustrating experience with a high failure rate if a set of instructions are for a slightly different version of the OS. For example, you may see a tar.gz package, and find that someone posted a set of instructions to install and get it working on ubuntu (or whatever distro you are using), then having to use a text editor like nano to edit a few of the files, then doing a bunch more commands, only to end up at step 15 and get tons of errors because because the instructions were based on ubuntuu 14.04 and you have 16.04 and due to differences in some of the newer versions of some of the dependencies, the old set of commands will not work, but if you do not have a deeper understanding of those other packages, then getting things working will be an uphill battle.
Overall, this is the primary point where people give up on linux. They try a distro, and things start off great, but they run into an issue, or find that they need to use some peripheral that the OS does not simply detect and just work, and that the software center lacks anything for, thus they google for it, find instructions that fail part way through because they have a newer version of the OS, or one of the dozen dependencies needs to be downgraded to an older version but there is no telling which.
Outside of that, I do have one family friend on ubuntu, and that person would likely be perfectly fine with the chrome OS since they pretty much only ever use the web browser, but for those with more needs, sooner or later, they find a limit to the software center, and end up in tar.gz hell.