They should have charged more, as this will do absolutely nothing to keep the mountains shovelware off Steam. In fact, it could even be worse now. Even if the Greenlight voting system was easily gamed and not very effective at filtering what got onto Steam, it at least slowed the process down a bit. With this, "developers" can easily release piles of tech demos thrown together from pre-built assets, or rough ports of mediocre smartphone games, and easily make that tiny submission fee back just by tossing them into third-rate bundles. Even if they make just 10 cents per copy sold in a bundle, they've already broken even on the submission fee once the bundle has moved 1000 copies. They don't actually have to sell any copies through Steam itself to profit from the shovelware.
Even as it is, Steam will apparently reimburse the $100 fee once a developer has hit $1000 in sales through their store, so why not make the fee a bit higher? Valve mentioned in their blog post that they were initially planning for the fee to be around $500, and that would have been much better. They could have simply not collected their usual 30% cut of sales until that amount is paid back. So for a $10 game, a developer would break even with just 50 copies sold, and would have completely recouped the initial fee after selling just 167 copies. If a developer isn't expecting at least $500 in sales through Steam, then that's probably a good sign that their game might not be something people want on the platform. Even just $300 would have been better, while retaining the current $1000 in sales to fully recoup that fee.
The problem with letting anything on Steam with next to no barrier to entry, is that the store gets cluttered with games people don't actually want to play, and it becomes harder for people to find the lesser-known titles that are actually good, and that have had actual effort put into their development. Perhaps the improvements to curator features and suggestion algorithms might help a bit, but I have doubts that things will actually improve much with such a minimal fee for Steam Direct.