Small differences in speed make a big difference in arena, yes, but they don't really make a huge difference elsewhere in the game.
Crit rate and Crit damage, on the other hand, scale *far* faster. You need only look at the big nuke teams on Spider to see why.
Even if they were to just add 1% per level - getting each of your six items and three accessories with six-star glyphs would translate into 56% more crit rate and crit damage, which would then allow you to basically ignore crit % as a substat on all of your other items. That would then allow you to get the other four more important stats for your champ (ATK%/HP%/DEF%, ACC, RES, and more CD%).
You'd be running into arena teams with upwards of 250% crit damage and 100% crit, and still able to be tanky or fast.