You're putting the cart before the horse. The reason that performance has not improved in the desktop and gaming are due to hitting fundamental limitations in the design of hardware.
The PlayStation 3 has 256 MB of RAM, not exactly on par with a current PC gaming system, which can easily have 16 GB.
I would argue that the rising production costs to produce games that actually make use of ultra-advanced hardware are more to blame for the slowing of the graphics race.
You can always do things faster by doing things in parallel. It just gets much harder to program them.
Gaming PCs also have a general purpose OS on them. It's not an apples-to-apples comparison. One is a general purpose computer, the other is a device designed for a single purpose.
But that's beside the point: the games industry drives the graphics card business, but that is only a one part of the consumer computer business. General purpose processors are a larger part, and processor performance improvement is not the exponential curve it was a decade ago.
But the hardware IS getting faster! Each successive release of GPUs for example still brings big performance improvements, us PC gamers are just going from 100FPS to 120FPS with a new card. People are running games at 5760x1080 resolution across three displays with just one mid-range graphics card, with anti-aliasing (and all the other settings) turned up to the maximum. Current games just don't make current hardware break a sweat.
Graphics cards are getting better, but not at the rate they did, say, a decade ago, and graphics cards are a small driver in the consumer computer market outside of games.
As far as I've heard, we haven't hit that limit in the consumer space yet, as we're just starting to get close to hitting it in other spaces. Is this not true?
We have hit it in the consumer space. A powerful processor from three years ago is still a decent processor today. That was not true in, say, 2000. This applies in consumer computers and high performance computers.
For a long time, processor architects were able to increase the frequency, increase the cache size, and increase the pipeline depth (to allow for more instructions in flight at the same time) to yield more powerful processors. As Moore predicted, processor architects kept getting more transistors to play with, and they were able to make processors more powerful by making designs that were "like the old one, just more so." But they've hit fundamental limits in the design: increasing the clock speed and the pipeline depth at the same time means that you have to communicate the same amount of data over a longer distance in a shorter amount of time in the silicon. We've hit the point where it's not feasible to do that anymore.
Hence, multicore. Processor architects are still getting more and more transistors to play with, so instead of using them to make a single core more powerful (as they did for a long time), they're using them to make multiple cores. But you no longer get the "for free" performance boosts that you did when you increased single-core performance. Now we need to change how we make software to take advantage of this new hardware. And some software can't take advantage of this new hardware.