A big warning for anyone trying this out, don't use it in places where exact placement is required. Browsers (and operating systems) render fonts VERY differently. Just changing the cleartype-settings in windows can make the text placement completely off. Especially after a few letters as the error propagates the more characters you accumulate over one row. Might not be an issue if you only place one letter/icon, which is probably the use case here, but keep this in mind.
This is, I think, the biggest issue with abusing fonts in this way. As a workaround, icon fonts are excellent—but they are still undeniably a workaround. Unfortunately, SVG is the closest we have, and isn’t a viable solution either, for reasons other commenters have thoroughly addressed. I do have (what I think is) a better idea in mind, but I’m not sure how it would ever get adopted.
They are a "workaround" with long history: Windows 95 shipped Marlett for exactly this purpose of using fonts as well-supported vector graphics formats.
Oh, absolutely. It’s just a dealbreaker for me personally that fonts are monochromatic, single-layer, and difficult to position in web pages with precision and consistency. If those aren’t concerns, then icon fonts are a perfectly useful tool.