90+% of queries are probably so common/evergreen that, if a cheaper model made all the different language varients and ways of asking the question into one query, they could be cached quite effectively.
> 90+% of queries are probably so common/evergreen that, if a cheaper model made all the different language vari[a]nts and ways of asking the question into one query, they could be cached quite effectively.
As I wrote in some parallel post:
"When I look at my history with AI chatbots, 90 % of the conversations revolve around research-level problems in math and computer science. :-D"
For quite a few of them, creating a decent answer involved quite a bit of thinking by the AI, so I assume they are not the easiest queries to answer, and the queries are too obscure to cache.
Sure, but you don't represent the average ChatGPT free user in that case probably. I regularly do super complicated queries that Pro spends ages replying to, but yesterday I also asked if chilis are toxic, or just unpleasant, for dogs, and if other species other than flamingos get colored by the food they eat.
