Taubes actually addresses this point very carefully in both Good Calories, Bad Calories and Why We Get Fat. He cites quite a few cases of communities with high rates of obesity, and varying rates of nourishment and sedentary behavior.
One very interesting result is that some of the communities lived in incredible poverty and with significant physical exertion.
I'd recommend that you read the book (Why We Get Fat: And What to Do About It) and see whether he addresses your concern. Far from ignoring our sedentary society, Taubes makes a strong case for other factors.
You can easily get away with eating a lot of carbs; being physically active not only allows it, but often requires it.