> So how exactly is Anthropic and OpenAI ever going to pay back the trillions that they plan on spending?
It's really simple: if they truly get to human-level AI (or even superhuman AI), then money and debts no longer matter, since our current economic system will be obsolete. They are betting everything on this outcome.
I don't know if they will manage to do it before their debts have to be repaid, but considering the rate of acceleration in the past few months, there is a non-trivial chance that they will, IMHO. We will see.
I would perhaps agree with you just a year ago. But now I am not so sure. It is clear that scaling up Transformers still leads to significant improvements, and they are now solving math conjectures and finding real vulnerabilites in software. We don't really know where the capability ceiling of the current approach is, and anyone telling you that we know it is lying to you.
> We don't really know where the capability ceiling of the current approach is
We do know, that the ceiling is below AGI. And it's not a matter of opinion - LLMs can not achieve AGI due to their design. Anyone telling you otherwise is lying to you.
And it doesn't matter how many or how severe bugs they can find, because it's not about what they produce, but how they produce it.
> We do know, that the ceiling is below AGI. And it's not a matter of opinion - LLMs can not achieve AGI due to their design.
[citation definitely needed]
> And it doesn't matter how many or how severe bugs they can find, because it's not about what they produce, but how they produce it.
AGI is defined by the practical outcomes, not by the way the outcomes are achieved. You have no way to know that scaled-up Transformers predicting the next token will never result in human-level intelligence, since we currently have no idea where the ceiling of that approach is.
If you're not even familiar with how LLMs work, perhaps you should restrain yourself from confidently talking about this topic until you educate yourself. You're only spreading misinformation.
> AGI is defined by the practical outcomes, not by the way the outcomes are achieved
That for sure would be a very convenient definition, especially for all those AI labs trying to convince investors that they achieved AGI. Unfortunately everyone knows, that knowing the right answer isn't the same as knowing where that answer came from.
> If you're not even familiar with how LLMs work, perhaps you should restrain yourself from confidently talking about this topic until you educate yourself. You're only spreading misinformation.
I am very familiar with how LLMs work, and I am telling you that there is no consensus that they cannot achieve AGI in the machine learning community. Some people think so (such as Yann LeCun), others disagree. We just don't know yet.
> That for sure would be a very convenient definition, especially for all those AI labs trying to convince investors that they achieved AGI. Unfortunately everyone knows, that knowing the right answer isn't the same as knowing where that answer came from.
No need even for "human-level" per se. It just has to be useful enough that it upends the current economic order, which it's already well on the way into doing.
1. they still have revenue though. it might not enough to cover all the r&d but it is surely enough to cover the hardware cost.
2. people tend to ignore this, but the salary budget of a US frontier lab and chinese frontier lab is nowhere comparable, the first can easily outdone the later by 100x.
3. us labs, like other US style startups, always throw ton of money to capture the market. I don't see the chinese company doing the same scheme at all.
so, surely chinese AI providers also lost money making new models, but they are not spending nearly as much as US ones.
>1. they still have revenue though. it might not enough to cover all the r&d but it is surely enough to cover the hardware cost.
>2. people tend to ignore this, but the salary budget of a US frontier lab and chinese frontier lab is nowhere comparable, the first can easily outdone the later by 100x.
Both arguments make it seem like there's a double standard for american vs chinese AI companies, where american labs are held up to strict standards for profitability, but chinese labs get a pass because [insert handwaving about how some aspect of chinese labs is different]. Let's do apples to apples comparisons here, what are both sides' run rates and revenue growth prospects?
>3. us labs, like other US style startups, always throw ton of money to capture the market. I don't see the chinese company doing the same scheme at all.
Right, instead they're releasing their models for free so competitors can undercut them on inference. American labs' prospect of "there are open models 90% as good but cost less" might seem bad, but chinese labs' prospect of "there are companies offering the exact same models but aren't on the hook for r&d spend" seems even worse.
> there's a double standard for american vs chinese AI companies
Not really, in a way. Things just cost far more in the US than in China; has pretty much always been the case, far back as I can recall. The Chinese state heavily invests in anything it wants to succeed at, and it has the resources to throw. Cost of living is generally wildly lower in China, along with salaries (although it's also pretty location- and role-dependent).
Overall I'd say labs are far cheaper to run in China than in the US, in more than just from the finances angle.
Literally nothing is known about how these companies are financed. The usual story is that Moonshot was chosen when 'Mythos' occasioned a huge state crisis. China is the ascended masters of insane amounts of capital poured into whatever the state takes into its head next.
By charging $$$ like they do now and having a non terminal business model.
PRC AI have lower opex and capex, i.e. export controls means they couldn't be trillions in the hole on inflated hardware in the first place. They only need to extract a few 10s of billions from domestic market have a healthy runway. If investors/gov wants to throw in a few billion to treat as utility, whatever, it's still rounding error.
Dirt cheap Chinese solar is a competitive advantage just sitting there waiting, but instead the US is trying to revive coal, restart a grossly ineffective small scale nuclear system with immensely bad fuel utilization, and spending billions to cancel renewable projects that were already approved. I'm so tired of these insurrectionist dog traitors to this country I love.
just like there were mistrals, coheres, llamas, etc, there will be new deepseeks and moonshots if those ever flame out (worst case, given out at cost by google, meta, alibaba or etc)
OpenAI and Anthropic are already in a ~200bil hole from previous model iterations and are committing to trillions of additional spending
OpenAI spent more TBPN than kimi spent on training K3
They are by all accounts, not. Z.ai for instance is a public company according to wikipedia. Moonshot AI is private but all their investors are private companies. Alibaba, as we all know, is a massive publicly traded tech conglomerate.
Moreover even if we take the more charitable view that they're controlled by the CCP, and therefore will continue releasing models for free, that seems as questionable as the prospect that private investors will continue shoveling money into anthropic/openai.
The core employees of z.ai are billionaires, because of the equity given to them in the past, which in their system is not antecedently evaluated. Much of the past annual expenditure of OpenAI has been the same, handing out equity - but because of the different legal system it is given an evaluation and listed as expenditure. Meanwhile the expenditure on compute for training and inference are apples and oranges again as the state is all over this with moonshot and z.ai and so on .
China has "classroom game capitalism", where companies can play the game, but the teacher still has uncontested, absolute, unilateral control in everything and anything. All the parameters of the game are managed by the teacher, and the teacher is the one who creates the foundations for the direction they want the game to move in.
Don't forget, Jack Ma of "publicly owned" Alibaba, had to go into classroom time out after seemingly forgetting that its classroom capitalism and not real world capitalism.
Shoveling 70% less money into a money pit is still shoveling money into a money pit. Not to mention that at least openai/anthropic has better prospects of making back the money because their models are proprietary, and won't be cannibalized by other companies serving the exact same models.
By charging $$$ like they do now and having a non terminal business model. PRC AI have lower opex and capex, i.e. export controls means they couldn't be trillions in the hole on inflated hardware in the first place. They only need to extract a few 10s of billions from domestic market have a healthy runway. If investors/gov wants to throw in a few billion to treat as utility, whatever, it's still rounding error.
I expect they're going to fight each other to become the vendor of record for the government, and whoever wins will get bailed out. This is one area where they don't have to worry about competition from Chinese models.
Its more Google Amazon Meta Microsoft who are spending trillions. They will be fine. So will Anthropic and OpenAI. Nvidia will presumably survive. The losses are all the real estate interests and contractors and contributory hardware companies etc.
I think most of us can see that the level of corruption we are seeing with the administration is quite historical. You can’t simply say “both administrations engaged in corruption” and consider the matter closed. Scale matters.
I've been experimenting with a few settings in my 4090 , and if 3.6 run at 90-110 tps, 3.8 staya below 80 tps and it's most often at 60 tps.
I'm using flash attention, mtp speculative decoding (n=2). I've looked at the club-3090 repo, but haven't found anything meaningful but get back to 3.6 performance
1. it's built for async
2. runs everywhere
3. interpreted, making it fast to iterate on
4. decent performance
5. most popular language, llms are decent at writing it
JVM apparently has the disadvantage that nobody under the age of 40 wants to touch it anymore. I admit I haven't worked in it in 20 years, but I do think it's a marvel of engineering and unfairly maligned. It used to be my career but I wanted to be closer to the metal.
Having Oracle's tramp-stamp on it may have been the final kiss of death in terms of totally-superficial "coolness" factor.
The JVM has a fixed size heap which for me it is wasteful.
IMHO, Microsoft made the correct approach on .NET.
For LLMs, I prefer C# and C++ instead of TypeScript, JavaScript or Python as the static + compiled language factor keeps the coding agents on track. Plus, they have a true threading/async implementation.
It's only appearing wasteful if you're not understanding how memory management works on modern operating systems. It's not wasting any RAM at all if you pay attention to RSS vs VSS.
The actual physical RAM is still entirely available to other applications. It's just made the OS know it might want that many pages. Until there's data in the pages, they will not count towards total RSS.
It's the kind of things some sysadmins used to gripe to me about and I would question whether they should be in charge of a machine at all.
To repeat: just because an application mmaps a large region doesn't mean the OS has actually given it all that physical RAM. It's merely made sure the pagetable knows about it.
I know how mmap works. The JVM is/was terrible on freeing allocated memory though.
If the program is actively using that allocation, that's fine. My problem is with the runtime hoarding RAM when it should have been freed after GC back to the OS.
Then there's also the JVM not handling peaks well because it hit the max heap size, while you still could rely on the OS doing its job to shuffle stuff to swap temporarily. I still see JVM OOMs in my $dayjob's product while the OS has plenty of free physical memory. It is stupid.
I mean, we have malloc() and free(), they are in the stdlib for a reason :)
The JVM seems to follow a philosophy where it assumes it is the only process running besides PID 1, which is valid for some scenarios, but not for others.
Well, you either spend energy freeing stuff or you waste some memory. It's a tradeoff, that you can trivially set with a single flag in Java.
In pretty much every other runtime's case you are stuck with whatever their GC uses, and almost every other GC is far less advanced than the JVM's implementationS. And manual memory management is not free of tradeoffs either, e.g. RAII can have pretty long destruct chains in both C++ and Rust.
Even when I was at Google (2011-2021) it was slowly falling out of favour and by the time I left, Go was fairly rapidly taking its place as the "garbage collected managed language" choice.
Not saying that's a good or a bad thing. I still think the JVM is remarkable.
and you could say the same if not better from C#. But those are now becoming niche languages and ecosystems. One for people around microsoft, azure, etc. The other around oracle solutions. There are still pretty interesting projects around it any of them, but they seem to be losing mindshare against other languages.
An additional benefit of interpreted, I think, is to make plugins easier to distribute and incorporate. With a compiled language you’d need message passing or something.
That actually don't like you're describing Python. I've been working on a couple JS/TS projects and it's like the models I use (Claude Sonnet and DeepSeek v4 Flash) continually struggle to do coherent work; I have to always keep close watch to reduce sloppiness. I go to Python and it's smooth sailing with minimal prompting (and reduced token burn) for acceptable outcomes.
aren't 3 and 4 a tradeoff though? Yes you have 3 but "decent performance" cannot be an extaled value as compared to "runs everywhere". If its used as a counter balance to 3 then it shouldn't be its own unique point basically saying 4 is true despite 3 in this case.
This line of thinking I feel like assumes it's the only program running on your computer. Using less of my CPU and memory means my computer can do more things in parallel, or even run more instances of the harness. My laptop is sweating when I got 5+ claude code sessions running.
I haven't monitored CPU usage so closely, but seems to get heavy with basic tool calls and editing. Memory usage definitely is out of hand, have an idle session right now eating 500MB
yeah fair enough, my entire point is not about the application itself but the contradiction on using superlative terms for all points but a compromising/normal term for one. Like if performance is not revelant why include it in the list of benefits.
decent performance, lol! compared to what? a shell script? "i'll only take up 200MB of disk and 4GB of RAM to output flickering text on a terminal. boy this is high performance"
fast iteration is for POCs. once you have the app built and working, you need performance and stability much more than fast iteration
Yeah some real main character energy from Dario as usual.
I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.
Oh. I thought a whole lot of the point and discussion of this post was AI being potentially weaponised and becoming means of economic and military control and coercion. Or did I really miss something?
Yes and no. Yes because it's more about oil than AI, no because AI is the oil industry and their supporting vested financial interests' (not so secret) plan to keep demand for fossil fuel high if/when the climate change deniers lose.
"We would love to use green energy, but all the batteries and solar panels come from China and China is evil, and we need all the energy we can get to run the data centres we need to spy on our citizens so they don't revolt once the environment is literally on fire, we can't feed them, provide enough energy to cool them, and refuse to build enough housing to house them."
Yeah I saw this in Cambodia when I lived there. Sihanoukville had become a Chinese enclave almost completely.
Different to the way that French colonialism worked, though. Less direct government, more influence of existing power structures and respect for the local government.
There is definitely an argument that this is plain business investment - China has a lot of foreign currency to invest because of its trade surplus, and there isn't the opportunity within China to invest it all, so it is engaging with trade partners to invest in their economies so that they can increase future trade with China.
You can also make the argument that this is not benign and China is trying to create control over foreign governments with this investment.
I'm kinda "both can be true, but either are better than how we did it"
> Reversing last year’s trend, a slim majority of ASEAN respondents selected China (52.0%) over the US (48.0%) if the region were forced to align itself with one of the two strategic rivals.
I’ve only been to Vietnam but the answer there is it’s… complicated.
They’re culturally part of the sinosphere but of course constantly living in the shadow of your much, much bigger neighbour to the north does breed some ill feeling.
The USA on the other hand, well, they’re not particularly popular either for obvious reasons
The irony is much of the China-ASEA acrimony seems to be over China's aggressive territorial claims in the South China Sea.
Which was a dumb move because, given Chinese economic superiority, they could have just quietly negotiated 99 year military base leases and oil/gas extraction with the UN-recognized territorial owners.
Same outcome, less bad blood.
Seems a bit of an own goal to militarily force the issue and antagonize its neighbors, who it's trying to convert to its sphere of influence.
Yes the whole "wolf warrior diplomacy" stuff in the late 2010s was massively counterproductive for China and just pushed everyone else in the pacific back into the arms of the Americans.
The mask (if there ever was one) slipped and they revealed a lot of information they probably should have kept hidden about their intentions for no real reason.
However they did eventually realise this, they've completely changed tack and they're having a lot more success (helped by the US adopting their own failed policy).
You're right though, I'd imagine politicians in SEA countries haven't forgotten
We don’t want anyone to colonize us just like you don’t want your country to be colonized. China offers business opportunities while the USA and Europe seem to always look down on everyone else and impose their own values on others. China does not give a shit about how we run our countries as long as they can trust that deals will be honored. We don’t want to be taught how great your democracy is or whatever, we want our countries to become better by our own efforts, with fair deals when dealing with other nations.
Yes, of course, sorry for the expression. I just mirrored the expression used in the parent comment. What I mean is that the US have caused a lot of pain in Latin America.
And the other half wants to be China's instead. Clearly they are a strategic partner when it comes to AI technology, as for other areas I'm not convinced they would be an improvement.
They've mostly been concerned with resource extraction rather than colonial exploitation like the West. And in fact, they've ditched most of their infra and tech projects in Africa, because of inherent instability in dealing with tinpot dictatorships.
On the other hand, every one in Asia is wary of too much Chinese presence and influence.
Single counterexample for single counterexample: Belgium's Leopold - not sure if Western colonialism is so much better (and notice I didn't even bring up "western India")
Not so much better but better in general compared to the middle eastern/eastern colonialism.. westerners have good competiton that the atrocities will be self corrected by itself but middle eastern/eastern colonies needed external western support like Korean war, hiroshima/nagasaki.. main example I can think of opposite is china under deng but there too deng had become relatively capitalist to fix the issues.
Yeah for the ones who get colonized, western ones were better than eastern ones. Japanese ones would've made Britishers look like kids when they were occupying eastern India.
You just put Russia and Japan as "Eastern"; they have nothing to do with each (and I would consider both as more Western than Eastern regarding imperial/colonial matters) other nor with China. China is a massive country that has always been massive and rich. Their style is more like the tribute systems they had before, as it is being observed in the BRI and other projects.
I understand China but in no way were japan and Russia close to western. IJA, MAO and soviet union makes westerners look like chuds. Russians still have the wagner group shit which is nowhere near what western countries have currently.
First, sadly, the west is full of similar examples, like Leopold II in Congo, Nazi Germany, the Bengal and Irish famines in the British empire, transatlantic slave trade... Not to talk about similar genocides perpetrated by third nations but with the support of the west like the Indonesian mass killings of 1965-66. I see quite a lot of similarities. The West is not a moral example of anything at all, but the opposite.
But second, Russia is not Eastern, it was more European than anything else until the Soviet Union. And Japanese Empire drew inspiration in the Western nations, even if they took it a even more horrible twist. So blaming this on "the East" is racist and reductionist.
Western countries have good internal competition that the atrocities they commit can be stopped by themselves whereas middle eastern/eastern ones needed external support for reformation famously hiroshima, deng reforms, soviet dissolution, korean war.. west is superior in terms of liberalism and capitalism that any successful country you can think of now has roots in utilising its ideas.. this is not to downplay atrocities committed by westerners but to think that somehow western nations are worse than eastern is being in denial. Russia was more european before the communists started their revolution and made themselves anti-west. They support many kinds of anti western movements like their war crimes in Africa, support for iranian/Islamist regimes, support dictatorship of china etc. these are all happening right now.. you are talking as though churchil wantedly cause famine in India when there's limited resource and he had to take care of his country before shipping to India and there was some amount of corruption too in Bengal at that time.. I'm not saying that they're blame free but it's a complex issue with multiple causes..
even the AI that was trained on mostly using western data and infrastructure that chinese models distill and all..
I did not say Eastern are better, but rather that they are not worse. And having good economies (that started with colonization, slavery and imperialism) has nothing to do with morals, rather the opposite. You can be extremelly efficient at exploiting people. Western genocides are complex, but Eastern ones are pure evil? That is what I mean, this is just racism.
Then you mean they are equal right? Which is also blatantly false. Capitalism and liberalism alone would compensate for many of the drawbacks caused by western atrocities. You talk as though colonisation, slavery, imperialism was only done by western when eastern/non-capitalist countries do that routinely since ancient times. Those are human nature and any system u can imagine has its traces rooted in ancient civilizations. I'm also from India and I know what the British did was bad but even before that there were issues within ourselves. But it was nothing to what the Japanese alone did when they occupied just a few parts of India. Exploiting people lol.. as though there aren't different levels of exploitation between now and then and the freedom to choose to work in different places now.. this is just plain old Marxism.. i must have internalized racism jz cz West is better in terms of liberalism and prosperity and reform than East..
Even with all the downsides of being colonized by Britain, it was still far better than being colonized by Belgium. Heck, being colonized by pretty much anybody but Belgium was better than being colonized by Belgium. Read up on the Belgian Congo sometime: it was a parade of horrors.
GP was making a relative comparison; pointing out that both were bad in absolute terms does not negate the point. -2 is still greater than -17.
For the love of God, you guys are brain broken.. there's a limited resource and churchill didn't want to provide to India because of that and add that there's some major corruption in Bengal at that time that caused it.. you guys don't see nuance at all..
India had a system in place to handle famines, Brits decided to do away with it setting the stage to allow millions to die. Was it also just corruption that caused millions to die during the Irish famine?
How do you not know about the policies like prioritising calcutta and ignore other regions of Bengal, surrounding states restrictions to supply food to bengal, price controls, local shortages + 1942 cyclone + british prioritizing their own citizens all caused such issues.. jz doing a cursory research would have provided you these details.. if churchill wanted to intentionally induce famine, how does that explain him asking FDR to provide as they surplus and why FDR had to not do that.. please jz do a cursory glance regarding irish famine too.. some folks jz can't do pros and cons of their own views as well which is a shame..
What a disingenuous question. Par for the course for the likes of you. Just because I view the west as less evil than the east doesn't mean we like to get colonised. These are my views:
1. We would not like to get colonised.
2. But history shows that the world was a place where you conquer or be conquered.
3. Mughals first invaded and as they were falling then, Europeans occupied, which is par for the course if you view the world history at those times.
4. It would've been better if they left early but they were better than japanese imperialists and nazis who bose was cozying up to.
5. The japanese were super violent to indians. I can't imagine the nightmare scenarios if japanese enacted what they did in nanjing/Andaman/few east indian places but they were weak at that time and British + indian army repelled them which is good.
6. After colonisation, nehru, being fabian, failed to open up the economy and he sympathised with soviets/china which exacerbated many economic and military issues at the border.
7. 1991 reforms changed india for the better and it would be better even if they pivot to green energy so that they can pivot off of Russia and Ally with west which biden and dems were trying but now trump worsened the relationship.
I keep hearing this and I fail to see any reason to believe that it will be the case. Any empire, in the history of our species, has always had an initial "inward-looking" period prior to becoming a full-fledged imperialist oppressor. Edo-period Japan into Imperial Japan, for example. Rather, what I think any budding empire needs a period of growth to become a large enough fish and also to delude its leadership and population into the convenient mindset that their flavour of imperialism is good and justified ("Hakkou ichiu", "The white man's burden", etc.).
To me, the PRC is at the very end of that process and I recommend anyone doubting this to go and read conversations and listen to the words of the populace that is turning increasingly nationalistic and you will hear the same old tales of revanchism and exceptionalism that we are used to hearing (during my recent visit, I watched the morning news every day for about a week and without fail a military inspection, new ship, new plane, etc. was presented each day). In addition, I think those outside of Asia are very much shielded from the early signs, but go and read about PRC influence and tensions in South Korea, Japan, RoC, Philippines, Vietnam, Laos, Myanmar, India, Pakistan, and Tajikistan and you will see something rather different than "inward-looking". To me, here the PRC is simply testing the waters for the extent of the influence of other powers and how far it can go. Likewise, we are seeing overseas naval bases being constructed which sure is an indication for a desire to project power outwards.
I want to believe that this time it will be different. I really do. Apologists around me say "It will only be Taiwan and the South China Sea, then then it will stop." and I would love to believe them. But can anyone truly internalise the narrative that international utopia will be spearheaded by a deeply authoritarian state that controls information like no other (and gladly exports that technology), disappears its own population at will, spins an increasingly strong nationalistic narrative, etc.? No, sorry, I think the "inward-looking" narrative is simply a convenient way for us to close our eyes and find comfort in ignorance, rather than in facts.
You may well be right, and it's really only that China has been more inward-looking recently, and given the chance it will spread its true colonial wings.
Not sure if "colonial" is the right word though, as we should be careful to think that oppression always takes on the form we have seen in the past (US imperialist oppression for example did not take on the form of the colonialism that preceded it) . Also, be careful with "true" there. I do not think this is some sort of subterfuge, but rather an inherent weakness in us as a species. Vest power in anyone, and before long their morals will give way and an oppressor will be born.
The way I look at it, until Deng the PRC's economic policies and internal instability kept it from growing at the pace of many of its neighbours. Then we had an era of intense growth (which is still to some degree ongoing). However, as a reaction to this era Xi and others needed a narrative to counter the increased corruption and a new unifying myth to replace the cult of growth as the economy would stagnate at some point and could then call into question the authority of CCP to rule. Their choice of nationalism is what scares me and I know PRC citizens (even CCP members) that share this perspective and would rather have seen the Shanghai clique to have remained. It is possible that in this alternative reality we would still end up with "Imperialism with Chinese characteristics" ("中国特色帝国主义"?), but I chose to believe that at the very least the chances of this would have been smaller.
So let’s just guess what China will do once it becomes dominant and act like it already did whatever we conjured up they will do. That’s how you get preventive wars that destroy civilizations without any actual basis on reality.
No, I do not think that is a fair portrayal of my position. Just like I will not say that your position is simply appeasement and hoping for the best. One must always be charitable in a discussion with strangers whose positions you do not fully understand.
Rather, I think we should look hard into ourselves and what is good and bad about the current world order. For example, the people of the RoC have the right to determine which direction they want to go. Regardless of the chauvinistic rhetoric coming out of Beijing. We should all stand up for this, because it is a universal right that we want everyone to enjoy. Similarly, we should push against the Eleven-dash line and support the 2013 ruling. The list goes on and I am sure these issues can be resolved without an outright war if we are careful, yet firm, in our beliefs and also diplomatically preemptive and thoughtful.
Being concerned about PRC imperialism should not be mistaken for the position that their people should "know their place" and be suppressed back to the stone age. They have the right to enjoy the fruits of their labour and pursue happiness, just like everyone else. We simply must be there to remind them (just like we must remind ourselves) that the course of humanity is a collective project if we are to stay clear of the darker sides of our nature as we venture together into the future.
Do you believe the UK has the right to rule over the Maldives?
And the USA has the right to have a military presence in Guam (and nearly all of the Pacific Ocean for that matter)?
The Chinese claims on the South Pacific Islands seem really similar to me.
Taiwan seems like a wholy different matter. It was united with China for hundreds of years until the Japanese colonialists took over. It united again with China after WWII but split up after a few years because of the Chinese civil war (notice it was an internal war). I think it's just fair that China wants to re-unite with Taiwan, though I definitely don't support a military takeover. Hopefully a solution similar to what was done in Hong Kong can be found. The Tibet region had a similar history and it's definitely unfortunate that China had to use military force to bring it under its own control (arguably completely unnecessarily - China would be just as strong today without it), but it's kind of understandable in the context of the time (China was trying to recover from centuries of being preyed on by other nations).
I am saying this because I can't agree that China is acting imperialistic - it's basically claiming sovereignty over its own historical lands - which were taken away from them by force by foreign colonial powers. But I admit that, if you go back far enough, nearly all land was once taken over by aggressors - the USA being just a more recent example of that.
Anyway, thanks for not being an absolutist and trying to understand the "other" side (I must acknowledge I have no relation to China whatsoever, in fact I am from South America and live in Europe).
> Hopefully a solution similar to what was done in Hong Kong can be found.
Citizens of Hong Kong lost their right to free speech and their ability to select their own leaders. If you publicly criticize Xi Jinping in Hong Kong you will go to jail. If you advocate democratic rights in Hong Kong you will go to jail.
China's claims on Taiwan are ethno-nationalist. Ethno-nationalism should be rejected in all forms because it is a rejection of fundamental human rights. Taiwan deserves the world's support because it is functioning democracy. That makes all the difference.
I do not have a stance on the Maldives, Guam, etc. as I lack enough historical and current context. What I always do is try to derive positions based on the human right to self-determination.
About Taiwan. I find the claim that since an absolute monarchy controlled the island 150 years ago, that then a government which was the result of two (is my count correct?) revolutions overthrowing that monarchy and then another government, a government which failed to conquer the land by military means by 1950, and now after people have lived independently for over 75 years (over 25 of which as a democracy) that said government has any right to dictate how said people should live to be simply absurd. If the people of the RoC wants to join with the PRC, that is for them to decide through their own decision processes. Historical claims like this may make sense for unpopulated tracts of land, but here we are talking about the rights to self-determination of more than twenty million people of which the vast majority were born well after 1950. This would set a terrible precedence and, frankly, it feels akin to how emperors and kings of old asserted their "rights" and not how we move towards a more just world.
Also, Hong Kong? If anything, Hong Kong shows that the PRC is a poor custodian for a pluralistic country with multiple parallel systems. I once thought it reasonable for Hong Kong to be "returned" after the historical travesty that were the Opium Wars, but I have heard enough first-hand accounts of the suffering and tragedy that unfolded over the last ten years to reconsider whether I prioritise history over the people that are alive here and now. It was not that Mao and Xi "unfortunately had to use force" against Tibet and Hong Kong. These were calculated choices on their part and history shall judge them the same way we judge any other oppressor for their moral failures.
Africans are more indebted to Western multilateral organizations and private Western bondholders than to the Chinese.
The idea that Africans are being finessed by the Chinese is racist and stems from a "we know what's good for you" imperialistic lens.
I find it unlikely to be a duoply of hegemony for long, some countries to watch are Germany, Japan, India, Nigeria, and Brazil. You could broaden the geography to continents were I expect major players to emerge on each.
We are only a few decades since the "end of history" and much has changed. What do things look like beyond 2050?
Brazil is basically the world's soy farm. It's at least half a century behind the times. I still have no idea how it managed to insert itself into the BRICS economic block. The notion that it's on the same level as China, India or Russia is just comical.
All of the small miracles you listed happened in spite of the culture, not because of it. It's also not a coincidence that both are deeply linked to the most successful brazilian enterprise: the brazilian government.
I'm trying to avoid getting too deep in these Brazil tangents so I'm gonna leave it at that. Anyone who cares enough to know what I think about the subject can just look up my comment history.
Timothy Snyder, in On Tyranny, has a chapter about staying in touch with friends from other countries, and while we are not friends, this thread is in that spirit and I have learned things from you, so thank you!
A lot of AI researchers were just in Brazil due to ICLR. A number of them got robbed in broad daylight in this supposedly nice part of Rio. I think the assesment of Brazil is accurate.
The average German in Nazi Germany probably didnt think the Nazis were too bad either (of course the average citizen wasn't Jewish, gay, communist or other undesirables). That the average Chinese think things are “just fine” doesn’t mean they are.
Unlike Nazis, I don't see that Chinese communists have expansionist goals to seize territory. Nazis wanted all of Europe. Do Chinese communists want all of say southeast Asia?
To be clear I don't think the government of China is as bad as the Nazis were. I'm just saying "the average Chinese in China thinks it's fine" doesn't mean it's fine.
There is no good imperialist power, there is no capitalism with a human face, there will never be. The UK murdered millions in India, Ireland and all around the world in their time. The US did the same. The only reason China isn't openly doing the same right now is because they are still the underdog and they still need cordiality to get through the door, just like the UK and the US did in their time. The solution is to rid the world of imperialism and capitalism as a whole. As long as there is a profit motive running through everything, international relations will be in the form of wars (military or economic). Only under socialism can the people of the world truly pool their resources together to build up instead of destroying each other.
You would have to eradicate the desire to command other people. The wish for power which is different from the wish to be praised, as forcing somebody to do something is different from convincing somebody to do something.
There are no good imperialists but it's a spectrum where western capitalist colonies fare much better than eastern communist colonies because capitalism and democracy is the engine for economic growth and western colonies fared much better than eastern ones. What's a better system than capitalism for imperialism? Socialism? Weren't they also pretty brutal imperialists? Japanese imperialists alone make western ones look like amateurs.
> western colonies fared much better than eastern ones.
Sure, Iraq and Afghanistan definitely benefited a lot from being bombed, occupied, and then handed over to even worse tyrants when the West got bored.
No, sorry, but "capitalism and democracy" is not the right answer everywhere, and when they're not, pushing them by force is no better than forcing communism. In general, forcing an incompatible ideology on people historically and culturally opposed to it will end in tragedy, no matter how great that ideology may be.
As tho they weren't shitholes before the intervention and those are the only examples when they themselves treat their citizens like trash by oppressing themselves and they got bombed because they're bored? Lol Idk if this is a parody of left wing thought or not.. surely you can make better arguments than boredom arguments and call them bogus..
How are you not racist towards Arabs to think that they're not capable of liberalism? There needs to be some people at the right time and a place like al shaara seems to be doing incremental changes to reform syria.. pushing them by force is okay when they oppress their own citizens and threaten other nations with their own form of imperialism on sorrounding countries/allies.. are we supposed to fold and not help ukraine for example even though Ukrainians never got exposed to liberalism? It's to help and condition them to open to liberalism.. same is needed for iran but retard in chief can't do it properly..
> the West could retaliate by halting shipments of
China quickly retaliated last time by stopping shipments of rare earths and magnets. The West has no answer for this, really up the river without a paddle for such critical supply chain elements.
China controls way more things, from medical needles to pharmaceutical ingredient.
Lots of them can be made in the west or west friendly countries, but that takes time, money, infrastructure and good execution. Yes, identical to what is covered in China's belt and road initiatives. See the gap now?
I was led to believe that the US does have internal sources of these, but they are largely undeveloped. For years the processing could not economically compete with China so shutdown.
Which is to say, given internal subsidies, the US could eventually produce some on its own.
Dario is more of a threat to the US, in terms of advancements in AI, than China. In Dario's mind anything that can't be controlled competitively is a threat to Anthropic, so he positions his FUD strawman so that Dario doesn't have to worry about the competition. And then he can artificially inflate token costs so his IPO can happen. Dario doesn't actually care about ethics, alignment or availability of LLMs - he just likes to use those words to sound like he does. Yet we've all seen how Anthropic actually acts vs what they say.
The scary part very few are talking about is that every compute device is Turing complete. So everything from the phone in your pocket to a DGX Spark is a threat to national security now since, technically, every device can run any model (how well is not a question of concern when you start to argue hardware should be gated just the same as Dario likes to gate models). I mean, along these lines of thinking Linux should not be available to the masses! What if someone runs some code that's not approved by the benevolent dictator for life, Dario? People will say: that can't happen, but the reality is it already is. If everyone has reasonable access to compute to run models that are mostly capable comparative to burning Anthropic tokens, why wouldn't they? It's risk reduction and price protection. Yet we can't buy those systems because of future production already being purchased by these organizations.
But back to the models themselves... We played this game with Metasploit back in the day: many who had no clue claimed exploit tools should be regulated and only available for use by those blessed, illegal elsewhere (I believe the closest this got was the Wassenaar delegation in the US, but only through collateral inclusion of "cyber weapons "). Except in that timeframe the authors of these tools weren't advocating for protection. Today the world is fine, systems improved because of security FOSS tooling. The same thing will happen with LLMs. Unless, that is, Dario gets his way. I'm not a fan of Altman but I think he's standing back watching this play out knowing what Dario is doing: either he succeeds and OAI benefits or Dario ends up the Chicken Little of AI and Anthropic fails to launch (their IPO).
The reality is Dario is only doing this because this is a real risk to his business. China's constraints in building competitively have given them an advantage: they are doing more with less. And if you think that their distilling from US models was in any way anti-competitive or illegal, then I guess maybe "deal with it", much akin to Anthropic, Google and OAI's response around taking the (copyright) content in the first place with no repercussions.
People who don't work in the AI bubble don't care at all about any of these people. They could all be gone overnight and the world would continue to innovate, probably in a much more productive manner, without them.
> And if you think that their distilling from US models was in any way anti-competitive or illegal, then I guess maybe "deal with it", much akin to Anthropic, Google and OAI's response around taking the (copyright) content in the first place with no repercussions.
Exactly. The cries in favor of distillation regulation from the US AI companies ring hollow and fearful.
OpenAI and Anthropic didn't realize that distillation was going to be so (a) effective and (b) un-technically-stoppable at scale.
Now they're seeing their IPOs at risk and clutching at governmental straws.
Dario's argument is transparently working backwards from {protect Anthropic's economic model} <- {need government regulation} <- {justify government regulation via AI fears} <- {we love open models, but so sorry they can't pass regulation}.
If the rise of the web in the 90s taught us anything, it should have been that companies that take economic reality as it exists thrive, while those that predicate their value on regulation fail.
If distillation at scale works and is technically feasible? That's reality. Deal with it.
Don't confuse political limitations and lack of a theory of victory with the inability to achieve a goal. A blockade needs ships those ships can be destroyed.
I think the argument is that decentralization leads to deceleration because it means less centralized funding and data. Those are the two primary ingredients for accel.
The problem with the decel/accel rhetoric is that it lacks nuance.
If your worldview is “most of the progress is made by closed labs, then open labs fast-follow” (which isn’t implausible given the documented distillation of Fable), and further that open labs cannot make make meaningful progress vs the closed labs except by fast-following and that they won’t pick up the ability to make progress after the closed labs are gone, then driving closed labs out of business slows down overall progress.
I think it's pretty hard to hold that worldview: Anthropic couldn't ship a reasoning model until they copied DeepSeek R1's homework, and they've all copied DS-style super-sparse MoEs at this point too.
That’s a really good point. Folks really need to read the papers coming out of these Chinese labs. Every paper from the DeepSeek team has been a step change.
With slightly different cherry-picking, you could equally well claim that DeepSeek couldn't ship a reasoning model until they copied the idea from OpenAI's o1-preview, and they also copied MoEs from Google Brain/Jagellonian University https://arxiv.org/abs/1701.06538 way back in 2017, too!
But ultimately these were ideas floating around in the air, if one group hadn't done the experiment, someone else would have.
No, OpenAI did not publish how they trained o1, and at the time there was significant misunderstanding and belief in the research community that they were using some kind of Monte-Carlo tree search. DeepSeek figured out GRPO on their own. Similarly, while others invented MoEs, DeepSeek's ultra-sparse variants were extremely novel, to the point where the revelation of how efficient they were to train temporarily collapsed Nvidia's stock.
Regardless I think it's impossible to believe that most LLM research was done by closed labs that don't publish, especially Anthropic (who missed out on and copied two of the largest pieces of important research of the last several years), and that none was done by open labs like DeepSeek, and that the open labs are just copycats. It's quite clear that isn't the case.
Open Source models decelerate growth of closed AI. For people who think (or want) AI = closed_AI then that argument has weight. Good luck getting them to update their priors.
the argument is that we should all fold and let sam altman burn trillions of dollars on naive scaling and pay monopoly prices for their closed APIs until the models are good enough to be closed off for "safety" reasons so that they can take an even larger cut by competing directly with us
Open source AI is actually a lot less "powerful" than genuine frontier models, i.e. it has a much tighter inherent capability ceiling. This is "decelerationist" from a purely AGI-pilled point of view but it's actually great if you're worried about a capabilities arms race putting AI Safety at severe risk.
Kimi K3 is plausibly a lot less dangerous than a totally jailbroken ChatGPT/Gemini/Claude Sonnet (let alone Opus or Fable!) and it's quite deeply weird how no one seems to be calling for those models to be banned or restrained by further regulation. Why the double standard against the less concerning (but more efficient!) open weight models?
If you're targeting widespread local/on prem deployment which is what many open weight models are doing, that inherently limits your scale in terms of total model weights/inference-time compute compared to running in a few centralized datacenters. A centralized model will always be able to leverage a larger scale of deployment, placing it much closer to the genuine "frontier".
It’s a different topic but it’s correct imo. Open sourcing things is the best way to accelerate development.
Put another way, if you want to slow things down, put it behind a paywall, tag ideas ans “intellectual property” (meaning you’re the only one who can use it) and get the lawyers involved (injecting our slow legal system).
None of the above is a judgement call on whether development should be accelerated.
I know it's a hard ask on this site, but I need you to start parsing content and not tone. It was a helpful bit of context, even if it was a bit vitriolic.
The content was 'you need to get your head checked'. That isn't tone, that directly implying that if you hold that position, there is something wrong with you. It's rude and unnecessary.
Because discourse around tone isn't productive. You could wipe this whole comment thread, starting with the parent of mine, and lose exactly zero information.
Speaking of tone, the phrase "I need you to X" is super condescending. You're not his superior, and he's not your child nor your subordinate. He is not beholden in any way to what you "need."
Much better would be to just use good ol' "please":
> I know it's a hard ask on this site, but please start parsing content and not tone
Oh, I understand. It's the nature of the voting system of comment feedback. Reddit behaves the same way. Arguments become competitions to see who can inject enough vitriol while still maintaining a placid genteel demeanor. The one who "wins" (convinces the peanut gallery to upvote them) is the one who can avoid looking like they got mad.
It's why people can advocate for ethnic cleansing here, and that's fine as long as they word it correctly, but if someone calls them an asshole about it, they're flagged.
Really common in rationalist circles from my experience as well because they believe that true statements aren't always normative, and that, since their arguments are true because they're rational, their statements aren't necessarily normative. Begging the question, of course, but I see that in situations like Scott Alexander's defense of "human biodiversity" theories (the whole HBD moniker is itself an example of everything I'm talking about condensed into two words).
> Oh, I understand. It's the nature of the voting system of comment feedback. Reddit behaves the same way. Arguments become competitions to see who can inject enough vitriol while still maintaining a placid genteel demeanor. The one who "wins" (convinces the peanut gallery to upvote them) is the one who can avoid looking like they got mad.
Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit.
> but if someone calls them an asshole about it, they're flagged.
That's because name calling is against HN guidelines.
Just an aside since it's not clear: I think the asshole is the one using labels like "decel" (or even "MAGA" unless the person self describes). It's irrelevant if I agree with the rest of the comment.
It's OK to call people out for name calling while still agreeing with the rest.
It's problematic to require one acknowledge the quality of the rest of the comment when calling them out on their name calling.
Put in a less twisted manner: If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment.
>Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit.
Really doubt that but feel free to give an example.
>If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment
I think the only acceptable response is to answer his argument if you can read it. If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.
> I think the only acceptable response is to answer his argument if you can read it.
When indulging in a discussion with others, one has to realize that the universe of what people consider acceptable responses isn't limited to yours.
Yes, it's clear that's what you think. It's also the whole point of discussion here. Merely repeating it isn't supporting your perspective.
> If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.
He is the one who brought the anger into the discussion, and thus it became part of the discussion. Not addressing it is a symptom of not handling it.
Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world, and so I decided to study what an effective conversation. I took a multi-day course, as well as read several books on it.
All, without exception, point out that failure to address the emotional content is a bad idea and one of the reasons conversations become ineffective.
>All, without exception, point out that failure to address the emotional content is a bad idea and one of the reasons conversations become ineffective.
I am not sure how you could walk away from this comment thread and still think that's the case. Minus this digression which does argue about central concepts, the discussion about the original point is pretty close to the bottom of Graham’s Hierarchy of Disagreement.
I guess maybe a better question would be why you think pivoting a conversation about how open weight models are changing the industry into a conversation about whether a commenter was rude is a positive outcome? I would understand either ignoring him or answering his content, but I'm trying to understand how getting into back and forth about tone (back in my day we call that a flame war) is a good use of one's time.
>Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world
I've found it to be quite the opposite! Not rankling at someone's emotionally charged language in real life has been incredibly useful, both for interactions with friends and strangers, but also with family. The people I know who are miserable about visiting family at Thanksgiving feel that way because they are unable to stop themselves from getting into conflicts because their relatives made some emotionally charged statement. There are also plenty of situations where people come in hot and angry, and you can turn it around by simply letting their tone slide and trying to help them. Happened all the time when I tutored compsci students. They'd come in angry and surly because they'd been beating their head against a problem or concept, and a little patience and forbearance went a long way towards improving their attitude.
the only reason other labs can catch up is because the frontier labs can be distilled, and they siphon a % of the labs' revenue to reinvest into the next iteration
full accel would mean nationalizing the big 2 labs and locking in manhattan project style until RSI
(Edit: some great counterpoints in the replies. my view has definitely been changed!)
This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up.
The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone.
Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements.
It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases.
Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago)
EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford)
Appreciate this point. Back in grad school, I published a peer reviewed paper with all the source code and datasets used. Got heckled at a conference talk by staff from a commercial lab. They shouted, we figured this out five years ago, lol. Also our approach is still better. But they don’t release their source code or publish much, so no one knows what this approach is or if it’s actually better.
And years down the line, lots of other research labs used my code and cited my paper.
Oh this definitely happens all the time. I was an early employee at Clarifai, which won imagenet a year after alexnet and we were able to stay on the frontier for about 2 years before a bunch of open source models were matching our results. It was always some random PhD research project spinout or some random kid in Boston named Alec Radford.
We had a bunch of things that we never published that ended up being major research findings years later at top conferences.
OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation.
Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs.
being able to replicate it in the open means that there's nothing special about frontier models.
Frontier models would have to do something extraordinary or unique, or unreplicatable, because clearly there is no moat, and US companies are sitting on huge nvidia valuations and get surprised when competitors beat them.
On Policy Self Distillation and Active Learning. Anything that increases sample efficiency by providing a richer more dense feedback signal and is more efficient at exploration / sampling.
