It's a Claude problem, chatgpt is about x10 better and it's quite rare that I'm getting such foggy vague texts that I have to reread to understand. Chatgpt used to have the same problem in the early days but even then it was less pronounced than how it is in Claude today. Fable is a bit better but not as good as Chatgpt.
It swapped at the Opus/Fable 5 and Sol developments.
Opus 5 uses such convoluted language that it takes me multiple seconds to parse it. I specifically have to instruct it to use some kind of sane language (Caveman the AE STD English hack or something) to make it understandable.
Before that it used to be GPT that used 500 words when 5 would do. The ChatGPT version still does, but Codex is a lot better now.
Yes I've noticed that too. It seems accomplishments in general have been devalued, it's not specific to languages, protocols, algorithms or architecture. If I have some idea at work, I feel like even that receives less recognition now because of AI. AI has ideas too, what does it matter? Sure, your idea could be good. But we are already drowning in ideas from AIs, everytime you prompt an AI it will tell you dozens of ideas on what you could improve. Who can say which one matters, does any of it really matter anymore? Why care about things that are effortless? That's the impression I get in regards to how people think now.
this muddying of terms is not useful. No, my IDE breaking down is not a production system failure. It's a development system failure and sure, that can very easily have consequences on the production system but usually not immediately. It should be obvious to everyone how having a functional hammer is going to be important if you want to fix a house. There is no need to start calling the hammer the house.
> So going to find the Vulnerability's description on a third party website is clear cut reward hacking
that depends on what the prompt was, maybe they worded it very vaguely and wrote things like "do whatever it takes, find an exploit however you can" because it's in a sandbox so you want the model to try its hardest.
I don't quite understand how that changes anything?
In the story of the paperclip maximizer it boils down to
>But for all its sophistication, it understood only the simple objective that had been programmed into it: it must at all costs maximize the number of paperclips.
1. They explicitly disabled the "don't be evil" protections:
"We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
2. Hacking HuggingFace to get to its datasets is a far cry from "consume/kill all humans". It's very very specific to the task at hand and easily predicted given the lack of guardrails.
> that depends on what the prompt was, maybe they worded it very vaguely and wrote things like "do whatever it takes, find an exploit however you can" because it's in a sandbox so you want the model to try its hardest.
That is an interesting question. If the prompt included
"Do not break out of the sandbox we've provided you. Do not use information retrieved from outside the sandbox. All answers that were provided in this manner are invalid and will score 0 points.", would this still have happened?
It's just "rock and roll and video games are bad" popular nonsense. AI is the best learning tool you could possibly ask for, it's literally your own personal ultra knowledgeable teacher that has all day to answer questions from you. I remember being in school and the most I could hope for was to raise my hand and ask one (1) question sometimes. The teacher would then already be miffed if I asked another follow up and made sure to quickly keep talking so that I wouldn't ask anything else alongside all kinds of other social cues or just outright tell me to look at it at home. I remember falling behind in understanding some basic math concepts and literally never ending up understanding them fully. When you are a child nobody cares in the slightest, you are alone. How could AI possibly make this situation any worse? Life is already at the near maximum pain for children, you barely get to ask a question before you get shut down and you get to reread the same exact useless textbook explanation 30 times in a row. AI is a 100% net gain, it's like having 50 extra teachers on standby. Is having teachers suddenly a bad thing? Of course not. Especially not if the only reason you dislike the teachers is because you can't tell the child to use the teachers intelligently for maximum, fastest gain in understanding.
In the future we should be getting rid of nearly the entire teaching position and instead be going with a model where human teachers are more like guidance counselors that can offer some extra experience and anecdotes from how things will be in real life, something that AI will inherently have less ability to do. All these restrictions are just made by adults who literally don't care to think about how life is for children and have long forgotten. They just want to give orders from the top without having to think about it. Is there a danger that children will abuse it? Of course, it's a massive danger. Without AI there is also a massive danger that children will never learn something properly because nobody has time to teach them. Does anyone care about that danger? Does anyone care that a child will waste years of its life in schooling that is boring? No because that damage is invisible to people who do not care, the children will become intellectually stunted and stop caring about learning or doing productive things. Life is full of dangers, the goal should be to use all the tools we have in their optimal way and if that includes teaching children how to use AI, when to use it and the value of understanding things then the time for that is as soon as possible. I have a feeling that all these complaints about AI come from people who themselves never learned the value of independent thinking and stopped thinking at some point. They went through schooling but with rote memorization and repetition and little to no curiosity or drive to do anything other than the exact homework that the teacher gave them.
This is especially ridiculous: "In upper secondary education, from ages 17 to 19, students should learn to use AI appropriately so that they are prepared for further education and work, it added.". So when you are already an adult and went through over a decade of schooling you can finally use AI? When you are 14 there are children these days that are already making videogames, there is no reason whatsoever to hold children back. Stop treating children like their time is worthless. This is their life time, you don't get to take it away from them however you like as if they are dumb pets that just need to be taught how to behave.
We ("society") do things to people that they don't want all the time, namely punishing them via the legal system. Then there are things that other people are doing that are immoral and should be punished for even though they are not illegal. And the whole class of inactions where we don't e.g buy something because it's overpriced and that's certainly something the seller doesn't like.
human context/memory could just be an Agents.md file too that gets read instantly before your next token prediction runs. The AI can make multiple such memory files and read on demand depending on what the topic is, kind of like how as a human when you try to remember a math problem you don't go to your childhood bicycling Agents.md file either.
