So in terms of raw brainpower it's there, but I've never really been able to apply it consistently for whatever reason and have ended up very average. Still haven't really come to terms with that in my late 20s.

I've got a perfectly good life but I have this vague "what if" about if I had applied myself more.

For vllm the official recipe [1] (for another model in the same family) is this:

  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser lfm2

Recommended amount is 5–10g per day (5 pieces of 100% xylitol gum, 36% is already effective), taken after meals.

I have no idea, why in that context, the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time, which has to put an upper bound of long context support in some way. There's just more things to lose focus to (or, mismanage the limited attention heads resources - per token)

After every prompt the model decides "I have a weight of 1 to distribute between every token in my context". If you have ten tokens, each gets a weight of 0.1 ...

... except it's not that simple, because the LLMs don't distribute that "attention budget" equally. If your prompt was "where is Paris", then any tokens in context it can associate with Paris will get a greater share. If the word Paris is in your context, it might get 0.3 or 0.4 weight, and close by tokens might get 0.2, while other (unrelated) tokens get 0.03 or something.

Now, add lots of context, and you start to see the problem: more context = greater distribution of the attention budget. Even if the prompt is about Paris, and the Paris tokens get higher weights, they are only getting (say) 0.002 ... while unrelated tokens are getting 0.001.

All LLM "answers" are just math, computation, based on the context and those weights. If it can't "focus it's attention" because it's distributed among too much context, it's far more likely to miss the relevant tokens (eg. the Paris ones) and give you an answer that ignores key parts of context.

But again, there's no fixed number of things it can pay attention to: it's a gradual degradation of the chance of it seeing key info it should, based on the amount of context.

EDIT: Ok, I guess there are an internal thing called "attention heads"; TIL. They work exactly as I described (essentially I described a single "head").

Still, what I just wrote remains true: whether you have 1 head or 96 or ... however many your particular model has ... the fundamental issue isn't the number of heads. It's not like the LLM can pay attention to exactly (say) 96 things at once: each one of those heads has all of the available context "competing" for attention as described, and then the heads themselves "compete" to produce the answer (although not through the same mechanism; it's more like the weighting between heads is baked-into the LLM).

At the end of the day, the problem isn't "there aren't enough heads to pay attention" and the LLM hits a fixed limit. The problem is each head is trying to weight the relevant stuff, and the more stuff there is to distribute weights through, the less likely it will get it right (and then when all the heads are combined to produce the final answer, it misses key context).

Yes there quite literally is internally in an LLM.

Just because something is called an "attention head" doesn't mean the terminology makes sense.

If I'm reading the article right, hungryhobbit is accurately describing a single attention head.

And... what having multiple attention heads means is that you do the "single attention head" thing several times, and average the results. There is no part of hungryhobbit's comment that changes if you make this adjustment.

In particular, he is clearly responding to this clause in the comment above:

>> the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time

Those are the "attention heads" that hungryhobbit is saying don't exist. You'll notice that this definition of "attention head" does not correspond to any part of an LLM; I fail to see the problem with saying that "attention heads", as defined by this clause, don't exist.

How was your trip to Los Angeles?

There were no movie stars!

Is the second person's statement "objectively false"?

Or is it more likely that you, Catloafdev, enjoy intentionally misinterpreting simple statements?

You might notice that in the remainder of the sentence you quoted, hungryhobbit takes the trouble to define the attention heads that don't exist.

Also, our brains will encode the differences in registers to evoke emotion differently, which is often used by horror films to make a scene scarier[0]. Evolutionarily this is probably to detect screams or babies crying, a rustling bush, etc.

Speech encoding, at least per this article, has little to do with that. We don't have music encoding so much as we have pattern recognition, instinctual emotional respond to sound, etc.

Another great video about how music is perceived in animals is [1], just while we're on the topic.

[0] https://www.atlasobscura.com/articles/how-the-hidden-sounds-...

[1] https://youtu.be/0ZYhyewNQMo?is=0mWSRAzObOD2p32E

I used "CD era MSDN reference and Raymond Chen blogging style" as a starting prompt for the styleguide and my work ability to digest AI plans raised a lot.

Couldn't recommend it more. Humble, insightful and respecting the reader

When people say LLM slop is disrespecting the reader, I don't think they are complaining about style.

I'd rather read natural sounding, non-repetitive, and actually useful LLM text than the majority of reddit comments (including the serious ones) for instance.

I'm asking to scan projects on gitlab, go through some docs to find more grounding material, write a subarticle (in the same style), scan logs on the test env, issue some curls, etc.; until the whole article is digestible - in the "backing knowledge graph" department.

- Understanding what you wrote

- Verifying correctness of any claims

- Putting in at least as much effort writing as your audience will reading

- Not sharing generated content if you can't do the above. If you must, then explicitly disclaiming your use of AI

It does not mean "prompting AI better"

This is the big one.

> Ideally not sharing generated content if you can't do the above. If you must, then explicitly disclaiming your use of AI

This is the reason why people get mad about AI generated open-source PRs and repositories. Rather than contributing thoughtfully to the commons, you make it a dumping ground when you do this.

slop just means "I don't like this style"

when AI writes more reliably in a way that people do like, they will stop calling everything AI does slop.

Obfuscating LLM output to trick the reader into thinking it wasn't LLM output is not respectful.

It's a single git project at my $USER home, that is referenced in global memory. It contains as much information about work things, as possible, to be productive.

I found that, if I allowed Claude to create the notes, it actually became more and more useful, but without the guideline, I just could barely get through reading it manually.

I'd never publish anything with such origin.

Yes there's a quality component to the role of communication in how it respects other people.

There's also honesty, transparency, truth and vectors along the dimensions of "Are we claiming and presenting the truth or are we bending facts and creating impersonations and warping reality?" Most AI is used for the latter today: people are having AI's write their words and speech for them and then the AI says things as though it were the human like "I said xyz" when the AI is NOT the human who did those things. That's lying and deception and disrespect to the reader.

The obvious response is of course, they're just completely unbothered by it. Why change it if it doesn't even matter (to me)? I presume the set of people who use AI like this for writing and the set of people who are annoyed by it are largely not overlapping, and there is a possibility that a lot of the text I read and think sounds human, might be written by an LLM with a style-guide like mine. Still though, if 5 words genuinely can reduce annoyance by a lot of people who read your article, why does it feel like so many people haven't picked up on it yet? Or is the LLM writing highly loved & popular amongst other people perhaps?