Fyi, I believe `--flash-attn on` doesn't do anything, you should instead use `--flash-attn 1`. I'm getting ~150t/s on a RTX 3080 10GB as well with f16 cache type.

Thanks.. updated my local docs :)

If that's true, then we're following the timeline of https://ai-2027.com/

> If that's true, then we're following the timeline

Literally just a citation of Meta's Coconut paper[1].

Notice the 2027 folk's contribution to the prediction is that this will have been implemented by "thousands of Agent-2 automated researchers...making major algorithmic advances".

So, considering that the discussion of latent space reasoning dates back to 2022[2] through CoT unfaithfulness, looped transformers, using diffusion for refining latent space thoughts, etc, etc, all published before ai 2027, it seems like to be "following the timeline of ai-2027" we'd actually need to verify that not only was this happening, but that it was implemented by major algorithmic advances made by thousands of automated researchers, otherwise they don't seem to have made a contribution here.

[1] https://ai-2027.com/#:~:text=Figure%20from%20Hao%20et%20al.%...

[2] https://arxiv.org/html/2412.06769v3#S2

Hilariously, I clicked back a bunch and got a client side error. We have a long way to go. I wouldn't worry about it.

Care to expound on that? Maybe a reference to the relevant section?

Ctrl-F "neuralese" on that page.

You should just read the thing, whether or not you believe it, to have an informed opinion on the ongoing debate.

I did read it a while back. Was curious what parent was referring to specifically

March 2027 -> Neuralese recurrence and memory

> For example, perhaps models will be trained to think in artificial languages that are more efficient than natural language but difficult for humans to interpret.

That's not supposed to happen til 2027. Ruh roh.

Only if you ignore context and just ctrl-f in the timeline.

What are you, Haiku?

But yeah, in many ways we're at least a year ahead on that timeline.

I got Opus 4.7 working on oh-my-pi with this commit if it interests you: https://github.com/azais-corentin/oh-my-pi/commit/6a74456f0b...