I don't mean to be super negative, but I read through this all the way and still have no idea what it does or how it works. I think the problem statement needs to be outlined much better, and concrete examples of turns with/without your solution would be helpful to illustrate the difference.

Unfortunately, the lack of an input cache discount makes it prohibitively expensive for most use cases that aren't one-shot prompts.

well same applies to GPT OSS 120. Qwen is just the much smarter model of the 2 public options on Cerebras.

The point is the speed.

If the model provides me with bad results because it's dumb, I don't care how quickly it does it.

But there are lots of use cases where a relatively "dumb" model is good enough.

Is it possible to construct a control system where bad, fast and cheap can become good, fast, and cheap through repeated sampling and a strong spec/eval harness?

I am trying to keep an open mind with AI, but I also have little understanding of control theory, trying to learn.

You can, but you need to break the problem into much smaller tasks, then check those answers, and finally have a harness that handles all the context, task breakup, task definitions, and validations each round.

Smaller models seems to get stuck in "loops" when you try to "handle" them this way.

fast results that you need to verify are better than slow (allegedly better) results that you still need to verify. REPL vs batch.

Depends on how many times you need to iterate to get the result you want. If you need to run the fast model 5 times to get the results you need, compared to 1-2 times for a smarter but slower model, you've just eroded any advantage that the speed gave you.

So, as always: it depends on the use case.

> It is, but isn’t it all warranty in your scenario?

There are numerous recent stories of companies refusing to make good on their warranties and instead offering customers a refund for the original price paid. One example: https://www.tomshardware.com/pc-components/hdds/toshiba-refu...

Because the end goal is to ban non-US AI companies from being able to do business in the US.

...because everyone saw how well that worked for the Jones act, what with all the naval yards the US has lost over time, and how nearly no US-built ships operate where not legally mandated /s

Just because it's a bad idea, doesn't mean they won't do it.

some US companies benefit from that act for sure.

For caching, only if you don't specify your preferred providers and let OpenRouter route each request itself. I have stuff like this in my OpenCode config for each model I use and I regularly get ~90-95% cache hit rates.

    "order": ["relace", "coreweave", "novita", "baseten", "together"],
    "allow_fallbacks": false

It still won't be quite as high as you'd get by just using DeepSeek because occasionally a request will fail and you'll get routed to a backup provider with nothing cached, but it's close enough not to matter in most instances.

But I can't argue with the lower off-peak pricing when using DeepSeek directly. The downside is they train their models on your input, which might be a deal-breaker for many users (as it is for me).

Honestly, I know several devs who do chores around the house or even play video games while AI does the bulk of the heavy lifting. If nobody cares or even realizes, does it matter? (To be clear, they all work remotely.)

Which doesn't matter if they're still not profitable.

I don't see any kind of input cache discount listed on your pricing page. Do you offer that, or is all input priced the same?

The issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads.