Golang is an amazing runtime with a bad language, one that conflates simple with easy. I view it the same way I view Java: a fine choice for a corporation, but nothing to love. Although Java’s gotten a lot better lately.

For me having the proper tool for the job trumps loving. I don't have to love the language, I have to love the process and the end result.

Opencode has really bad cache stability issues that they seem uninterested in fixing at the moment.

The OpenCode devs talk about this on Twitter a lot, e.g. https://xcancel.com/thdxr/status/2048268697790300343

> tool call pruning breaks cache and people will tell you this is horrible and expensive

> except i looked at some anthropic data and real user behavior ends up with better cache hits and 30% less spend

> even this is needs to be analyzed further, it's just not simple

> for openai data it's inverted! cache hit ratio is actually better [sic: I think he meant worse based on the screenshot] with tool call pruning turned on

> but the net $ saved is only 5%

> kimi is a funny one - it has better cache hits with pruning on...but is also more expensive!

There was also another thread recently where he discussed that pruning improves user experience (models are smarter with less context) but I can't find it.

This can also be disabled in the config: https://opencode.ai/docs/config/#compaction

My understanding of caching with most models/providers is that a prefix substring of the context has to be reused for a cache hit, but not necessarily the whole entire context window. So if you prune tool calls from the history, you're going to get one cache miss on the newly-pruned history, and then you're going to be getting cache hits on every subsequent turn, with a lower number of input tokens. If you prune subsequent tool calls after that, you would still get a cache hit for the already-pruned portion of the context, just not the full context.

So it makes sense to first send stable prompt, reasoning and files content, tool calls summary and actual tool calls at the very end?

The way you do this (and the way opencode does it) is you do most of your pruning in more recent history. Last I looked at opencode, they start pruning tool call results after 2 full agentic turns. So you probably dont get quite as good hits on cache for the most recent 1-5% of your turns, but after that everything else caches fine and those tool calls that likely aren't relavent to your session anymore are gone.

You didn't quote the interesting part:

> our implementation is it only prunes calls from > 3 user messages ago, if context is > 40K, and only if there's at least 20K tokens to be removed

Seems reasonable to me and explains why I can have long sessions (way longer than with zed agents) while still hitting cache. Opencode is just missing per-provider TTL.

I found that keeping current context utilization at 18% of total context length was best for minimizing spend, across all models with 400k context length or more

They are. Empirical evidence on my side. Because attention is sparse across the context. It's not truly treating a million token the way it treats a fraction of that count. For performance.

I can't confirm this. Having utilized Opencode for a large project over the past 10 months, with multiple models and agents, we've never run into such 'cache stability issues'."

That'd be really easy to spot and also fix, most likely. Any open issue you could point us to, must surely been reported already?

> That'd be really easy to spot and also fix, most likely

Ah, reminds me of good old "There are only 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

> Ah, reminds me of good old "There are only 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

You quip, but LLM KV caching (from the harness side) is quite easy: You get a cache hit on stable prompt prefixes, period. That means you want to keep the prefix stable, and only append at the end of the conversation. Made up example: Don't put the git branch name into the system prompt part (that comes first), as whenever the branch name changes, that'd trigger a cache invalidation of the entire prompt.

Getting this right requires some care to not by accident modify the prefix, basically, and some design on communicating the things that can change (user configuration, working dir, git information, ...).

That sounds like the experience of writing Containerfiles; since steps are cached you want to pull the thing you are iterating on as far down as possible.

All of this work has been done before in different contexts. Memory management with bigger blocks and weaker definitions that change whenever some grad student gets a bright idea.

100%. Since you mention memory management: Generational GC is pretty much the same idea: Keep the stuff that's least likely to change an important property (liveness) together.

Conceptually the underlying general idea is to sort things based on stability if you can avoid recomputing properties of the stable part.

It's even closer to prefix matching on super long strings by chunk

Opencode (and other coding agents) have hundreds of open issues reported. It is quite discouraging when they are not being closed/fixed.

These projects have also been the recipients of PR spam, lots of duplicates and unconfirmed in there for less technical people and clawd operators

Yeah that's frustrating. Everytime they pop up on my Twitter timeline I shame them into addressing their issue backlog.

They're aware of the issues length and they're "looking into a solution".

I'm not sure that is really the case, or relevant in practice. I have been using OpenCode with DeepSeek lately (regular coding). For instance, today I got 120 million input tokens hitting cache, vs just 2.59million missing cache.

Reads like a LOT of tokens to me. What does your usage /workflow look like? I'm v curious because although I do use Claude code, my token counts aren't nearly as much

I want to know if I'm missing something cool!

Not OP, but I routinely load 150k tokens into context. A full sub-package to work on, select other files in the monorepo, e.g. front-end visualization and back-end data loader. Then work some 150k tokens, then start again.

At the end, cache hit rate is like 99.5% if Novita is not having issues.

For official DeepSeek API, 99.9% or something.

Custom harness that never compacts or otherwise doctors the history.

Those numbers make sense to me...120 million input tokens is like 120 sessions of hitting the full context limit, which seems like a lot to me though

What I noticed when using OpenCode with llama.cpp, was that the default host RAM prompt cache size in llama.cpp was way too small for say 128k Qwen3.6 27B.

The default is just 8GB and a full 128k context for the dense model can take most of that. So then comes an agent and causes eviction and subsequent cache miss.

Bumped the cache size (--cram IIRC) up to 48GB and had much better results.

I am getting 98.6% cache hit ratio on deepseek-v4-flash with opencode

That’s impressive!

On the sheer performance it’s comparable to Opus ?

Here are my stats (from DeepSeek directly, with a script I wrote). The prices are what equivalent Sonnet usage would have cost, the actual amount I paid was $10. On performance, DeepSeek V4 Pro is comparable to Sonnet for me.

     ./cost.py amount-2026-5.csv 0.3 3.75 15
    input_cache_hit_tokens: 472,971,520 tokens -> $141.8915
    input_cache_miss_tokens: 13,299,013 tokens -> $49.8713
    output_tokens: 3,334,962 tokens -> $50.0244
    cache hit rate: 97.27% (472,971,520/486,270,533)
    cache miss rate: 2.73% (13,299,013/486,270,533)
    total: $241.7872

All of this usage was with an OpenCode subagent exclusively.

out of curiosity, how do you measure cache hit rate in opencode ?

opencode stats

So the calculation is:

Total input token = input + cache read + cache write Cache hit rate = cache read / total input token.

That is 71% in my very limited use of opencode.

The first

There are some that are specific to certain models like qwen/gemma

I switched to vLLM and those went away. Need to look at my opencode config and adjust some others based on things I see here

But if you look at the node compliance tests, deno has better compliance now days…

Insanely better, at 76% Node compliance in Deno 2.8.

Bun 1.3.14 is at just 40.6% with same compliance test.

https://node-test-viewer.deno.dev/

I guess Bun had the better marketing then. I liked how every new feature came with a benchmark against the previous version and node. See this for example: https://xcancel.com/bunjavascript/status/2048228152397459590

I'd love to see a site comparing the 3 of them in a similar way.

What do you mean? The whole point of Ruby on Rails is the rails way? Also the problems you are describing are not new and the community settled on adding some sort of service layer

https://shopify.engineering/shopify-monolith http://sporto.github.com/blog/2012/11/15/a-pattern-for-servi...

I’ve always found Ruby to be way more readable, what keeps me using python is the depth of libraries is unmatched.

So unless you’re into burning tokens having AI generate untested libraries, I’d stick to using the most idiomatic tool for the problem you are tackling.

So, it's really interesting. We've started moving away from python libs because 25% OSS is out of date and another % is custom tweaks to the software help our use cases. In both scenarios it means our own fork.

And honestly it's not burning that many tokens if you've got an existing example lib to point to.

Tbh that is some engineering teams I’ve worked on…

These models are open and there are tons of western providers offering it at comparable rates.

I can not find a description of how it works on the site, magic hands daemons !

Cool story, but what runs when?

tl;dr : https://charlielabs.ai/how-it-works/ and https://docs.charlielabs.ai/daemons

to get started, look at https://docs.charlielabs.ai/installation, but essentially

1) signup with your github login

2) install the CharlieCreates GitHub App on the repos you want Charlie to work in

3) create an issue, tag @CharlieHelps to help you create your first daemon!

the daemon will run on the charlie runtime and follow the watch / schedule conditions you set in your DAEMON.md file

full disclosure: i am part of the engineering team behind this.

Fix them if needed, the OP’s point is that for a lot of applications it is not needed.

For most cases you will still be comfortably in the JVM/golang performance window.

Rust is great language, fighting the borrow checker sucks, don’t do it if you don’t need to.