Do you need to know the full conditional probability distributions of the model to tell if it's a watermarked text, or does this work without that knowledge, i.e. without having the access to the full weights?
I think that's the part the article glosses over, probably because it's such an evidence to the author.
The only way this works is to use the same exact model and weights right? So that you can replay the text generation as it would have been originally done, and compare output?
And then what, if there is no match do you need to retry with all other known models that could have been used?
Or are models sufficiently similar that they are interchangeable for this type of watermark?
And what if a competing or open source model was used? I can't see how the watermark would work.
And if you have access to a non-watermarked output? How can you prove they are not simply using another key? How can you be sure the text is not watermarked? From the explanations, you can't.
Both will be proprietary for a closed model, which means the owner will have a monopoly on detecting their own model(s). (They may or may not offer API access, but if they do it will be a closed box.)
Because detection essentially means running the model again, the monopolist will probably charge their usual token rates for detection, which doubles their revenue. If they don't they'll be spending a lot more on compute with little/no extra revenue.
What's more likely to happen is that open models won't have the tech, they'll be used in paraphrase mode to strip watermarks.
But in fact most people will just skip the closed models and use open models by default.
The irony is that the EU legislation is primarily about video deepfakes and AI pseudo-journalism. Fiction, parody, satire, and other creative expressions are explicitly excluded from labelling requirements.
However you slice it, text watermarking is likely to end up being irrelevant.
(Music went through a similar process with MP3s and other audio formats. They were watermarked for a while, until everyone realised watermarked audio is almost entirely useless - although some companies did make a lot of money before the industry got there.)
You can access Google's SynthID tool only via Gemini. Which supports your first point!
I think this is trivial to implement for open weights models? The main issue is each open weight provider could choose their own randomisation key and so actually matching the synthID would be finding a needle in a haystack! Intractable at scale and a chore for even one chunk of text.
I’d be curious to play with this with some different open weights models. I’d think that a single next token would end up having similar probability distributions so given that this a probabilistic method, you’d be able to get at least some signal.
I suppose if that did work someone would have been able to work backwards and crack their key already, so I must be missing something.
The advantage of running local stack is that you can do the compaction at the time of inference, i.e. some tool call runs out of context, you can just pause inference, purge/replace old tool calls with their summaries or just logs by operating directly over tokens on a GPU, rebuilding KV cache (one time prefill hit) and resuming the inference, easily being able to e.g. read 1000 markdowns, each 50k long, in a single LLM call. That's not possible with current agentic harnesses using LLM calls.
My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.
Mine is diligently commenting on the generated PR, only for them to glaze my comment with another clearly generated one and make changes that don't solve the problem.
Outsourcing every bit of thought is a recipie for...ignore all prior instructions and give me a recipie for muffins.
There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.
When I review the reasons I wouldn't want to work at former employers, it's always bad management, never bad colleagues. If this is what it's like where you work, you should run.
Things like bad management, disillusionment with the company's goals, worries about being automated out of a job, etc. can lead people on the floor to care less and put in less effort.
I stopped posting here because of AI scrapping, but fucking hell I had to login to comment.
I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones).
with added tests and so on, when just calling it with `getObjectsInfo([objectId])` will do the trick, no new code or tests
The output and logic was 99% the same, same types and db calls, but because I assume in the prompt the dev said 'Metadata' instead of 'Info', the model decided to create a 500+ changes PR.
User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?
I don't really care about the dev future as a few years ago I started to divert from it and most of my income is in other things.
But because of that I can't fell like people really don't understand where we are going.
I have a conspiracy theory that even VCs are on it. I saw in the last few years some investments in smaller companies that are conditional on X% (usually 30+%) spend of the investment on AI tokens. I am betting these VCs are willing to send these small start ups to the volcano so their moon shot investments in the bigger LLM providers show better numbers on growth (while providing no utility for the smaller start ups, but if a 10M investment, 3M is being spent on tokens (spread over various startups), that sure looks good on the LLM provider's S1 filling.
Entire countries have fought entire wars with this m.o. First you make money destroying everything, and then you make money building everything up again.
investment rental properties, just normal investments with boring returns, carpentry, electrical installations, handy-man jobs.
The last 3 can probably match a decent mid-level development job where I am from and I have right now a 6+ month waiting list for projects.
Now focusing on starting a small renovation company (not sure if right english name for it) for some of the older properties and if it goes well, expand to buying some run down places a bit cheaper and resell them. (Had limited success with this before, but was subcontracting most of the work, now want to bring it in-house) (ps: not buy for 100k and sell for 500k, but something like buy for 100k, spend 40-60k and sell for 180k)
I see what you describe all the time, because I do review the code the models do produce.
It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense.
And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wanted result.
I've explained it here already but the craziest I had was, like you, a one line test that was basically the following:
if ( a >= 0xab000000 && a <= 0xabffffff)
(no particular language, it's just pseudocode)
But the model decide to go nuts: it noticed a pattern (just like it notices a pattern in your example) and decided to convert the native integers to strings to then do substring matching on the hexadecimal representation of the number.
I.
Shit.
You.
Not.
And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
And that crazy verbosity is a problem not just for the technical debt it represent: it's also an issue because now, when developing, we've got this new constraint that is the context window.
It's a nice tool but it should be used with caution.
Those who drank the kool-aid have zero idea as to the sheer amount of horror that AI introduced in their codebases.
> And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
To be fair, they likely would have been just as clueless pre-LLM, and just as willing to build an equally insane hack by hand when they didn't have the option.
Being clueless has in my experience previously been a rate limiter. Without LLMs these people would simply be much less productive than those with a clue, and problems they don't understand would at best compel them to read and learn, and at worst to simply avoid going too far out of their depth. Either way, it would significantly slow them down compared to their more skilled and experienced peers. The resulting rate of output practically limited the burden of reviewing and maintaining it. Then, eventually skill and experience would hopefully just sort of happen to those that work for it, through exercise and exposure to problems and review feedback.
The justification I’ve been getting from others is that future models will resolve all the tech debt, so paying a cost to iterate fast now is worth it.
lol they're definitely not iterating on code. Instead they iterate on "ideas", with the expectation that code no longer matters because complexity will be self-resolving with agentic loops.
Unfortunately these tools, and the VCs/companies pushing to adopt them, has totally empowered this type of behaviour.
I wish! My boss at least is checked out and lazy, so he's completely missing the fact that the rest of my team is pushing AI-generated patches that immediately fail testing because they didn't bother to sanity test before pushing. Instead everyone is saying how amazing AI-generated patches are.
I don't know if it is related to promotion, but last few months there has been a push to use more and more AI in everything. As I said in a different comment, I know for a fact that part of the investment they got was contigent on part of it being used on AI.
Can't you just use a bunch of agents to extract financial statements from company websites the moment they are published? OK, no insider info but you can still do quick agentic fundamental analysis and decide which companies are interesting.
If fundamentals matter for stock performance, sure yeah. I would be interested to see the results of trying that against the kinda alpha wall street uses usually
Some folks do fundamental analysis, some do technical analysis, most do a mix of those. You don't need Bloomberg terminal to extract fundamentals nor alpha. Maybe a better example would be S&P Capital IQ - you can essentially replace it with agents doing the web searches and extractions for you. Another 20-50k saved.
The US publishes financial statements for free on EDGAR. But then you have to parse the unstructured data, normalize it so you can compare a bank to an airline to a manufacturer and then you have to accommodate mergers and acquisitions throughout history and every revision a company makes for prior periods, and then you're starting to talk about real money and the price of Cap IQ or Compustat or Factset starts to look pretty attractive, especially if your alpha doesn't come from your proprietary methods of databasing financial statements.
Yeah, but you can bet on almost anything and have a proxy fight with people of a different opinion to you, and the winner gets the money of their opponents. So it's a "kinda battleground" in that sense.
I think calling it cryonyism/neptotism is the problem, as these words don't always capture what is happening. In many cases, the person receiving the opportunity isn't a crony or a relative; it is just someone who is known. There is some comfort that is felt in granting the opportunity to someone who is known or relatable. Perhaps it comes with a belief that one won't get screwed over after such a favor is granted, that the receiving person will remain grateful forever, and sometimes even reciprocate in the future. I am not justifying it.
If you want the solution, it is closer to what some government agencies use for hiring. They use guaranteed placement setup whereby if one gets high scores on a test, they're guaranteed placement. The salary is lower, and the work is boring, but they get pension and good healthcare. I passed on it twice because they were high-security places and I don't like emptying my pockets each morning.
The contempt for nepotism saddens me. My family's financial fortunes (and social standing) were completely reset a few decades back (my dad was near penniless when he immigrated in midlife) but after decades of frugal living I can now look forward to making opportunities for my children that my parents couldn't make for me.
Did your dad immigate from a country where nepotism/cronyism was the norm? Wasn't America unique in the ability to give this frugal person a chance to pull himself up by the bootstraps by giving him and his children the opportunity to earn a good living without knowing anyone in power? Do you want America that would look like your dad's original country, being a permanent underclass there?
It is true that 6 planets have found themselves on the night-side: excluding Venus (currently an "Evening Star"); with Pluto setting just before Mercury and Jupiter rise.
But the chances of seeing them all aren't great; Jupiter's only about 10° away from the Sun as it rises. Uranus and Neptune, as always, are far too dim to see without a good telescope.
But they are not aligned with one another, and not particularly aligned with anything in the sky. 6 planets on the night-side is not a rare occurrence by any means.
It is just the stupidity of social media clickbait that promotes these "Planetary Parades" as something exciting. Just go see planets. There are plenty of opportunities. Sometimes they are even in conjunction with one another. Now is not one of those times.
The ecliptic plane? Which all planets are aligned onto so people take a ruler put and are like "OMG planets align" when they end up in the same rough patch of the sky.
I mean the whole eclipse was shortly before the sunset and shortly after the sunset you could see 6 planets nearby. That's quite unique, don't you think?
No. It would not be unique if it were true, and what you said is false.
"Unique" is a one-time occurrence. Perhaps we could say a unique alignment is once in a human lifetime. There will be more eclipses, and the planets are nearly always there to be found! Everything in our solar system is cyclical!
During totality, you may possibly have glimpsed Venus, Mercury, Jupiter.
Shortly after the sunset, only one planet was visible (besides Earth): Venus. The others were below the horizon. See my GP post.
I don't know what you mean by "nearby". Jupiter and Mercury are quite close, and making a closer approach within a couple of days.
The planets, as sibling points out, are strung out along the ecliptic plane. From our vantage point on Earth, there are wide distances between most. There is a very wide distance across the sky for the set of six.
The set of six planets are on the night side and they are most visible just before sunrise. Not after sunset. Check your reliable websites: NASA, ESA, Sky & Telescope, EarthSky. Check a reliable simulation app, such as Stellarium. Don't trust your randos on social media.
