Mythos and Glasswing
Anthropic, the company behind the Claude AI chatbot, made two security announcements that were shocking for many but seen as inevitable by those of us working in AI security. First, it announced Mythos Preview, a new, non-public AI model that turns out to be startlingly good at finding security flaws in software. The second was Project Glasswing, Anthropic’s program for getting that capability into the hands of the companies best positioned to fix those flaws before anyone else can exploit them. Apple is one of those companies.
As much as I’d like to downplay the announcements, Mythos and Project Glasswing are very big deals on their own, and harbingers for the future of digital security. Mythos was able to find and exploit new vulnerabilities in every major operating system, including a bug in OpenBSD, an operating system famous for its security, that had been sitting there unnoticed for 27 years.
[…]
We are at the start of a period in which finding software flaws that affect everyday users will become dramatically easier for both attackers and defenders. […] However, over the long run, I believe using AI to identify security vulnerabilities favors defenders, because developers can find and fix many more bugs before shipping software to the public.
Anthropic has a habit of making wild and scary public statements that seem designed to generate headlines and funding but sort of fall apart upon scrutiny. I initially dismissed this as more of the same, but people seem to be taking it seriously.
Our model is so good, it’s not safe to release, yet. Has to be one of the greatest AI marketing stunts ever.
There’s reason for cynicism, given Anthropic’s history, but the part of the “Boy Cries Wolf” myth everyone forgets is that the wolf did come in the end.
If Anthropic has really developed an LLM that can suss out security weaknesses better than any other AI, the US government would be foolish to continue shunning them.
Or, rather, if the government believes the marketing, it may want to take control of the company and its technology, like how it restricted restricted civilian nuclear research.
In fact, Amodei already answered the question: if nuclear weapons were developed by a private company, and that private company sought to dictate terms to the U.S. military, the U.S. would absolutely be incentivized to destroy that company.
Previously:
- iOS 18.7.7 and iPadOS 18.7.7
- LLMs and Software Development Roundup
- curl Removes Bug Bounties
- Common Vulnerabilities and Exposures (CVE) Funding
- curl Takes Action Against AI Bug Reports
6 Comments RSS · Twitter · Mastodon
"If Anthropic has really developed an LLM that can suss out security weaknesses better than any other AI"
What does that mean, though? I've had GLM5 find tons of vulnerabilities in existing source code.
So what does "better than other AI" mean? It finds them faster? It finds vulnerabilities other LLMs can't find? Is this really a fundamentally more dangerous threat than previous models? If it is such an issue, why wasn't it one with Opus 4.5 or 4.6?
I call BS.
It's quite real and the article/paper goes into sufficient depth to prove it. Human-reviewed patches have been accepted by multiple well known projects/organizations for incredibly difficult to discover *chains* of multiple obscure vulnerabilities that when combined make for high severity exploits. It's effectively an automated nation-state level offensive cyberattack generator for 5 figures USD investment.
Regardless of what you think of AI and/or Anthropic, they're absolutely right to proceed cautiously and assist securing widely deployed common software in advance.
What’s so hard to believe about this? Mythos is trained on 10x the parameters of any other frontier model.
> Regardless of what you think of AI and/or Anthropic, they're absolutely right to proceed cautiously and assist securing widely deployed common software in advance.
It appears to be the consensus, but is it really? At the risk of reopening the discussion about "Responsible" vs "Full" disclosure, I suggest that relying on mere computing scale to fortify software against security bugs is a terrible idea, and even putting aside the problems of exclusivity today, whatever is being done now in public by a frontier lab could just as easily be done by another frontier lab in contract to the great and good and with far fewer scruples.
But hey, maybe that's just me.
