For this testing to be really effective at stopping "dangerous and misaligned" models from leaking out, you need a mechanism for banning failed models that prevent them from being released in the first place, not just prevent US companies from using them.
The only way to stop this from happening is blocking the model's release at the first place. Which requires China agreeing to the same framework. Dario says exactly the same thing himself.
So if he's being truthful here, he's not advocating for the type of ban people are talking about (usage ban). This kind of ban would be helpful to Anthropic's business in the short term, but it won't prevent Chinese models from improving, and it won't prevent them from getting money selling to other countries.
He is openly advocating for an international effort to enforce tests on public models, but I think this is highly unlikely in the current climate. Even if both the US and China agree that public models should be prevented from being used in designing bioweapons, they need to agree on a test and enforcement framework and that requires a lot of negotiation and trust. I don't see this as likely in the near future.
I'm sorry, but if Dario's goal was to try and get international cooperation and he recognizes that china is one of the countries that he needs cooperation with, then putting in:
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat.
Isn't exactly going to go anywhere in convincing the Chinese politicians that they should also be thinking about AI safety. You'll get nowhere by openly insulting people whose cooperation you need.
Half this article is him framing china as an evil enemy to be defeated through boycotts and embargo. Not exactly the diplomacy needed to get them on board with safety regulations.
The law is airtight. Acceptance must be informed and freely given (this includes forcing through dark patterns and annoying banners that force you not to read), and withdrawal should be as simple as acceptance.
GDPR article 7 and its various recital already include that. GDPR wisely doesn't get into technical details like "cookie banners" anywhere, but various national agencies did set guidance and it's usually quite explicit: Rejection must be as simple as acceptance and reject buttons or link must be as prominent as the accept buttons and links.
For example, CNIL, the French data privacy authority, clearly says[1]:
"The CNIL has received complaints about dark patterns on cookie consent banners encouraging data subjects to accept cookies.
As a reminder, with certain exceptions, cookies can only be used with the consent of data subjects. Moreover, rejecting cookies should be just as easy as accepting them."
And gives examples of dark patterns such as different button sizes, multiple accept buttons, hidden reject buttons, etc.
The law and specific guidance is pretty unambiguous. This purely an enforcement problem. The regulatory bodies do not have the resources to go and chase most individual companies, and the non-profit NGOs that go after the violators apparently don't have the budget to make enough impact and scare companies into
compliance.
Despite this being called a "cookie banner", this is not _just_ about cookie. When you click "Accept all" you are giving your consent to any form of tracking and information sharing mentioned in the details. The site you visit may share everything they know about you with any third party they mentioned. They can even use fingerprinting (if you've agreed to it) to keep tracking you after you've deleted the cookies.
And then? What's the practical, concrete, real-life consequence for me? Nothing, not even a bit more relevant ads as these run into my adblocker anyway.
It's not only about you. It is about a practice, that is manipulative and even endangering certain kinds of people. By not caring about these things, we as a society have made the Internet a dangerous place in some regions for journalists and other types of people.
It doesn't even have to be something as bare-bones as pass. You can have a full-fledged password manager that is open-source and local-first. KeepassXC (and the OG Keepass) were always OSS and local-first. The original version of Keepass 1.0 for Windows was released long before Lastpass or 1Password[1], so we had an open-source local-first password manager before we had commercial cloud-based managers.
[1] To be more accurate, although it was always proprietary, 1Password was also local-only at first, with syncing only supported by putting it on something like Dropbox. They only added native cloud syncing later and eventually made it cloud-first.
"supposed" is doing a lot of heavy-lifting here. According to who? The FIDO2 or Webauthn standards? Or in a perfect world?
FIDO 1.0 started as two different standards: UAF and U2F. U2F was for USB keys used as second factors (so almost always stored in a TPM-like chip and device-bound, but not provided by your platform and there could be multiple of them). UAF were either provided by your platform or by any software and there was no requirement for them to be stored in TPM. Back in the day, very few platform had any FIDO support built-in, so in practice UAF was always done in software (usually based on whatever biometrics/TPM the hardware provided).
So competing options were the default for early FIDO, getting a default platform option is something that came later.
Passkeys is basically a brand name for "discoverable credentials" (a Webauthn term). They do a little more than that technically, but in practice their purpose is what you said. Replace passwords. Or more accurately usernames and password pairs. This is in contrast from 2FA, but even before Apple started marketing Passkeys the FIDO standard supported the concept of Passwordless authentication, alongside 2FA.
Passkeys came together with multi-device syncing when Apple introduced them and IIRC it was pushed as their killer feature by Apple back then, but passkeys can also be completely device-bound. The marketing around this was all quite confusing, but it's a bit too late to fix now.
What we got, as far as the average consumer should be concerned, is that "passkey" is any authentication mechanism (not the actual credential) that can replace a password. And it's still confusing.
I think in Apple stack they cannot be made hardware bound anymore. Platform claim is ignored on creation and the keypair is always in Keychain and syncable unless iCloud sync is disabled.
The FIDO set of standards (UAF, U2F which predated passwords and passkeys) haven't even started as enterprise standards. There are multiple origins for what became FIDO, but the main ones I know are:
1. PayPal was looking for a physical authentication solution for their users, Michael Barrett was their CISO at that point and he became the president of FIDO.
2. Google developed Gnubby (which was internal, and therefore enterprise) and they wanted to push a similar authentication to their end-users, supported directly on Chrome. They wanted this to become a standards, so donated the underpinnings of the Gnubby technology which became FIDO U2F.
I might be wrong but at least these are the two parts I know.
And while the original FIDO could be called dual-use, Webauthn and especially Passkeys were developed to be first and foremost a customer-facing standard.
It doesn't mean they are not confusing, but they are clearly designed with end users in mind.
True. But XSS stealing your token (which is always possible with localStorage) is still worse than XSS using your token. It's the principle of least privilege all over again.
The main reason I don't like the id token is that I've seen way too many instances of the ID token being used as a trusted identity assertion sent across multiple services or to third parties. This is very dangerous, since ID tokens tend to have longish expiry (several hours), are not revocable, and generally do not carry any concept of authorization (e.g. restricted scope).
It would have been better if instead of implementing ID tokens, OIDC only supported the authorization code flow and returned a JSON payload of claims (which nobody would incorrectly assume to be trustworthy).
The OP was talking about sessions (which include session cookies and API tokens). I'd argue these use cases are far more common for the average programmer than tokens and signatures that are used for federation, but I'll bite the bullet here:
JWT is a serviceable solution for service trust and federation. This use case often just requires a very-short-term token, so lack of revocation support is not an issue. Replay attacks are still an issue, but they can also be prevented with single-use nonces that are included in the token claims.
The OP's take (and my take as well) is that JWT is rarely the BEST solution for this use case. You kinda have to use it if you need to implement a standard that mandates JWT such as OpenID Connect. But OpenID Connect is a great example for a place where JWT was used, but was never really necessary. If you do use the authorization code flow securely (on the server side, with a strong client secret and proper CSRF protection) you don't need the ID token. In fact, you don't need to use any cryptography at all! Just like random session IDs, you've got a stateful solution that works reliably without any cryptography.
If you cannot do a series of authenticated network requests between HTTPS endpoints to verify trust, then a signed payload could be useful, but you've got better standards than JWS/JWT for that. That's all.
> If you do use the authorization code flow securely (on the server side, with a strong client secret and proper CSRF protection)
This restriction precludes all desktop clients, mobile clients, and webapp clients -- any place where you can't trust the client code to protect a secret.
I don't exactly disagree with you: Security becomes much easier once you rule out handling all the hard edge cases.
PKCE, OAuth 2.0 for Native Apps and the Device Code flow are a thing. In practice all of these clients work so well with OAuth 2.0, that the implicit and resource owner password credential grants have been removed from OAuth 2.1 and are the latest OAuth 2.0 BCP forbids the password grant and strongly recommends against the implicit grant.
For what? For just authenticating a client application to a server without getting access to any user data? By all means. You usually don't need the client credentials in OAuth 2.0 if you don't want to use it.
But if you want access to user information, you can't just give access to an app that has a shared secret key to a all user data for every single users in the planet. You need to authenticate the user somehow, and sharing the user's password directly with the app is not a good idea for many reasons.
Even if you trust the app to be "written without any bugs", the moment you allow the password grant in OAuth, your service cannot use CAPTCHA, MFA or Passkeys efficiently. The password grant does not allow any of these, so this app that you've let through becomes the weakest link for all cybercrime groups that want to attack your service.
The only way to stop this from happening is blocking the model's release at the first place. Which requires China agreeing to the same framework. Dario says exactly the same thing himself.
So if he's being truthful here, he's not advocating for the type of ban people are talking about (usage ban). This kind of ban would be helpful to Anthropic's business in the short term, but it won't prevent Chinese models from improving, and it won't prevent them from getting money selling to other countries.
He is openly advocating for an international effort to enforce tests on public models, but I think this is highly unlikely in the current climate. Even if both the US and China agree that public models should be prevented from being used in designing bioweapons, they need to agree on a test and enforcement framework and that requires a lot of negotiation and trust. I don't see this as likely in the near future.
reply