They are still making billions from highly sticky enterprise customers, which is more than enough to survive. Otherwise, Microsoft would have gone out of business years ago.
Sure, but then they're out of the "growth" stock category they so badly wanted to be in, with its higher valuation multipliers, then in the stodgy "keep the lights on" niche. If a company makes $10B this year, and you had a way of knowing it was going to make $10B every year from now on, there's not much incentive to invest in it.
Note that this doesn't make for a bad company! You can run a $0-profit company indefinitely and give employees nice jobs for providing nice services to your customers. There's nothing wrong with that. It's great! But you don't get the same multiples for that arrangment as you do for a company that's growing quickly. Lower multiples = stock drops = One Medium Wealthy Asshole Called Larry Ellison, and that's something I simply do not think he could tolerate.
Nothing about this is surprising, something like this was always going to happen, because you - or an LLM for that matter - can always find a line of motivated reasoning that justifies any course of action. One would have to be extremely naive to believe that "alignment" provides any kind of actually robust guardrails. Simultaneously, we have seen decades of security vulnerabilities. Unless your testbed is truly and fully physically airgapped, any current SOTA model will find a way to break out.
The reality is that the current approach to AI safety is little more than a fig leaf, but you also won't be able to put the genie back in the bottle, because the technology is simply too powerful to abandon. There is always going to be someone developing it further from now on.
So, the real question is what a novel and actually effective approach to AI safety looks like and how to get there.
>effective approach to AI safety looks like and how to get there
It's probably impossible, or it may only be possible in hindsight which means it's already too late.
If for example you make an entity smarter than you all you can do is hope it will be safe. Any entity smarter than you has more freedom of choice of actions than you do, or at least the ability to explore them. A common example here would be a 2D entity trying to contain a 3D entity. The 3D entity can simply rise up and over any line you draw to stop it.
And that's for cases where intelligence has a will to break out of it's box. It doesn't even need that. Instrumental convergence can sit around and build solutions until one of them passes the "don't do bad things" classifier.
And lastly, you're assuming that models will remain expensive to train well into the future. If the cost drops significantly you can expect someone to develop an intelligent but completely unhinged model at some point. And it may even be AI itself that creates it. There is no natural evolution that occurs that natively makes safe models.
It's honestly a smart decision. They already have access to the common market just like Norway. Becoming a full member just means being subject to a EU governing body that has proven itself to be largely ineffective and incompetent, sometimes reaching the realm of "outright harmful" with things like Chat Control. Why would you want to be a part of that, especially if you would also be paying for that privilege?
I love how people comment very confidently on shit they clearly know nothing about.
Iceland is in the EEA. It already has to comply with pretty much all EU regulations (with some exemptions, such as their fishing thing). It is also a part of Schegen, freedom of movement, etc.
Unlike other countries, it doesn't get to vote on those regulations, so it doesn't even get the veto power.
I honestly think it may make more sense for Iceland to not be a full EU member, the fishing industry may be too important for a fairly small island nation, etc.
But it has nothing to do with the reasons you fantasized about.
I would say it’s the other way round: being in the common market means that they are already subjected to EU regulations, without having any representation in the governing bodies however.
You should probably realise that the world has radically changed since then. This kind of thing works when you have a significant lead in the field that makes keeping vulnerabilities open sufficiently low risk for your own side. But if your adversaries have similar capabilities, then the calculation changes.
Has anything changed? Governments are hoarding undisclosed vulnerabilities, using them as they see fit instead of fixing. Every espionage, surveillance, or war campaign (see Russia v Ukraine, US/Israel v Iran etc) is followed by a ton of burned 0-days.
>This kind of thing works when you have a significant lead in the field
No? It works even if the adversary has the same capabilities. It only stops working when everything is fixed.
Would be interesting to compare the Chinese version. Because, obviously, their English version is for users, not for investors or government officials, while the US labs are always addressing those too.
That always seems to happen upon release of a new model. People look at the benchmarks, which look good, because the model was almost certainly optimised for that. Then they start actually using it, and after a week or two we get the real assessment that is almost always less impressive than the initial reactions.
This is not even news. This piece of information was already included in the original release, and the issue was apparently a miscommunication between the two companies that left a gap open in the test environment.
I hold out hope for (human) work to coalesce on creating a stable/secure microkernel (SeL4 anybody???) and getting it mature enough to build an OS on.
Once that is done, then you build out the absolutely massive collection of userspace drivers that would be needed to replace linux by leveraging LLMs and other automated techniques. The good part is that by keeping the drivers in userspace, the security risk of generative techniques for driver development is pretty low.
reply