Any data center that can remain profitable selling open source tokens at commodity prices will be fine. Any data center that relies on OpenAI/Anthropic level token prices and margins might be in trouble. After clearing their debts through bankruptcy, they'd likely be quite profitable selling open source tokens at commodity prices.
Eventually these models will get commoditized (we're already at the "good enough" stage for real work), then they will get turned into custom hardware and get 1000x faster, then that hardware will get commoditized (like DSPs) and they'll be everywhere and cost $1.
From a very high level view DSP's and GPU's are the same thing: highly programmable number crunchers. A better example would be Google's Tensor chip. It's not as general purpose as a DSP or GPU, it's more optimized to do inference.
That trend will continue.
I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:
- more specialized / better chips
- IC technology: smaller feature size, higher clocks, etc.
- more efficient algorithms
- solar power is getting cheaper at an order of magnitude per decade, batteries even faster.
- pricing pressure from open source models
- breaking of the Nvidia monopoly and it's 75% gross profit margin
Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.
AMD bought Talaas specifically to make AI accelerator pieces to be embedded into generalized chips.
At work (we're a medium sized manufacturing firm), we bought our own inference server for $107k and run Kimi 2.8 for nearly all of our use cases (and dropped our cloud AI spend to $0).
You dropped your cloud AI spend down to the price of capital plus the cost of electricity and maintenance on that server. When/if tokens become a commodity, the price of tokens would be the marginal cost, AKA about the same. Big when/if, though.
After bankruptcy they likely have no debt, so can out-compete those that didn't go through bankruptcy. It's the bankruptcy that makes them profitable -- it's a common pattern in nascent commodity industries.
> Bad news for wall Street though if they have to go bankrupt first?
Don't worry, the already stretched taxpayer will be on the hook for everything just like in 2008!
This time the relief mechanism is already baked into the system (capital does learn from its past mistakes, even if it may not be the lessons you'd hope for!)
By the time the bankruptcies come, they'll all be owned by index funds and main street. The price will crash at the break of the bankruptcy, and wall street will swoop in to buy up the "distressed assets" at bargain bin prices.
I think there's still some low-hanging fruit with thin clients and colocated-to-AI applications. I don't think the math for beefy personal computers is going to hold up, for most use cases.
If it costs you more to generate the tokens that the market is willing to pay for those tokens, then not even bankruptcy will save any of the costs invested in one of these datacenters.
If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.
The point being made here is that most of those costs are amortized capital costs, which get wiped in bankruptcy.
That $5 per 1M token doesn't literally cost $5 per 1M token. It's more like they had to build a datacenter for $500M that can service 100T tokens over its lifetime. They did this by borrowing money on the capital markets, and now they have to pay interest to those bondholders, interest that they can recoup with their $80/1MT prices. But if it turns out they can't charge $80 and have to charge $1, they won't be able to make those interest payments. They enter bankruptcy, the court wipes the debt clean, and now they don't have to pay interest, only the actual operating costs, which may be more like 50c/1MT. The company gets recapitalized with the new owners being largely the bondholders, the existing equity holders get wiped out, and they can compete with the commodity producers now.
You have land taxes and or rent, building upkeep, staffing costs, electricity, water, hardware replacement costs.
And new build DCs have blown all these costs through the roof justifying the decision because the price of compute is so high. When the prices come crashing down, the expenses will remain fixed where they are now.
In general the bondholders don't win unless they're holding senior secured debt. If they were expecting a return of 5% on their bonds, the fact that the company entered bankruptcy means that the profit it can generate is less than the 5% interest the bondholders were asking for. It may get recapitalized with them as the shareholders, but the profits on the new recapitalized company will be less than the interest previously owed to the bondholders, just by the fact that the company entered bankruptcy.
The two conditions where they could win are:
1. When they have liquidation preferences over the other bondholders. In this case, their claims come first at bankruptcy, which means they can end up owning the company at the expense of the other bondholders and stockholders. The company's overall profits might not be sufficient to generate a return at the interest rate of all bondholders, but it might generate returns over what a select group of bondholders would otherwise get.
2. When the company can't generate sufficient profits now, but their revenues and earnings are expected to grow over time. In this case, the new equity holders would take a significant haircut on the value of their investment at the time of bankruptcy, but improving financial positions means the value of their investment could grow to be worth significantly more than the bonds over time.
I can't rule out either of these for AI companies. The principals of many of the companies involved have a record of self-dealing that's very similar to #1 - it's illegal if it can be proven in court, but it's often very hard to prove, particularly if there are other parties involved. And the economics of AI are likely very similar to #2.
assuming demand remains elevated and growing, maybe. But spend on AI is pretty stratospheric right now... companies are already starting to clamp down on spend. This makes you really wonder if there will be sufficient demand at current commodity prices for eg; OSS models to justify all these data centers.
Yes, I believe so. Using a sibling commenter's number of frontier models being 80X the price of commodity models, I think companies who switch will spend a minority of their savings to increase their token usage and only pocket the majority of the savings.
Most of the affordable rentals in my city and many others is luxury housing that was built ~100 years ago. They built lots of cheap housing back then too, but it was junk that was torn down a long time ago. The luxury stuff was built better and lasted.
The affordable housing from ~50 years ago was single family housing and is crazy expensive today.
We should be building lots of high density luxury condos today. Those are the affordable apartments of the future.
The Europeans did a survey and pegged the value of BYD's government support at 17%. I trust that number as an upper limit because the European government would have had lots of incentive to be thorough and to err on the side of over estimating.
Is Firefox or Mozilla perfect? Heck no. But we're not comparing Mozilla against pre-IBM Red Hat or Ben & Jerry's or whatever your concept of the ideal corporate citizen is. We're comparing Mozilla against Google. That's a very low bar that Mozilla clears with ease, especially if you're criteria includes shoving AI down your throat.
Countries aren't leaving NATO yet, but they are setting up and strengthening alternatives which will make such an option easier. For an example, Canada just joined SAFE.
You are using the wrong metric. The supply of t-bills is increasing rapidly because of the massive deficit. That is sufficient to explain the increased number of holdings.
The correct metric is price. If there is decreased demand, it will show up in the yield. And it does.
> That is not explained by increased deficit, but by net increase in demand.
A yield going up means you pay more for the same thing. So if the US wants to issue more debt, they can. The fact that more debt was bought but the yield went to means the supply grew faster than the demand. So an absolute increase in demand, but a net decrease, thus a higher price as shown by the yield
That is not true. Yield is going up globally, so you need to adjust for the difference in yield.
For example, the US and Euro (average) yield have gone up by almost the same amount in that period, and other currencies like Japan and Australia have experienced an even larger increase.
I’m not sure why the level of discussion in this post is so poor.
"I’m not sure why the level of discussion in this post is so poor."
Its because bond yields and fixed income in general isn't well understood by the public. Even in finance, its often not correctly understood except by those working in fixed income or the IT teams that support them. Funny thing is, often the devs in those departments understand global finance better than the CEOs running those firms because of how fixed incomes is seen by other departments. Basically, its the lowest department because it doesn't get great yield while ironically requiring the best math and economics knowledge to do.
You are misunderstanding some basic things about bonds. Bonds are weird. Higher yield means the bond gives out more coupons (yield, money, etc). But the bond itself costs exactly the same no matter the yield when first bought.
The actual thing being bid on in the bond market is the yield itself. Higher yield is sort of like a higher price in that it means you have to offer more to the lenders. However, what they are actually betting on isn't the ability of the US government to repay. What they are actually betting on is the future inflation rate. So a higher yield doesn't mean what it means for corp debt (ie we don't think you will be able to pay this back). A higher yield for t-bills actually means lenders think inflation will increase in the future. Hence the FED raising rates to fight inflation.
PS But seriously, the bond market is very weird and most people mess up what changes in yield mean for different kinds of bonds because they don't mean the same things (unlike securities ie stocks).
PPS This is all because of the reduction in the amount of oil available worldwide, which triggers increases in global rates, which triggers increases in US rates.
Until Rust proved actually you can get really good or better performance if the language itself is better. I really don’t know how C++ digs itself out of the UB hole it has dug.
Probably by working together with Rust. Eliminating undefined behavior from unsafe Rust is a big deal for the Rust community at the moment. And given that most unsafe rust code exists to call into C or C++, concepts like pointer provenance need to be extended. And proper pointer provenance guarantees can both decrease UB and increase optimization potential.
That's a niche level thing that helps in some scenarios, and generally not as much for C++ which is much more weakly typed than Rust is. Weak typing + static typing is why safety problems in C++ are going to be really difficult to fix without fundamentally changing the language.
I expect some changes to the language from this direction, some way to attach provenance information or limitations to a pointer. Presumably through a #pragma at first. Strict typing in the C++ sense, not the Rust sense. An annotation like "volatile".
Pointer provenance is just one example, there are others.
This particular case is likely an example of that. Rust used to have this problem, but it wasn't ever intended to. So IIRC it got fixed in LLVM for Rust, and this is probably now C++ taking advantage of that.
Actually Claude can teach humans more about humans by providing the language and personality knobs. For casual conversation, have 100 of personality profiles and language styles and upfront tell the user that they are engaging with P profile currently and see how humans relate to it. This will teach humans how to peel back layers of style, fluff, flair from language vs facts.
I think this is a welcome overcorrection though. Any good businessman will tell you they'd rather be backed by an insufferable nerd than a yes-man.
Maybe it's just me.
For like, 90% of conversations, I don't want it to let technical inaccuracies and rhetorical flourishes slide. I want it to tell me that the point I'm making is technically wrong because an expert would recognize subtle misuse of terminology, or because there's an exception or edge case that I didn't proactively insert as a caveat, so that it is my decision to ignore that advice and be a little wrong on purpose to suit my writing goals.
What I don't want is for the AI to assume my writing goals, and be incorrect because it believes that is what I want. I want it to "well ackshually" me so I can say "shut up, nerd".
Like, there's another comment in this thread that I ran by claude to check my understanding about today's post-training methods and how they avoid sycophancy, and claude responded by splitting a bunch hairs over like, "well, technically this is still RLHF, its just that there's other feedback signals mixed in, and the preference is detected in other ways, and ai judges are involved as a filter for examples, this and that and blah blah blah". Shut up, Nerd. In the context of this conversation, RLHF is already being used as synecdoche for user preference feedback, readers understand that, and even if they don't, their misunderstanding is completely harmless. I will not be taking all the wind out of the sails of the point I'm trying to make inserting your three paragraphs of irrelevant clarification in the name of technical correctness, thank you very much.
As long as receiving nitpicks and technical minutiae implies 1. there are no larger structural problems and 2. the model isn't rolling over to please me with sycophancy, I figure this is ideal.
Granted, 66% chance is still essentially a toss up.
reply