I am finding that I am now less interested in better models than I am in token budgets. My issue with Anthropic models now is that I don't feel like I can rely on them as a daily driver because they'll dry up before my quota resets.
I am becoming dependent on AI to make a living, and I need predictable spend on it. If I know I can't use a model regularly all month, my enthusiasm is limited.
I urge Anthropic to get better at this aspect of their business so I can come back to it.
IMO, if you depend on AI to make a living, I'd invest in hardware for local inference, and learn on how to effectively make a living using AI inference you control, on hardware you control. Sure, economically speaking it's way cheaper to use one of these heavily subsidised services (for now), and their models are faster and more capable, but if your livelihood depends on AI inference, and you are renting AI inference, you are a being a serf of the tokenlord. And your livelihood depends on the whims of the tokenlord. They can increase rent prices, they can decide you can no longer do whatever you are doing, and you have no recourse, because you are dependant on them to make a living.
There are a lot of things in my toolchain pre-AI that I did not own and relied on to make a living. Mobile developers are in even worse shape, and iOS developers doubly so. The idea we were somehow less beholden before AI, I think, is silly.
None of us can wholly do our trades without support. Local inference is a fun idea, but you'll be out-competed by the serfs, as you call them.
You listing things that make developers dependent doesn't mean they weren't less dependent before. Now they have all those things, and more.
It is debatable whether local inference is a competitive disadvantage. One key advantage is consistent performance. No unexpected model downgrades or yanks, no silly safeguards imposed, and no quotas is a lot of advantage.
And by the way, it is not an either-or decision. You can use local inference as your daily driver while still leaning on frontier models when you get stuck.
It's okay to be a prepper, but you don't have to be. Assuming access to internet, water, electricity and increasingly AI is a entirely fine way to live. It might not be anthropic which you will want to use but current capability models will be abundant and access readily available.
I don't read that recommendation as prepping. If I were to start an earthworks business I likely would rent heavy equipment at first, but once I get the business going I'd likely begin to buy my own machines.
That's not prepping in the sense of having a bunker full of canned beans. Its taking control of a key piece of my business, and likely saving money in the long run.
I don't think that analogy holds up. Owning heavy equipment requires lots of capital, and only makes sense if you keep utilization high. Even large companies will rent or lease equipment if it's something they use infrequently.
A large company might own their equipment, but an individual operator probably won't. So it might make sense for some large software companies to own their LLM hardware, but it probably won't make economic sense for individuals.
Of course the economics are different in different industries. Trucking owner operators account for ~15% of truckers, but buying a rig is six figures against 5-6 figure income. Buying a mac mini is 4 figures against a 6 figure income, so maybe lots of people will do it even if it's not economically optimal.
This may be a difference in location or urban vs rural? I live in a more rural area and many people I've hired over the years own their equipment (well, if having a loan on it counts). That goes for tractors obviously, but similarly for wheel loaders, excavators, etc that they use for hired work.
> once I get the business going I'd likely begin to buy my own machines
only when it's cheaper than to continue renting it in your calculations
same goes for renting vs buying a house, or anything.
the breakeven point here would come when they stop subsidizing the subscriptions, or when local ai gets to run on basically everything, making it ~free (sans electricity)
Except there's a huge gulf of self-hosting and using API hosts - no way you can reach the economics of a shared host. Privacy is a problem but you can chose who you host with and where it's hosted (which jurisdiction).
When privacy/compliance really starts to matter it's up to the client/business to provide you with tooling - you're not running that on your own hardware anyway.
So the local AI for individuals is just a hobby/gimmick at this point not a rational decision. Self-hosting for business is a different story.
I'm not sure. The problem with the cloud llm's is they are complete black boxes that change frequently and randomly day by day.
If you run Qwen 3.8 on your own hardware, every single day, it's the exact same model running in the exact same way.
Yes, it's nowhere near as "smart" as the cloud based models. But it's consistent.
So the workflows and "ways of working" you create will work mostly similar day to day.
With Claude/OpenAI you frequently find days where the models are useless, and days when they are out of this world.
So I guess the choice comes down to:
1. Randomly the smartest thing on the planet with unpredictable rate limits that is mostly amazing, but frequently messes with your workflows
2. A really good local coding model that is consistent every day with no rate limits
I'm not sure. My gut feeling is maybe the right answer is a mix of both.
Gambling on the biggest models, hoping they are working smart that day, when planning or doing very complex work. Then doing most of the tasks/daily work using local models??
You can run any open model on a shared API host via OpenRouter and pin to which host you want to go for the quant/privacy/etc. mix you care about. You can pay them directly if you don't want the OpenRouter overhead - but the convenience of switching, having one invoice, etc. is worth it IMO
It's not closed hosted models vs open local models, it's hosted open models vs local open models where the math doesn't work for local LLMs.
The only local inference use-case I can think of is porn generation (because most providers don't want to deal with it) and illegal shit like hacking to minimize the tracing.
And if you're super paranoid - but honestly giving sensitive info to LLMs in any scenario is a gamble.
If you game and can use your GPU I guess then it works as well but models that fit into a gaming GPU suck too much to bother IMO.
This is pretty terrible advice when there are dozens of AI inference providers out there serving great models with significantly more cost effectiveness than you'd get from buying your own hardware.
I literally only make it halfway through the week until my weekly usage runs out. This is using only Opus, no fable, and I'm on the max x20 plan. It's become ridiculous.
If you just say that you run out of tokens, it does not mean anything about the token quotas themselves being reasonable or not. That depends on how much you use it.
For instance, if you had 10s of agents running all the time, it is not that unexpected that you run out of tokens quickly.
I'm curious what your methodology is that results in that? Are you running multiple teams of agents all adversarially reviewing each others code? Lots of different projects in parallel?
I've only rarely maxed things out and then it's t through doing extreme things.
Agreed. The area I think will become more prevalent in the future for organizations are cost per intelligence -- effectively efficiency. An unoptimized model that costs 90x more than another that is only 10-15% less intelligent is something I would say is not a good deal.
Invest a bit of your time into optimising usage cost. Anthropic has first class docs, actually read it or ask llm to read them all for you and summarise most important points / ask to to reflect it on your .md files. Maybe silly thing like dropping your default thinking effort by one level or adding (sub)agent pinned to other model is all there it to completely fix it or maybe you have instructions that encourage big dumps in CLAUDE.md/AGENTS.md that needs splitting so progressive disclosure works correctly? Naively sending everything to the most expensive model on high thinking effort is anti pattern and will drain quota quickly.
My personal guess is that it's one of those. With effective context engineering it's hard to use all 20x quota, the limit becomes your own attention and time really.
You may argue that you're doing multiple, parallel extreme effort tasks – which may be true but then again, there will be results to actually look at sooner or later and that takes time.
I am on the Claude Max 20x plan, and this still happens when using Fable 5/Opus 5. I would run out of weekly quota in 2 days, whereas Opus 4.8 would last the entire week, and sit at about 80-90% at the end.
Same here. Over the last two weeks I switched back to Opus 4.8 and turns out that still seems to last the week like it used to. These new models must be eating tokens.
I am a huge enthusiast of running local models, but when multiple quality USA vendors provide models like GLM 5.3-flash, I run locally just for the fun of it.
For the purposes of comparing to Fable 5.1, I would mention GLM 5.3 that is about 1/12 the cost.
I'm with you, for what I usually do most models are already more than enough.
What I'm really keen on is better auto-reasoning so I don't have to constantly have the constant inner debate on which reasoning effort to pick for each task.
I seriously hate the none-low-medium-high-xhigh-max-ultra etc that we have now, with companies frequently recommending different ones on each new model release, etc.
It's apparently called Adaptive Test-Time Compute or Dynamic Test-Time Compute and companies are apparently working on it (according to some LLM :shrug:)
Adaptive reasoning is known to be an extremely hard problem to solve, though. It requires you to predict whether a certain LLM, with a certain effort level, with a certain prompt, will give you the right answer.
All of the subscription AI platforms are trimming down quotas across the board to push users into higher tiers. Whatever they can do. Local inference needs to meet pricing sooner
overthinks, been slow lately through the official api (slower than glm 5.3 somehow), and tries to run every conceivable e2e test once it does literally anything.
like yesterday it ran for like an hour to build a fairly basic frontend...
I am becoming dependent on AI to make a living, and I need predictable spend on it. If I know I can't use a model regularly all month, my enthusiasm is limited.
I urge Anthropic to get better at this aspect of their business so I can come back to it.