People are free to do what they want in their free time, including the freedom to decide which incentive structures they follow. Having incentives to read just... sounds like a good thing to me.
The pong demo is giving "Point to Jev" before it's even reached my paddle on mobile. I also don't really understand the frontpage model-speed comparison: it looks like you're slowing the ball down to compensate for the models' decision-making speeds? If so I think a more effective metric would be N-x, e.g. we slowed this game down 8-x (compared to Jev's game) for e.g. Sol to stay competitive. Or, just watching them fail would be cool too, though I know it's just a simulation and no real inference is being ran (except for Jev, maybe).
What an interesting construction. Thank you from a curious layman for your write-up. I thought it was pretty easy to follow. I'd heard of the surreal numbers before and never knew about the construction mind-game behind them.
What? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
> Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
Floating point math is 100% deterministic, but different hardware/OS have different but deterministic behavior in some corners. The same code run on the same hardware with the same inputs (including access to timers, peripherals, etc.) will behave the same way, unless you're talking about cosmic rays flipping bits or something.
It's certainly possible to write LLM inference on the GPU in a deterministic way, but it's somewhat nontrivial and trades off against performance, so by default most LLM inference engines aren't deterministic even at zero temperature. The classic post about that is https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
> Running two tasks simultaneously with Fable and Opus 4-8 on ultra reasoning
That's abnormally heavy usage for Pro plans which don't include a whole lot of usage to begin with. Opus is generally too much for them but you can get a lot of mileage out of Sonnet.
simonw's pelicans probably wouldn't get posted in response to a request for a more quantitative analysis.
You and others are right though, that there's potentially interesting or enjoyable stuff in there (maybe I should have lead with that?). It's just a large volume of it is not useful in response to a question specifically looking for more quantitative or detailed usage analysis.
I agree with you that labs are benefiting from those outputs but I'm skeptical that labs are purposefully training the models to produce those outputs.
Raw pre-training data includes plenty of conversations between professional builders and some of those include estimates.
I believe the outputs are a training coincidence with consequences that are opportunitistic for the labs.
Insincere apologies ought to be mocked to shit but this apology seemed well-meant. (I know you're not mocking them and the last sentence is actually something Claude would say.)
I post this angry comment because LLMs are colonizing the language we use for creating an earnest and genuine tone in online discussion and I sometimes wonder if the suspicion surrounding LLM-ish language is worse for the health of our online spaces than the LLM slop itself. Thinking about it, I don't think it is; and it would be impossible to measure anyway.
reply