I would expect it happens more organically where people’s discussion of the benchmark and posted results find their way into the training data, like anything else on the internet.
Sam previously worked on Overtone, which is a Clojure library for driving the Supercollider synthesiser. I understand that he decided to create Sonic Pi in Ruby in order to make it more accessible to beginners. (And Ruby was all the rage back then.)
I recall at some point the author started working on yet another version which used its own language instead of a ruby DSL but I don't know what happened with that
I'm now working on a new audio DSP engine which builds on all this learning which combines my favourite parts of scsynth and Erlang. I'll be discussing and sharing progress over on my Patreon (https://patreon.com/samaaron).
As a sneak peek - here's an experimental prototype of the kind of direction I'm headed in: https://sonic-pi.net/tau
I was a skeptic until a few months ago. It’s my daily driver now. I’ve got sizable amount of people committing to the same repo, many are not skilled with git. So I deal with weird crazy “how did you ever get the repo into this state” issues not infrequently.
JJ is just so low friction. The seamless and painless shuffling of commits, working up and down the commit chain without explicit rebasing, unchanging change-ids. The min identifiers in change-ids being highlighted. Being able to make commits and worry about the message later. Snapshots of changes while you are working serving as a tool agnostic undo.
Painless and sensible cli args that are vastly more consistent than git.
Superior conflict handling.
You can “jj undo” anything and it works perfectly.
It’s fabulous.
> "@--+::"
I have no idea what that means and I’ve never had to type anything remotely like it.
The rev specification language is pretty elaborate but you don’t need to know it.
Same experience. At first I really didn’t get it and now I could never go back. The ease of rebasing, renaming, reordering, splitting, etc. Has drastically changed the way I work. My PRs are undoubtably much better now, and I am never afraid of messing up thanks to jj undo and the op log in general
Do you have a public example of an open source project that you are using that showcases those features?
In your opinion, is jj losing steam? How do you feel about it requiring a CLA to contribute to, which would enable the developers to go closed-source and keep all the contributions of the community? Are you afraid of vendor lock-in?
> a public example of an open source project that you are using that showcases those features?
IMO that's almost fundamentally impossible. The whole point is that it makes maintaining a "clean" linear commit history easier than git, so by the time you push anything the observable differences are already gone. At best you just see a repo without any WIP-style commits, which can be done using more effort and git alone.
CLI is preferred from my point of view if only because it's the supported standard interface. VSCode's version control interface is reasonably close to a standard, and I love the tree view there. But generally I don't want to use random third party things because they either aren't consistent (i.e. now I have to learn what joe schmoe thinks a version control interface should be), or don't stick around long enough.
But I don't tell my team what they have to use as long as it's compatible. The last thing I want to do is try to teach them a different VCS when they've spent time learning how to get by with git.
I've told them about JJ, what I like, what pain points it solves for me. They are all adults, they can make their own decisions about it.
JJ is for me. It helps me unwind some rather elaborate problems we've had with rebasing, or dealing with conflicts from bringing some old support branch up to date, or back porting some particular thing. It helps me create pretty clean MRs, break things up into smaller reviewable chunks, etc. You can do all this with git, but it's easier in JJ. In particular if you have to do shuffling and rebasing and conflict arise. JJ lets you solve conflicts asynchronously to rebasing. Git stops you in your tracks. There's some rerere thing that is supposed to help but give me a break. Git's a train wreck of dumpster fires. But it's still been basically better than everything else since svn.
Until JJ IMHO. Valid arguments can be made for other things like maybe mercurial or even fossil. But JJ is the most pain-free vcs evolution I've seen.
Mercurial's big failing was making too many critical things optional. It's the same screw-up riscv is going through now in the chip architecture space.
This is interesting and I think the analogy to playing a song in your head is a good one.
With regard to how visual the minds eye is, I have this anecdata:
For me, when I’m visualizing mentally, usually with a memory, replaying something, I stop registering what my eyes are presently seeing. No visual memories form of what my eyes were seeing while I was absorbed with reliving the memory or visualizing some idea.
I have to be careful about this when I get bored while driving.
I'm aphantasic, but I can relate to almost going blind with inattention. Not like I now have a black visual field, but just losing awareness of what is actually in it, like it is some sort of matte painting put up during intermission. But there remains some lower-level processing, because I may reflexively kick back into full seeing due to some unexpected change.
For my "mind's ear", imagining sound is not quite as rich as hearing, but quite similar. I can even "feel" aspects like bass thumping my chest or my ears tightening up due to an imagined or remembered loud noise. I can nearly feel dampness on my skin if I remember the roar of a waterfall.
Still, it's a bit more hollow than real hearing. Also, it is nearly a second channel of sound processing. I can have music playing on speakers and then think about a different song. The one going in my mind's ear does not mute out the real signal from my ears, nor do they really blend together. If the real audio is loud or catchy enough, it will derail my effort to sustain the secondary imagined program.
It's all about the test suite. The test suite becomes an executable specification, and the better the spec, the better the results you can get from AI.
It's part of it but you need good systems thinking in order to write good tests. The kind of thinking which requires years of software engineering experience to acquire.
Also the architecture really matters now because a poor architecture will lead to more weird edge cases and require more exotic tests to plug all the holes.
The very first thing many people did with AI is start calling the automated tests it writes good enough to capture the desired behavior. Writing tests isn't a whole lot of fun for most, so now the thing that it is supposed to help ensure we're not evolving our software into piles of trash is, in fact, one of the most neglected parts of vibed out codebases.
No it really is about the test suite, and provably so. As another poster pointed out, speed is a superoptimization problem and the test suite provides the constraints. If the constraints are appropriately set, even a naive genetic algorithm will eventually improve the outcome over time, provided suitable mixing of the proposal scheme. LLMs provide measurably better proposals than naive approaches, so the entire chain is sound. The issue really is an inability to set appropriate constraints on what the user is looking for, and poor quantification of the multiple objectives one should try to balance in practice. What's great is that's a human problem. Diverting that to the models is obviously a disaster.
I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.
I hope you understand the context in which that was said. The point of that statement is that the only way to rigorously verify correctness of a program is by using formal methods. Those are often too difficult to achieve by humans, which is why there is an entire program of developing autoformalization using LLMs. You are supporting what I have said.
In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.
The test suite is the same thing as the code, just approached from the other side.
i want the code to do this --> write the code to do this
i want the code to do this --> write the tests to specify this
the hard part is not the code or the tests. it's knowing what 'this' is supposed to be, exactly.
if you have the code, it is trivial to write the tests that will perfectly test the code.
if you have the tests, it is trivial to write the code that will perfectly pass the tests.
however, none of this has anything to do with whether the code or the tests are good, and having a bad version of one will not help you write a good version of the other.
It doesn't sound very intelligent to just brute force iterate until you pass all the tests. This sounds like what Rich Hickey called "bashing into the guard rails". Even if we assume the tests suite is perfect and will guide the model to a correct solution, maybe this is one case we really should just say "LLM" rather than "AI".
> Only the small and unimpressive programs can be checked exhaustively.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
Not true at all! Most of the HTTP APIs, and a good chunk of the webapps, that I've worked on can be defined as a combination of an API spec that carves out valid and invalid behaviors, and a set of behavioral tests for the workflows that the client users care about. Working from a codebase which is generated from a spec document (e.g. OpenAPI or gRPC) and use of tools like https://pkg.go.dev/net/http/httptest and https://bun.com/docs/test/dom makes this a pretty achievable goal in practice.
reply