> EmDash depends on Dynamic Workers to run secure sandboxed plugins. Dynamic Workers are currently only available on paid accounts. Upgrade your account (starting at $5/mo) or comment out the worker_loaders block of your wrangler.jsonc configuration file to disable plugins.
Seems like a pretty poor replacement for WordPress, which will run on pretty much whatever random system happens to be within eyesight.
The sandboxing requires CF workers though, as it depends on V8 isolates. Seems fair enough, but it'd be nice to see easily deployable cross-platform support someday for isolates or something similar, and not just for emdash.
This doesn't "verify" anything. I can't see the code before, I can't see the code after. I can't verify they do the same thing. It won't be updated if there's a long churn of issues and breakage coming from this work plaguing the team for years. The only thing it verifies is that Asana did indeed make the claim, which I don't think anyone was doubting.
> Back in 2022, we set out to migrate Asana's frontend test suite off Enzyme, our aging testing library, and onto React Testing Library (RTL).
Your telling me they had a full team of engineers at Asana, doing nothing but rewriting tests for 4 years until AI came along and did the last year in a couple days? I'm extremely doubtful. I don't doubt for a minute they were one-track to take 5 years, but not because it was 5 years of engineering effort for humans.
Far more likely they finally cleared up some tech debt they had been plugging at off and on for 4 years, then a PR flack got ahold of it and it became a breathless "AI did 5 years of work in a couple days".
I'm not sure I agree with the extremely easy to verify part. How do you validate that your new test suite covers the exact same edge cases as your old?
But yes, LLMs are great at tests. I'm not doubting that an LLM migrated some legacy tests, or that it was a task taking a long time. I am doubting the way it was presented in the article, that this was taking a full team of engineers dedicated to nothing but this, 5 years to accomplish (and presumably already spent 4 million working on this, since the total estimate was 6 million, and they've been at it for 4 year).
I think some PR flack got ahold of the fact that an LLM wrapped up migrating some legacy tests that a team had been slowly chipping away at for 4 years, between their normal feature work, and were on track to finish in 5, then wrote it up like it was that teams entire focus instead of a piece of tech debt.
That makes far more sense to me than spending millions for a team of engineers dedicated to nothing but rewriting an existing test suite.
> Migrating a test suite is exactly the kind of work that LLM's excel at beause it's extremely easy to verify
I don't think that's true. All you see is all the test pass. You don't know that the tests still cover everything that they used to.
LLMs excel where there is an excellent test suite, and you ask them to modify the thing that the test suite tests, and forbid them from changing the tests.
How is it bad faith to question corporate marketing blog posts? That's not bad faith, that's table stakes for critical thinking.
We absolutely should be skeptical when an AI company makes big claims. The fact that the company the AI company is talking about also claims the same thing doesn't change that. OpenAI is getting awareness and marketing out of this, and I'm sure Asana is getting something out of it too.
And I'm not even saying OpenAI or Asana are necessarily lying. Asana might not find out for months or years that some tests had been rewritten poorly, and don't sufficiently test the thing they were supposed to test anymore. For example. If they truly had 5 years of work, then I find it hard to believe that in two weeks of the LLM churning, they had the time to review all the new tests. They spot-checked, at best.
Maybe everything is great. Maybe the LLM did a wonderful job, and this was awesome for Asana. But we have no idea, and we're unlikely to ever find out. Unless, of course, it's in Asana's interest from a marketing perspective to tell us.
(Not sure what the URLs you posted in your comment are supposed to prove. They're unrelated to the issue at hand.)
This is something that frustrates me a lot when I see people jumping on a piece of writing as AI generated, these AI-isms didn't show up in a vacuum. Yes, AI commonly overuses some patterns, but it got those patterns from it's training data because real people wrote (and continue to write) that way. A few matching sentences here or there is not the smoking gun people make it out to be.
The other thing that is characteristic of AI text is messing up the underlying meaning or fine details of what it's saying. It's not just that it's imitating people who write bad contrasting clauses, it also can't write good ones reliably.
edit: Ironically that last sentence is not super clear (but hey writing is hard and I'm constantly exposed to slop). My point is the nature of AI makes its writing worse than its training data, it's not all bad training data.
To be fair to committees, the above quote is explicitly from a single person and I expect that is true of most AD copy simply because having one person write it is cheaper.
Committees are more about stamping out flavour by making what exists more bland, rather than adding endlessly regurgitated idioms.
Catching a murderer is much, *much* easier than catching someone using a modern AI model to generate code, especially if they actually read it and fix occasional LLMisms, and not just vibecode from the hip. The societal damage is also infinitely more serious in the case of murder.
If your argument is "this is unenforceable" and "failure to enforce is a low-stakes problem" then I'm not sure what your concerns are, other than a desire to control how others choose to spend their time. Either of those might be a valid objection (although I do not consider them to be) but together they sort of cancel each other out, no?
Regardless, some efforts like this are explicitly performative. In the event some problems arise and it turns out someone used a specific tool in the process of causing those problems, administrators (and insurers) can say "look we forbade the use of that tool, so this is on them and not us." In other words, enforcement is not always even the point, but it always seems to be the first stop on the concern-trolling trolley.
Finally, yes, people can misbehave and break rules, and the more effort they put into doing so, the more successful they will be at the rule breaking. Nothing about that is justification for not having rules, even when they are hard to enforce.
It's worse than that! Even when the product managers inputs to the engineers are constrained for many problems no two engineers will produce consistent results! In fact recent studies suggest that even the same engineer may produce different results based on mood, if they've had coffee yet, and how near to EOD it is.
Far be it from me to doubt a stranger on the Internet, but can you point out any medical textbook or dictionary written pre-2020 that limits the definition of "vaccine" to "attenuated viruses"?
I mean, a company I worked at had a significant amount of money stolen after the attackers spent 6 months sitting on their access waiting for the right moment to fake an (expected) reply to an email exchange. The original breach (or at least the breach of this executives account) involved a very targeted phish. When the potential payout is millions it justifies a lot of effort.
https://github.com/emdash-cms/emdash
> EmDash depends on Dynamic Workers to run secure sandboxed plugins. Dynamic Workers are currently only available on paid accounts. Upgrade your account (starting at $5/mo) or comment out the worker_loaders block of your wrangler.jsonc configuration file to disable plugins.
Seems like a pretty poor replacement for WordPress, which will run on pretty much whatever random system happens to be within eyesight.
reply