That's a better way to put the goal. But does it actually raise the cost or reduce asymmetry in a meaningful way? If it makes crawling 0-5% slower then it's really not worth the annoyance.

I know having the Anubis javascript at all gets in the way of some bots. But is the calculation helping enough?

> does it actually raise the cost or reduce asymmetry in a meaningful way?

It's complicated. I think anubis currently only collects low hanging fruit so to speak but in practice that's the bulk of what was causing problems so it's good enough.

PoW in general absolutely yes - I think monero clearly demonstrates this. Last I checked it remained viable to mine monero on at least somewhat low end and at least somewhat outdated consumer CPUs. It seems to me that fact implies that a well constructed PoW scheme can in practice squash the impact of any performance differences to within at most a couple orders of magnitude.

But a couple orders of magnitude is still a ton of asymmetry.

And even if you can guarantee the same CPU time on bots and legitimate requests, you still have a bunch of asymmetry in favor of the bots.

Sure. Even if PoW in general is sound, when applied to scraping it's always going to be a matter of what the client is willing to "pay". It's entirely within the realm of possibility that a scraper could be willing to "outbid" the user of a 5 year old tablet.

But it's also true that charging anything is going to be expected to reduce server load, that the cheaper something is the more carelessly people will tend to make use of it, and that at least in general bot operators are expected to be more cost sensitive than regular visitors on account of scale.

To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.

I keep seeing comments along the lines of anubis only working because it bounces bots running lightweight setups without js but stop and ask yourself why are they running such lightweight setups to begin with?

> why are they running such lightweight setups to begin with?

It's easier and still gives access to almost everything. I don't think CPU limitations are the main issue there. As far as computer resources, I'd bet RAM is the bottleneck to running the script, not cycles. That suggests a RAM-heavy hash might do better than SHA256, but still agrees with the idea that the PoW in Anubis is not the important part.

> To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.

I considered that. But while a visitor is idle they need the result quickly and they face the full brunt of the challenge all at once. A scraper can be working on a thousand sites at once so the per-request impact of the challenges is a lot lower, and patiently working through challenges in the background usually reduces the real cost, potentially to almost zero.

And you'd better not need multiple gigabytes of RAM to visit your site, that'll freeze my computer.