Comments

You must log in or register to comment.

neku wrote

dude sssh she's on this website. you'll hurt her feelings

6

devtesla wrote

This is true but I like the anime girl and I think it would be complex to adjust the site to require JavaScript like this.

4

nitori OP wrote

I like the anime girl too, and I appreciate that jstpst uses a difficulty of 2 which is practically instant lol

4

flabberghaster wrote

This is intriguing to me but I lack the access to the site to tell if the bot plague is affecting us, and also the know-how of how to configure our webserver to do this properly.

I will say that anubis frequently hits me with false positives on my phone and I can't so much as browse the site sometimes, i'd like to not need it anymore.

3

twovests wrote

Oh these should just be in the bog standard caddy logs

3

nitori OP wrote (edited )

I wonder if JavaScript is even needed at all if one just wants to keep out badly-written scrapers that DDoSes their server. If the scraper doesn't keep state then simply return a 418, require a cookie to be set with Set-Cookie, and use meta refresh?

2

twovests wrote

I'm hesitant, because (1) every scraper I've ever written used a webdriver (usually Chrome) and it ran Javascript and everything you'd expect from a browser, and (2) I'm lazy.

There definitely still are bots using JavaScript, and getting by Caddy, since we do have the bots that are getting through Anubis and are obviously poking for vulnerabilities.

We had a litany of different bots before that, not just Claude bot, but I don't think I actually did (or could, from the logs) see if they use JavaScript. It did seem to stop the big LLM scrapers, though.

2