46 Commits

Author SHA1 Message Date
Alan Orth
85a8f0d9ec examples: remove erroneous whitespace 2025-05-17 23:45:39 +03:00
WeebDataHoarder
3abdc2ee5b examples: add private / localhost networks to snippets and forgejo/generic examples 2025-05-13 03:06:23 +02:00
WeebDataHoarder
fb6c5c3eb4 examples/forgejo: remove standard-bots rule, it's redundant 2025-05-03 22:43:09 +02:00
WeebDataHoarder
44c9114ae5 challenges: add refresh via JavaScript window.location 2025-05-03 21:35:12 +02:00
WeebDataHoarder
4b1878f1ac examples/forgejo: exclude fetchers from suspicious crawler 2025-05-03 21:21:13 +02:00
WeebDataHoarder
1ea19c5a6c state context: Added proxy-safe-link-tags to proxy <link> tags, use specific LinkTags ranger on templates instead of raw elements 2025-05-03 04:12:58 +02:00
WeebDataHoarder
736c2708e9 examples/forgejo: exclude fetchers from TLS Fingerprint rule 2025-05-02 22:21:40 +02:00
WeebDataHoarder
3047dcfd4b examples/forgejo: Restrict meta tag fetching for likely bots 2025-05-01 16:15:28 +02:00
WeebDataHoarder
868c76eeb9 examples/forgejo: add commit graph endpoint to heavy resources 2025-05-01 14:20:03 +02:00
WeebDataHoarder
6e47cec540 examples/forgejo: allow releases summary-card fetch 2025-05-01 02:34:14 +02:00
WeebDataHoarder
e49c4ae72f action/context: add capability to set response headers 2025-04-28 12:40:03 +02:00
WeebDataHoarder
3234c4e801 feature: Implement <meta> tag fetcher from backends with allow-listed entries to prevent unwanted keys to pass 2025-04-27 21:40:59 +02:00
WeebDataHoarder
d36d8354a2 examples: clarify rules order, default action and standard-tools rule 2025-04-27 20:53:30 +02:00
WeebDataHoarder
d665036d98 examples: move desired-crawlers before undesired-networks 2025-04-25 17:59:16 +02:00
Alan Orth
6f3d81618c examples: add TikTokSpider
Requests using this user agent are coming from the same Amazon net-
works as Bytespider.
2025-04-25 11:02:48 +03:00
WeebDataHoarder
1f84f5e981 examples: forgejo: Add branches/tags listing on repo to API endpoints 2025-04-24 20:51:15 +02:00
WeebDataHoarder
a0224cb21c policy: allow fetching ASN directly via RADb WHOIS service 2025-04-23 20:35:20 +02:00
WeebDataHoarder
3b11792594 Implement policy snippets 2025-04-23 20:35:20 +02:00
WeebDataHoarder
d83fe3653a examples: update bot matches, allow badges to be fetched 2025-04-23 20:35:20 +02:00
WeebDataHoarder
ead41055ca Condition, rules, state and action refactor / rewrite
Add nested rules
Add backend action, allow wildcard in backends
Remove poison from tree, update README with action table

Allow defining pass/fail actions on challenge,

Remove redirect/referer parameters on backend pass

Set challenge cookie tied to host

Rewrite DNSBL condition into a challenge

Allow passing an arbitrary path for assets to js challenges

Optimize programs exhaustively on compilation

Activation instead of map for CEL context, faster map access, new network override

Return valid host on cookie setting in case Host is an IP address.
bug: does not work with IPv6, see https://github.com/golang/go/issues/65521

Apply TLS fingerprinter on GetConfigForClient instead of GetCertificate

Cleanup go-away cookies before passing to backend

Code action for specifically replying with an HTTP code
2025-04-23 20:35:20 +02:00
Alan Orth
5189dd41f9 examples: fix duckduckbot regex
Each IP entry in the table is wrapped in a div.
2025-04-23 00:40:12 +03:00
WeebDataHoarder
629e6b40b4 Expand homesite match for forgejo template 2025-04-19 13:32:51 +02:00
WeebDataHoarder
f04d801a43 Add user/org profile to homesite rule 2025-04-18 11:03:19 +02:00
WeebDataHoarder
2866cf8e80 Add undocumented Google-PageRenderer to Googlebot preset on forgejo 2025-04-16 23:34:28 +02:00
WeebDataHoarder
6458e6d019 Create negative match based on Forgejo reserved word list for embedding homesites 2025-04-16 18:48:34 +02:00
WeebDataHoarder
f690cfaac3 Move bots early 2025-04-16 18:12:00 +02:00
WeebDataHoarder
ce8bc52d94 Explicitly support plaintext browsers (Lynx) and serve challenges that they can solve 2025-04-15 21:03:22 +02:00
WeebDataHoarder
c84c67439e Only error when target network list is not reachable 2025-04-15 19:23:04 +02:00
WeebDataHoarder
a5be4faa8a Default to forgejo-auto on forgejo 2025-04-13 19:06:37 +02:00
WeebDataHoarder
b1620e4d92 Target alpn-less scrapers 2025-04-13 17:31:06 +02:00
WeebDataHoarder
1c0e015c69 Added JA4 sample to forgejo example 2025-04-13 16:43:27 +02:00
WeebDataHoarder
6d5aaddd03 New README 2025-04-13 13:10:56 +02:00
WeebDataHoarder
06e8556d68 Add generic template, update README 2025-04-13 11:50:03 +02:00
WeebDataHoarder
cb0a3732bd Update forgejo template 2025-04-13 11:24:58 +02:00
WeebDataHoarder
15472b00b8 Add older clients, change standard-tools challenge 2025-04-09 00:08:53 +02:00
WeebDataHoarder
ce111f6ae9 Add DNSBL querying in conditions 2025-04-08 22:11:58 +02:00
WeebDataHoarder
8c815ed808 Update readme with light instructions 2025-04-08 18:03:47 +02:00
WeebDataHoarder
713db376b5 Add login/activate paths to forgejo example template 2025-04-08 17:50:41 +02:00
WeebDataHoarder
f80d6ebd15 Set X-Away-Id on response 2025-04-08 13:13:19 +02:00
WeebDataHoarder
baf9df9f0a Allow conditions on challenges, and early hint deadline 2025-04-08 11:40:16 +02:00
WeebDataHoarder
f272a5ae72 Load challenge as style 2025-04-08 02:38:53 +02:00
WeebDataHoarder
2ce9709667 New challenge for HTTP/2 clients, preload-link 2025-04-08 02:17:03 +02:00
WeebDataHoarder
e08a5697f6 Minor cleanup of challenges code, bind session http cookies to issued tokens 2025-04-07 19:00:53 +02:00
WeebDataHoarder
40aa9709ea Add packages / actions API and tools to forgejo example 2025-04-07 08:56:09 +02:00
WeebDataHoarder
65561ab00e Add wasm helper, wasm test utility 2025-04-06 11:44:06 +02:00
WeebDataHoarder
6623824d44 Move policy examples to examples folder, rename to forgejo 2025-04-06 03:26:02 +02:00