Is your feature request related to a problem? Please describe.
i'm currently investigating why some crawlers (namely: headless browsers like lightpanda) are not blocked, even though they should be, as per
|
- name: lightpanda |
|
user_agent_regex: ^LightPanda/.*$ |
|
action: DENY |
afaict, the above rule should be triggered, as I'm using import: (data)/bots/_deny-pathological.yaml which in turn does an import: (data)/bots/headless-browsers.yaml.
however, my tracking done is solely based on comparing my local policy with the files found in https://github.com/TecharoHQ/anubis/blob/v1.26.2/data/, which is rather tedious.
also, I never know whether these are the actual files included in my anubis binary.
Solution you would like.
it would be ace if there was some -dump-policy flag that would dump the actually used policy with all imports resolved. (ideally with some annotation, where a given configuration originates from; but I guess this is harder to do, and just for extra effects).
Describe alternatives you have considered.
probably having an option to export the entire built-in (data).
ah well, only now i see that the debian package of anubis comes with /usr/share/doc/anubis/data/ which already does this. but again, i have to trust that the package keeps the anubis binary in sync with the shipped data files (afaik, the files in /usr/share/doc/anubis/data/ are only shipped for the convenience of the user, and not actually used by anubis)
Additional context
No response
Is your feature request related to a problem? Please describe.
i'm currently investigating why some crawlers (namely: headless browsers like lightpanda) are not blocked, even though they should be, as per
anubis/data/bots/headless-browsers.yaml
Lines 1 to 3 in b1d52ba
afaict, the above rule should be triggered, as I'm using
import: (data)/bots/_deny-pathological.yamlwhich in turn does animport: (data)/bots/headless-browsers.yaml.however, my tracking done is solely based on comparing my local policy with the files found in https://github.com/TecharoHQ/anubis/blob/v1.26.2/data/, which is rather tedious.
also, I never know whether these are the actual files included in my anubis binary.
Solution you would like.
it would be ace if there was some
-dump-policyflag that would dump the actually used policy with all imports resolved. (ideally with some annotation, where a given configuration originates from; but I guess this is harder to do, and just for extra effects).Describe alternatives you have considered.
probably having an option to export the entire built-in
(data).ah well, only now i see that the debian package of anubis comes with
/usr/share/doc/anubis/data/which already does this. but again, i have to trust that the package keeps the anubis binary in sync with the shipped data files (afaik, the files in/usr/share/doc/anubis/data/are only shipped for the convenience of the user, and not actually used by anubis)Additional context
No response