post-consumption-example.sh

[tido-]

Hi,
I run Paperless in Docker and I would like to amend the filename "post-consumption" with some valuable information, if Paperless finds some with tesseract.
To start, I have simply adjusted the paperless.conf within the docker-compose.env (I pasted the whole paperless.conf in the .env) and uncommented the line :
PAPERLESS_POST_CONSUME_SCRIPT="/home/paperless/post-consumption-script.sh" and pasted the text from the example post-consumption-example.sh in there.

If the script works, where would I see the output?

[Tooa]

Try docker-compose logs -f consumer in the directory containing the docker-compose.yml file. Does this answer your question?

[tido-]

Does it answer it.. well, it helped to identify my problem. Thank you for this command, interesting. I added the -t for time.
I have re-created the container and lost all the logs, arrrgh. Now, I cannot check for the same pattern. Maybe I can forward the logs to the hosts /var/log/ not to lose it anymore?

Within this command I found out, that the container cannot access the file I guess:
consumer_1 | 2020-05-01T20:14:30.699681946Z FileNotFoundError: [Errno 2] No such file or directory: '"/home/tido/paperless/scripts/post-consumption-0.sh"'

So, I have to hunt for File, User, Group permission - I guess. However, I gave Read + Execute permission to everyone:

tido@rockpi-4b:~/paperless/scripts$ ls -la
total 32
drwxr-xr-x  2 tido tido 4096 May  1 17:33 .
drwxr-xr-x 13 tido tido 4096 May  1 17:19 ..
-rw-r--r--  1 tido tido 4039 May  1 17:05 docker-entrypoint.sh
-rw-r--r--  1 tido tido 1241 May  1 17:05 gunicorn.conf
-rw-r--r--  1 tido tido  230 May  1 17:05 paperless-consumer.service
-rw-r--r--  1 tido tido  280 May  1 17:05 paperless-webserver.service
-rwxr-xr-x  1 tido tido  678 May  1 17:33 post-consumption-0.sh
-rwxr-xr-x  1 tido tido  678 May  1 17:05 post-consumption-example.sh

tido:x:1000:1000:Tido,,,:/home/tido:/bin/bash?
I also tried: chmod 0777 post-consumption-0.sh, but I got the same error. Strange side effect, the document gets deleted in the consumer-folder, but it is not visible in Paperless. Also the logs in Paperless do not show details like 2020-05-01T20:36:31.781933685Z Parsing for deu just empty, like nothing has happend.
Any suggestion or information I can provide to identify the problem?

[Tooa]

The path for PAPERLESS_POST_CONSUME_SCRIPT i.e. /home/tido/paperless/scripts/post-consumption-0.sh looks like a path on your host machine. As long as the directory is not mounted into the container, it cannot access the file. Have a look at how docker handles volumes and isolates containers from your host. Note this is more like a general docker question rather than paperless specific.

You can pass the scripts into the consumer container via a host mount:

consumer:
        ...
        volumes:
            - /home/paperless/scripts:/path/in/container/scripts/

where PAPERLESS_POST_CONSUME_SCRIPT is /path/in/container/scripts/post-consumption-0.sh

[tido-]

I have added these changes to the .yml :

        volumes:
            - data:/usr/src/paperless/data
            - media:/usr/src/paperless/media
            # bind mounts = The file or directory is referenced by its full or 
            # relative path on the host machine.
            - /home/tido/paperless/scripts:/home/scripts

and these changes to .env:
# PAPERLESS_POST_CONSUME_SCRIPT="/home/tido/paperless/scripts/post-consumption-0.sh"
PAPERLESS_POST_CONSUME_SCRIPT="/home/scripts/post-consumption-0.sh"

I then destroyed both container images: docker rm CONTAINER-ID' and spawn new containers with docker-compose. The bind mount seems to work:

root@rockpi-4b:~# docker exec -it paperless_consumer_1 bash
bash-5.0# cd /
bash-5.0# cd /home/scripts/
bash-5.0# ls -la
total 32
drwxr-xr-x    2 paperles paperles      4096 May  1 17:33 .
drwxr-xr-x    1 root     root          4096 May  2 13:24 ..
-rw-r--r--    1 paperles paperles      4039 May  1 17:05 docker-entrypoint.sh
-rw-r--r--    1 paperles paperles      1241 May  1 17:05 gunicorn.conf
-rw-r--r--    1 paperles paperles       230 May  1 17:05 paperless-consumer.service
-rw-r--r--    1 paperles paperles       280 May  1 17:05 paperless-webserver.service
-rwxrwxrwx    1 paperles paperles       678 May  1 17:33 post-consumption-0.sh
-rwxr-xr-x    1 paperles paperles       678 May  1 17:05 post-consumption-example.sh

And exactly the same situation as before just like @lionawurscht, strange side effect, the document gets deleted in the consumer-folder, but it is not visible in Paperless. Also the logs in the Webinterface of Paperless (Django) do not show any details, just this: May 2, 2020, 1:34 p.m. | Starting document consumer at /consume with inotify, like nothing has happend. If I look into the logs of the container, I can see all messages like again:

consumer_1 | 2020-05-02T11:34:43.253933808Z FileNotFoundError: [Errno 2] No such file or directory: '"/home/scripts/post-consumption-0.sh"'

[tido-]

Just for fun, commented this line:
# PAPERLESS_POST_CONSUME_SCRIPT="/home/scripts/post-consumption-0.sh"
destroyed both container images and spawn new containers. Everything works just like before.

[tido-]

At all, if you have a bare metal or Virtual Environment installation of Paperless, can you please try what happens if you uncommented this line:
# PAPERLESS_POST_CONSUME_SCRIPT="...
in paperless.conf and restart your Paperless. Feed it with a new document and look at the log files and if the document gets added in Paperless?

While reading the issues, I saw in the PRs often the requests for tests.
Is there a debug-mode that could help to identify the problem?
And if so, is this easy to run?

[tido-]

Hi, it's me again - never give up :-$

I got the hint to try a different path within the container: - /home/tido/paperless/scripts:/usr/src/paperless/scripts

This looked very promising at the begin, the log started to appear in Django until the guessed language wasn't available.
This gave Paperless a greater hiccup, and it fall back in its old behavior.

consumer_1   | 2020-05-04T20:14:42.527122969Z OCRing the document
consumer_1   | 2020-05-04T20:14:42.529519049Z Parsing for deu
consumer_1   | 2020-05-04T20:14:51.291139199Z Parsing for cym
consumer_1   | 2020-05-04T20:15:00.445359592Z PARSE FAILURE for /consume/Receipt_002506.pdf: The guessed language (cy) is not available in this instance of Tesseract.

Here it says run_post_consume_script Popen(( ... should it look like that? Popen is to call Subprocess management or so?

consumer_1   | 2020-05-04T20:46:11.622742085Z   File "/usr/src/paperless/src/documents/signals/handlers.py", line 77, in run_post_consume_script
consumer_1   | 2020-05-04T20:46:11.622790211Z     Popen((
consumer_1   | 2020-05-04T20:46:11.622810628Z   File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
consumer_1   | 2020-05-04T20:46:11.622822586Z     self._execute_child(args, executable, preexec_fn, close_fds,
consumer_1   | 2020-05-04T20:46:11.622832211Z   File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
consumer_1   | 2020-05-04T20:46:11.622841545Z     raise child_exception_type(errno_num, err_msg, err_filename)
consumer_1   | 2020-05-04T20:46:11.622851170Z FileNotFoundError: [Errno 2] No such file or directory: '"/usr/src/paperless/scripts/post-consumption-0.sh"'