tty device used inside toolbox gets odd permissions on host causing pinentry/gpg-agent to not work

[dustymabe]

Describe the bug

My setup includes running gpg-agent on my host and doing development (git commit, etc) inside my toolbox container. When I git-commit I sign the commits, which means the process talks to the agent that then runs a pinentry program to ask the user for the passphrase.

When running in the terminal I prefer for the passphrase to be asked to me in the terminal window and not in a popup. So I'll unset DISPLAY to force it to use the terminal window. This stopped working sometime in the past few months. I finally tracked down some more information on why.

When running inside toolbox the permissions on my tty device are as expected:

$ ls -l $(tty)
crw--w----. 1 root nobody 136, 14 Sep 27 13:37 /dev/pts/14

However outside of toolbox (from the perspective of the host) the
permissions look odd:

[dustymabe@media ~]$ id
uid=1001(dustymabe) gid=1001(dustymabe) groups=1001(dustymabe) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[dustymabe@media ~]$ ls -l /dev/pts/14
crw--w----. 1 100000 tty 136, 14 Sep 27 14:39 /dev/pts/14

I would expect the owner of the tty to be the user who launched the toolbox container (1001). If I change the ownership to the expected UID then my pinentry program appropriately asks for the passphrase on the same terminal window (inside toolbox) where I am running git commit.

I'm running Fedora 33 with:

$ rpm -q toolbox podman
toolbox-0.0.95-1.fc33.x86_64
podman-2.1.0-0.187.rc2.fc33.x86_64

Indeed if I go back to an old machine I haven't updated in a while I can see that the permissions are what I would expect them to be. This out of date machine has:

[dustymabe@hattop ~]$ rpm -q toolbox podman 
toolbox-0.0.18-1.fc31.noarch
podman-1.8.0-2.fc31.x86_64

Steps how to reproduce the behaviour

1. Launch toolbox container
2. Inspect owner inside the container: ls -l $(tty). Should show up as root since you are inside a user namespace.
3. Inspect the owner of that same device outside the container and verify the permissions looks weird.

[debarshiray]

Thanks for narrowing it down to those working and broken versions.

I wonder if this might be solved by using podman create ... --mount type=devpts,destination=/dev/pts as introduced in containers/podman#7209

I don't think Toolbox itself does anything funky with the TTY devices, but it does bind mount the entire /dev from the host into the container. So I wonder if we can reproduce the same with podman run ... ?

[dustymabe]

Thanks for narrowing it down to those working and broken versions.

No problem :) - It's worth noting those were just two data points. I don't know exactly when the issue was introduced.

I wonder if this might be solved by using podman create ... --mount type=devpts,target=/dev/pts as introduced in containers/podman#7209

Can you recreate locally? I'm assuming pretty much anyone running latest toolbox/podman should see a weird owner when running ls -l on one of the tty devices used for a toolbox session. If you see the problem maybe you can try that in a development version of toolbox?

I don't think Toolbox itself does anything funky with the TTY devices, but it does bind mount the entire /dev from the host into the container. So I wonder if we can reproduce the same with podman run ... ?

Right, I suspect it was a change in podman that caused the regression, just wasn't sure so I reported here first.

[debarshiray]

Adding --mount type=devpts,destination=/dev/pts to podman create only changes the group ownership of the TTY device inside the container from nobody to tty:

$ ls -l $(tty)
crw--w----. 1 root tty 136, 0 Oct 14 20:07 /dev/pts/0

That seems to be a step in the right direction in terms of matching the TTY device's ownership to that on the host, but I don't know if it solves your problem.

I don't know if you figured it out already, but the strange ownership that you are seeing is related to the use of --userns=keep-id. The 100000 UID on the host is the UID of root inside the container's user namespace, the nobody GID inside the container is due to the host's tty group being absent inside the container.

[dustymabe]

Right, but what we need is for the owner of the tty device on the host to be the user who started toolbox. For example, to fix this problem right now for me I sudo chown dustymabe /dev/pts/14 on the host.

Before changing the ownership whenever I git commit it will fail because I have it configured to sign commits and pinentry from the gpg-agent running on the host can't access the tty toolbox is using. After correcting the ownership I can type my password in to the pinentry program that pops up in my terminal.

[debarshiray]

By the way, I submitted #581 for the --mount type=devpts,destination=/dev/pts change. @giuseppe had been asking me to do so for a while, so better late than never.

[debarshiray]

Right, but what we need is for the owner of the tty device on the host
to be the user who started toolbox. For example, to fix this problem right
now for me I sudo chown dustymabe /dev/pts/14 on the host.

Yes, I understand.

Right now I can think of a (crude?) way to fix this. /usr/bin/toolbox is the entry point of the containers. So, instead of merely sleeping, we can install an inotify watch on /dev/pts and have it do chown <user> every time a new file is created there.

[dustymabe]

I'm not sure if this can be done inside the container or if it needs to be done as part of container setup (i.e. podman). I can't do the sudo chown dustymabe /dev/pts/14 from inside the container (inside the container you get chown: changing ownership of '/dev/pts/14': Operation not permitted). It has to be from the host.

[debarshiray]

If you have a new devpts file system mounted on /dev/pts in the container (ie., --mount type=devpts,destination=/dev/pts), then I think you can. :)

Otherwise, the /dev/pts is shared with the host, and the nobody group owner (because the host's tty group is absent inside the container) blocks the chown inside the container.

Just wanted to bump this up as it is unfortunately still a problem as of toolbox version 0.0.99.2 (fedora silverblue 34). The chown hack works temporarily as changing tty or exiting the toolbox resets permissions.

If it does help, the ssh credentials dialog works normally. I tried unsetting both GPG_TTY and DISPLAY, but to no avail.

Poked around a little bit more and found something interesting. I setup an empty git repo to test the issu. I found that this sequence of commands makes signing work inside the toolbox:

[user@fedora:repo]$ git commit -S -m 'Test' # you can cancel this
[user@fedora:repo]$ toolbox enter
[user@toolbox:repo]$ git commit -S -m 'Test' # this will sign

However this sequence will break signing both for the host and the toolbox:

[user@fedora:repo]$ toolbox enter
[user@toolbox:repo]$ git commit -S -m 'Test' # this will fail
[user@toolbox:repo]$ exit
[user@fedora:repo]$ git commit -S -m 'Test' # this will fail

Moreover, on my system the chown hack seemingly stopped working. I don't know if something magically broke or if it was working due to a rogue gpg-agent on the host...

These findings make me think that this issue also involves gpg-agent other than toolbox/podman. What exactly is going wrong I can't tell since this is out of my scope. However if any testing is needed I am happy to help

[giuseppe]

what are the permissions in /dev/pts/?

Is the container running with --mount type=devpts,destination=/dev/pts?

The container is running with --mount type=devpts,destination=/dev/pts and the permissions (inside the container) are the following:

$ ls -l /dev/pts
drwxr-xr-x.  2 root   root        0 Oct 27 22:03 .
drwxr-xr-x. 20 nobody nobody   4300 Oct 27 22:01 ..
crw--w----.  1 root   tty    136, 0 Oct 27 22:04 0
crw-rw-rw-.  1 root   root     5, 2 Oct 27 22:03 ptmx