username instead of just uid in subuid now required for rootless podman

[qhaas]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

In podman 3.x, one could use one's uid (i.e. uid -u) e.g.

$ grep $(id -u) /etc/subuid
16642:4294836224:65536

instead of one's username (i.e. whoami), e.g.

$ grep $(whoami) /etc/subuid
nqh:4294836224:65536

when mapping a subuid range for rootless podman (ditto for subgid). It is unclear if this was intentional, given the troubleshooting guide says to use the username. In podman 4.0x, using the uid causes some podman commands to fail. Oddly, some functionality works (e.g. podman unshare), but others do not (e.g. podman pull), which might mean something is amiss under the hood given the inconsistency.

My systems were using uid instead of username for subuid mapping (ditto for subgid), so this resulted in some troubleshooting when we upgraded from podman 3.0.x to podman 4.0.x before it was discovered that replacing the uid with the username resolves the issue that occurs with a subset of podman commands. So, thought I'd report the observation.

Steps to reproduce the issue:

1. Use numeric uid (i.e. uid -u) instead of username (i.e. whoami) in /etc/subuid (ditto for gid)
2. With podman 4.0.x, verify podman unshare works as expected
3. Observe podman pull registry.access.redhat.com/ubi8/ubi-minimal fails
4. Downgrade to podman 3.0 using dnf module switch-to container-tools:3.0, followed by podman system migrate
5. Verify podman pull registry.access.redhat.com/ubi8/ubi-minimal succeeds
6. Revert back to podman 4.0 using dnf module switch-to container-tools:rhel8, followed by podman system prune -af and podman system migrate
7. Repull the image to verify it still fails: podman pull registry.access.redhat.com/ubi8/ubi-minimal
8. Replace uid with username in /etc/subuid, ditto for gid and subgid
9. Verify podman pull registry.access.redhat.com/ubi8/ubi-minimal succeeds

Describe the results you received:
For expanded debug output, see podman4_log.txt

$ podman pull registry.access.redhat.com/ubi8/ubi-minimal
Trying to pull registry.access.redhat.com/ubi8/ubi-minimal:latest...
...
Error: writing blob: adding layer with blob "sha256:6963f872abface896838f7f855db3c316f6d9ded4aa57deae35c0600c8ecb61d": Error processing tar file(exit status 1): potentially insufficient UIDs or GIDs available in user namespace (requested 0:12 for /var/spool/mail): Check /etc/subuid and /etc/subgid if configured locally: lchown /var/spool/mail: invalid argument
$ echo $?
125

Describe the results you expected:

$ podman pull registry.access.redhat.com/ubi8/ubi-minimal
...
$ echo $?
0

Additional information you deem important (e.g. issue happens only occasionally):
Every time

Output of podman version:

Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.17.7

Built:      Tue Apr 19 06:16:32 2022
OS/Arch:    linux/amd64

Output of podman info --debug:

See podman_info_debug.txt

Package info (e.g. output of rpm -q podman or apt list podman):

podman-4.0.2-6.module+el8.6.0+14877+f643d2d6.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes: Latest stable version of Podman 4.x in the x86-64 RHEL 8.6 repo was tested, i.e. the one deployed using dnf module install container-tools:rhel8

Additional environment details (AWS, VirtualBox, physical, etc.):
physical RHEL 8.6 system

[vrothberg]

Thanks for reaching out, @qhaas.

@giuseppe PTAL

[giuseppe]

PR here: #1265

[vrothberg]

@qhaas, if you desire a backport in RHEL, please go through the Red Hat customer channels (e.g., Bugzilla).

[qhaas]

Thanks for the quick response and turn-around!