mirror of
https://github.com/qdm12/ddns-updater.git
synced 2026-08-02 10:38:41 -04:00
Bug: PUID, PGID, and UMASK Being Ignored in Docker Compose - Fails to Start - Unable To Read / Write to NAS Folders #436
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @geekau on GitHub (Sep 14, 2024).
TLDR: PUID, PGID, and UMASK Being Ignored in Docker Compose - Fails to Start - Unable To Read / Write to NAS Folders
Is this urgent: Yes
DNS provider(s) you use: Cloudflare / DuckDNS
Program version: 2024-08-28T14:16:17.081Z (commit
bc272e0)Running version latest built on - 2024-08-28T14:16:17.081Z (commit bc272e0)What are you using to run the container: docker-compose with centralised environment file.
Extra information
Our downstream project (https://github.com/geekau/mediastack) uses Gluetun and DDNS-Updater to manage some of the networking, we use a single docker-compose.env file for all of the variables across 25 Docker applications, to minimise complexity, and make it easier for new users to get into Docker, and deploy their MediaStack / SelfHost environment.
We have the following 3 variables which are shared across all Docker Compose YAML files, however DDNS-Updater does not seem to pick them up, and we can't read / write to some NAS, as the UID/GID is fixed at 1000:1000.
2024-09-14T18:47:05+10:00 ERROR setting initial up to date status: opening file: open /updater/data/updates.json: permission deniedThe DDNS-Updater is failing to start on some NAS, where the read / write permissions do not allow 1000:1000 access.
On our Synology servers, we needed to change our folder permissions to 777, before the container will start.
I've reviewed the DDNS-Updater documentation again and noticed reference to UID and GID, however these appear to only be for build-time configuration, which won't suit many new users.
Changing PUID / GUID / UMASK manually without varibles, still doesn't address file permission - they don't appear to be listed as valid environment variables in documentation.
Logs:
Configuration file (remove your credentials!):
Host OS: Synology RS1221+ (DSM 7.2.2-72803)
@qdm12 commented on GitHub (Sep 14, 2024):
The docker image has a default 'hardcoded' user and group (see
USERinstruction in Dockerfile) and runs as that user from start, so it cannot change user/group at runtime. This is for security reasons and also since the docker image is based onscratch(no OS).What you could do however is set the
userfield in docker-compose.yml to specify the user id to use to run the container. For exampleuser: "1001"and you should be able to use a variable such asuser: "{PUID}", does that solve your issue?@qdm12 commented on GitHub (Sep 14, 2024):
See https://docs.docker.com/reference/compose-file/services/#user
@favonia commented on GitHub (Sep 14, 2024):
@qdm12 Sorry for my uninvited comment, but I think you meant
user: "${PUID}"(with a$), and for this case, maybe @geekau wanteduser: "${PUID}:${PGID}". Also, you can usedocker run --user ...with the same effects of settinguser: ...in the compose file. Maybe I misunderstood what you meant by "cannot change user/group at runtime".My 2 cents:
PUID,PGIDandUMASKare only a convention made popular by e.g. images produced by LinuxServer.io. They are just regular environment variables read by certain images to set user ID, group ID, or umask by themselves during run time. These variables have no special meanings and the Docker engine provides no special support for them. Even people at LinuxServer.io say the following in their documentation:In my opinion, unless we are talking about the Docker images that explicitly want you to set
PUIDorPGID, it's safer and more reliable for the Docker engine (instead of individual images/containers) to set the user/group IDs, as recommended by @qdm12. Once https://github.com/moby/moby/issues/19189 is resolved (if ever), we can also set umask directly via the Docker engine. I am hoping that we can move on from the legacy workaround one day.@qdm12 commented on GitHub (Sep 14, 2024):
@favonia exactly on point, thank you for correcting my comment 👍
@geekau commented on GitHub (Sep 14, 2024):
Thank you so much @qdm12 and @favonia I was incorrect in my thinking that
PUID,PGIDandUMASKwere Docker-wide variables, but do seem to be isolated to the LinuxServer and Hotio image fleet. However, the compose-file link and recommendation ofuser:have provided some progress.I've updated the docker-compose-ddns-updater.yaml file to:
This now allows the DDNS-Updater running container to read / write to the local Docker host filesystem as the PUID:PGID values specified in the docker-compose.env file, which is really what we're after - we don't want to change uid/gid internally of any docker images, just the values so they can read / write to the local filesystem, so
user:works well here.I also tested
user: ${PUID:?err}:${PGID:?err}in the Gluetun docker-compose, but it failed as Gluetun needs to run asroot- And that's OK, I understand why Gluetun needs to run asroot, its more mapping thePUID,PGIDandUMASKvalues to the local Docker host filesystem, so the logging / backups can interact with the local host with relevant permissions.I initially marked this as urgent, as I thought there would be a lot of NAS users not being able to map running users / processes to the local filesystem, however we can drop urgency on this now.
So outstanding questions I have:
Am I correct in understanding
user: ${PUID:?err}:${PGID:?err}allows the DDNS-Updater container to "run-as" these IDs, which means the PUID:PGID will be the same inside the container are inherited the same as the local docker host filesystem?Using
user: ${PUID:?err}:${PGID:?err}inside the Gluetun container fails to run, as Gluetun needs to run asrootuser, therefore Gluetun is not able to read / write as a different PUID:PGID mapped from the local Docker host computer?Is it possible (as a feature request), that Gluetun and DDNS-Updater are able to be updated, so the containers still run internally as their intended users (root / 1000), but there is a mapping of
PUID,PGIDandUMASKto the local Docker host values?Please correct me if I've misunderstood some of Docker's technical capabilities / implementation, you're work on these images are amazing - thank you.
Regards.
@qdm12 commented on GitHub (Sep 15, 2024):
Thanks!
@geekau commented on GitHub (Sep 15, 2024):
Ref 3... Am referring generally if the Gluetun / DDNS-Updater containers can still run as they've been designed, but allow the Docker processes for these containers to interact with the local Docker host, with different
PUID,PGID, andUMASKvalues.In our downstreat project, we use the
PUID,PGID, andUMASKvalues in an ENV file, which are then used by each of the containers, when Docker Compose deploys each of the YAML configurations - these are mostly LinuxServer and Hotio images.Then the containers still deploy / run with the build in user values, however they interact with the local filesystem with the values from the Docker Compose ENV file. This allows the container to read / write with a local user account the user may choose from... a bit like a port mapping, but file permissions - excuse the analogy.
Our project currently has 25 Docker containers, and our vision is to make it very easy for new users to be able to deploy Docker images with very little (to no), prior knowledge - hence we put all of the varibles into the one ENV file, which all the containers use when they deploy the Docker Compose YAML files.
We use the
PUID,PGID, andUMASKvalues in all the other containers, the then interact with the local Docker filesystem with a single dedicated account.Just curious whether this is technically feasible for Gluetun / DDNS-Updater?
Hope this explains it better.
@qdm12 commented on GitHub (Sep 16, 2024):
UMASK is added in
3a6262ef2cand should be effective from here. Although that will obviously depend on the user+group configured to run ddns-updater.Now PUID and PGID cannot be used for ddns-updater, since the user configured to run ddns-updater (even before the program starts) cannot be changed at runtime (unless it's root, but would there be a use case for this??)... this is as far as I know, maybe I'm missing something. I just learnt all about umask to be honest, so I may be lacking other knowledge!
@favonia commented on GitHub (Sep 16, 2024):
My 2.5 cents: personally I'm against using environment variables along the line of
PUIDandPGIDbecause they should have been part of Docker and it could be difficult for this project to drop it when Docker (finally) supports that, but this is my project... 🤷 @geekau A possible workaround or alternative approach might be usingsetfaclto set default permissions for the directory, but I didn't test it carefully.PS: podman already supports umask, so there's hope that Docker will follow, eventually.
@qdm12 commented on GitHub (Sep 16, 2024):
So that means you could set the system default umask right?
For ddns-updater previous to the UMASK commit I mentioned, it would arbitrarily set permissions on directories (700) and files (600) without taking care of umask. Now it's quite better, since it uses the OS default umask if UMASK is left empty, and uses 777-umask for directories and 666-umask for files, which does make a lot more sense I think, even if the variable UMASK is not set.
Also, fun fact I found, to get the system umask (via syscall) you must set the system umask to something to get the existing old umask. So in ddns-updater, it sets the system umask to 022 to get the system umask, and then restores the old system umask. All this in probably microseconds, but still strange you cannot get the system umask in a read-only fashion.
@favonia commented on GitHub (Sep 16, 2024):
Yes, see https://docs.podman.io/en/latest/markdown/podman-run.1.html#umask-umask. You can do
podman run --umask=.... Nothing stops Docker from adopting the same thing at any time---at least it's no longer a technical problem.It depends on whether those files should be readable and writable by others. For example,
sshthinks the answer is "absolutely no" for SSH keys and whatnot, which would match how ddns-updater previously treated its data.Anyway, it seems to me the reason @geekau needs
UMASKis because of the mismatched UID and GID? Now that we know--userwould work, maybe there's no need to set umask anymore?And even if we wish to take care of umask, umask will automatically affact
openandmkdir, so a simpler solution is to callsyscall.Umask(...)once when UMASK is not empty, which will take care of the calculation and avoid the hack to read the current umask.If we really want to emulate the umask by ourselves, I would do
0o666 & ^umaskinstead of0o666 - umask. The latter is wrong whenumask = 0o001, for example. (A umask0o001should not affect0o666.) The subtraction only works in special cases.The other issue is that, if we are emulating
umaskby ourselves without changing the system umask, then we are effectively using a umask that is the union of our own umask and the system umask. I'm not sure if that's the intended behavior.Yes, or you have to read some Linux-specific files under
/proc. The simpler solution (callingUmaskwhenUMASKis not empty) can avoid all the hacks.@qdm12 commented on GitHub (Sep 19, 2024):
@favonia Thank you for all these amazing information and fixes needed, they should all be bundled in
c16287e48aUMASKis setUMASKis setAgain, thank you 🎖️ !!
EDIT: Also I'll do a v2.8.1. release soon with this, since it seems a bit... urgent-ish.
Good question. Anyway, it doesn't hurt me to support it, especially since it should be compatible with Podman's
--umask+ emptyUMASKvariable, but yes it feels redundant a bit. @geekau ?@qdm12 commented on GitHub (Sep 19, 2024):
One more thing I'm wondering, for sensitive files such as the default empty json config file created, should the permissions be left to 666 (then umasked)? At least the user has the choice of changing this with umask, but I feel like defaults should really be 640 or 600. Thoughts on that? 🤔
@geekau commented on GitHub (Sep 20, 2024):
Hey Team,
So I've pulled down the latest image
Running version latest built on 2024-09-19T14:24:22.858Z (commit c16287e), removed all of the persistant files from the Gluetun and DDNS-Updater volume mounts, and redeployed DDNS-Updater with the following config (shortened version).When I deployed the DDNS-Updater container, it successfully used docker:users (PUID:PGID) and created the
config.jsonwith correct permissions / umask.I then updated the
config.jsonwith the running config, and it successfully created theupdates.jsonfile with correct permissions / umask.I'm happy with this advancement, as users can still apply their configurations in the
docker-compose.envfile, and we now have a way to make DDNS-Updater behave with the filesystem the same as the other apps.Gluetun still runs as root, and I understand why, but its only generating the
server.jsonasroot, and we don't want general users trying to edit this file, so I think its best we leave it as is.I had a look for the setfacl / getfacl, and they don't appear to be available in Synology OS (they're probably hidden somewhere), but I think this should be suitable for most user's needs. We can always make a mention of the Gluetun root requirement as a note to users.
Very much appreciate your assistance on this... as the work on the apps - TY.
@qdm12 commented on GitHub (Sep 25, 2024):
Thanks, everyone's happy to close this issue then?
@geekau commented on GitHub (Sep 25, 2024):
Thank you very much, closing now
Regards.