Bug: PUID, PGID, and UMASK Being Ignored in Docker Compose - Fails to Start - Unable To Read / Write to NAS Folders #436

Closed
opened 2025-11-20 04:24:01 -05:00 by saavagebueno · 16 comments
Owner

Originally created by @geekau on GitHub (Sep 14, 2024).

TLDR: PUID, PGID, and UMASK Being Ignored in Docker Compose - Fails to Start - Unable To Read / Write to NAS Folders

  1. Is this urgent: Yes

  2. DNS provider(s) you use: Cloudflare / DuckDNS

  3. Program version: 2024-08-28T14:16:17.081Z (commit bc272e0)

    Running version latest built on - 2024-08-28T14:16:17.081Z (commit bc272e0)

  4. What are you using to run the container: docker-compose with centralised environment file.

  5. Extra information

Our downstream project (https://github.com/geekau/mediastack) uses Gluetun and DDNS-Updater to manage some of the networking, we use a single docker-compose.env file for all of the variables across 25 Docker applications, to minimise complexity, and make it easier for new users to get into Docker, and deploy their MediaStack / SelfHost environment.

We have the following 3 variables which are shared across all Docker Compose YAML files, however DDNS-Updater does not seem to pick them up, and we can't read / write to some NAS, as the UID/GID is fixed at 1000:1000.

2024-09-14T18:47:05+10:00 ERROR setting initial up to date status: opening file: open /updater/data/updates.json: permission denied

PUID=1050
PGID=100
UMASK=0002

The DDNS-Updater is failing to start on some NAS, where the read / write permissions do not allow 1000:1000 access.

On our Synology servers, we needed to change our folder permissions to 777, before the container will start.

I've reviewed the DDNS-Updater documentation again and noticed reference to UID and GID, however these appear to only be for build-time configuration, which won't suit many new users.

Changing PUID / GUID / UMASK manually without varibles, still doesn't address file permission - they don't appear to be listed as valid environment variables in documentation.

Logs:

========================================
========================================
============= ddns-updater =============
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2024-08-28T14:16:17.081Z (commit bc272e0)

🔧 Need help? https://github.com/qdm12/ddns-updater/discussions/new
🐛 Bug? https://github.com/qdm12/ddns-updater/issues/new
✨ New feature? https://github.com/qdm12/ddns-updater/issues/new
☕ Discussion? https://github.com/qdm12/ddns-updater/discussions/new
💻 Email? quentin.mcgaw@gmail.com
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2024-09-14T18:47:05+10:00 INFO Settings summary:
├── HTTP client
|   └── Timeout: 10s
├── Update
|   ├── Period: 5m0s
|   └── Cooldown: 5m0s
├── Public IP fetching
|   ├── HTTP enabled: yes
|   ├── HTTP IP providers
|   |   └── all
|   ├── HTTP IPv4 providers
|   |   └── all
|   ├── HTTP IPv6 providers
|   |   └── all
|   ���── DNS enabled: yes
|   ├── DNS timeout: 3s
|   └── DNS over TLS providers
|       └── all
├── Resolver: use Go default resolver
├── Server
|   ├── Listening address: :8000
|   └── Root URL: /
├── Health
|   └── Server listening address: 127.0.0.1:9999
├── Paths
|   ├── Data directory: /updater/data
|   └── Config file: /updater/data/config.json
├── Backup: disabled
└── Logger
    ├── Level: info
    └── Caller: hidden
2024-09-14T18:47:05+10:00 INFO reading JSON config from file /updater/data/config.json
2024-09-14T18:47:05+10:00 INFO Found single setting to update record
2024-09-14T18:47:05+10:00 INFO Reading history from database: domain homelan.au owner @ ipv4
2024-09-14T18:47:05+10:00 INFO [healthcheck server] health http server listening on 127.0.0.1:9999
2024-09-14T18:47:05+10:00 INFO [http server] http server listening on [::]:8000
2024-09-14T18:47:05+10:00 INFO [backup] disabled
2024-09-14T18:47:05+10:00 ERROR setting initial up to date status: opening file: open /updater/data/updates.json: permission denied

Configuration file (remove your credentials!):

{
  "settings": [
    {
      "provider": "cloudflare",
      "zone_identifier": "REDACTED",
      "domain": "REDACTED",
      "ttl": 600,
      "token": "REDACTED",
      "ip_version": "ipv4",
      "ipv6_suffix": ""
    }
  ]
}

Host OS: Synology RS1221+ (DSM 7.2.2-72803)

Linux Synology 4.4.302+ #72803 SMP Mon Aug 19 20:01:58 CST 2024 x86_64 GNU/Linux synology_v1000_rs1221+
###########################################################################
###########################################################################
##
##    Docker Compose File: DDNS-Updater (qmcgaw)
##    Function: Update Dynamic IP addresses for DNS A and/or AAAA records
##
##    Documentation: https://hub.docker.com/r/qmcgaw/ddns-updater
##
###########################################################################
###########################################################################
services:
  ddns-updater:
    image: qmcgaw/ddns-updater:latest
    container_name: ddns-updater
    restart: always
    volumes:
      - ${FOLDER_FOR_DATA:?err}/ddns-updater:/updater/data
    ports:
      - ${WEBUI_PORT_DDNS_UPDATER:?err}:${WEBUI_PORT_DDNS_UPDATER:?err}/tcp
    environment:
      - PUID=${PUID:?err}
      - PGID=${PGID:?err}
      - UMASK=${UMASK:?err}
      - TZ=${TIMEZONE:?err}
      - CONFIG=
      - PERIOD=5m
      - UPDATE_COOLDOWN_PERIOD=5m
      - PUBLICIP_FETCHERS=all
      - PUBLICIP_HTTP_PROVIDERS=all
      - PUBLICIPV4_HTTP_PROVIDERS=all
      - PUBLICIPV6_HTTP_PROVIDERS=all
      - PUBLICIP_DNS_PROVIDERS=all
      - PUBLICIP_DNS_TIMEOUT=3s
      - HTTP_TIMEOUT=10s

      # Web UI
      - LISTENING_ADDRESS=:${WEBUI_PORT_DDNS_UPDATER:?err}
      - ROOT_URL=/

      # Backup
      - BACKUP_PERIOD=0 # 0 to disable
      - BACKUP_DIRECTORY=/updater/data

      # Other
      - LOG_LEVEL=info
      - LOG_CALLER=hidden
      - SHOUTRRR_ADDRESSES=

# NOTE: DDNS-Update MUST NOT connect to the Gluetun VPN container

networks:
  default:
    name: mediastack
    external: true
Originally created by @geekau on GitHub (Sep 14, 2024). <!-- YOU CAN CHAT THERE EVENTUALLY: https://github.com/qdm12/ddns-updater/discussions --> **TLDR**: *PUID, PGID, and UMASK Being Ignored in Docker Compose - Fails to Start - Unable To Read / Write to NAS Folders* 1. Is this urgent: Yes 2. DNS provider(s) you use: Cloudflare / DuckDNS 3. Program version: 2024-08-28T14:16:17.081Z (commit bc272e0) `Running version latest built on - 2024-08-28T14:16:17.081Z (commit bc272e0)` 4. What are you using to run the container: docker-compose with centralised environment file. 5. Extra information Our downstream project (https://github.com/geekau/mediastack) uses Gluetun and DDNS-Updater to manage some of the networking, we use a single **docker-compose.env** file for all of the variables across 25 Docker applications, to minimise complexity, and make it easier for new users to get into Docker, and deploy their MediaStack / SelfHost environment. We have the following 3 variables which are shared across all Docker Compose YAML files, however DDNS-Updater does not seem to pick them up, and we can't read / write to some NAS, as the UID/GID is fixed at 1000:1000. `2024-09-14T18:47:05+10:00 ERROR setting initial up to date status: opening file: open /updater/data/updates.json: permission denied` ```Environment Variables ( imported from docker-compose.env) PUID=1050 PGID=100 UMASK=0002 ``` The DDNS-Updater is failing to start on some NAS, where the read / write permissions do not allow 1000:1000 access. On our Synology servers, we needed to change our folder permissions to 777, before the container will start. I've reviewed the DDNS-Updater documentation again and noticed reference to UID and GID, however these appear to only be for build-time configuration, which won't suit many new users. Changing PUID / GUID / UMASK manually without varibles, still doesn't address file permission - they don't appear to be listed as valid environment variables in documentation. 1. Logs: ```log ======================================== ======================================== ============= ddns-updater ============= ======================================== =========== Made with ❤️ by ============ ======= https://github.com/qdm12 ======= ======================================== ======================================== Running version latest built on 2024-08-28T14:16:17.081Z (commit bc272e0) 🔧 Need help? https://github.com/qdm12/ddns-updater/discussions/new 🐛 Bug? https://github.com/qdm12/ddns-updater/issues/new ✨ New feature? https://github.com/qdm12/ddns-updater/issues/new ☕ Discussion? https://github.com/qdm12/ddns-updater/discussions/new 💻 Email? quentin.mcgaw@gmail.com 💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12 2024-09-14T18:47:05+10:00 INFO Settings summary: ├── HTTP client | └── Timeout: 10s ├── Update | ├── Period: 5m0s | └── Cooldown: 5m0s ├── Public IP fetching | ├── HTTP enabled: yes | ├── HTTP IP providers | | └── all | ├── HTTP IPv4 providers | | └── all | ├── HTTP IPv6 providers | | └── all | ���── DNS enabled: yes | ├── DNS timeout: 3s | └── DNS over TLS providers | └── all ├── Resolver: use Go default resolver ├── Server | ├── Listening address: :8000 | └── Root URL: / ├── Health | └── Server listening address: 127.0.0.1:9999 ├── Paths | ├── Data directory: /updater/data | └── Config file: /updater/data/config.json ├── Backup: disabled └── Logger ├── Level: info └── Caller: hidden 2024-09-14T18:47:05+10:00 INFO reading JSON config from file /updater/data/config.json 2024-09-14T18:47:05+10:00 INFO Found single setting to update record 2024-09-14T18:47:05+10:00 INFO Reading history from database: domain homelan.au owner @ ipv4 2024-09-14T18:47:05+10:00 INFO [healthcheck server] health http server listening on 127.0.0.1:9999 2024-09-14T18:47:05+10:00 INFO [http server] http server listening on [::]:8000 2024-09-14T18:47:05+10:00 INFO [backup] disabled 2024-09-14T18:47:05+10:00 ERROR setting initial up to date status: opening file: open /updater/data/updates.json: permission denied ``` Configuration file (**remove your credentials!**): ```json { "settings": [ { "provider": "cloudflare", "zone_identifier": "REDACTED", "domain": "REDACTED", "ttl": 600, "token": "REDACTED", "ip_version": "ipv4", "ipv6_suffix": "" } ] } ``` Host OS: Synology RS1221+ (DSM 7.2.2-72803) ``` Linux Synology 4.4.302+ #72803 SMP Mon Aug 19 20:01:58 CST 2024 x86_64 GNU/Linux synology_v1000_rs1221+ ``` ```docker-compose-ddns-updater.yaml ########################################################################### ########################################################################### ## ## Docker Compose File: DDNS-Updater (qmcgaw) ## Function: Update Dynamic IP addresses for DNS A and/or AAAA records ## ## Documentation: https://hub.docker.com/r/qmcgaw/ddns-updater ## ########################################################################### ########################################################################### services: ddns-updater: image: qmcgaw/ddns-updater:latest container_name: ddns-updater restart: always volumes: - ${FOLDER_FOR_DATA:?err}/ddns-updater:/updater/data ports: - ${WEBUI_PORT_DDNS_UPDATER:?err}:${WEBUI_PORT_DDNS_UPDATER:?err}/tcp environment: - PUID=${PUID:?err} - PGID=${PGID:?err} - UMASK=${UMASK:?err} - TZ=${TIMEZONE:?err} - CONFIG= - PERIOD=5m - UPDATE_COOLDOWN_PERIOD=5m - PUBLICIP_FETCHERS=all - PUBLICIP_HTTP_PROVIDERS=all - PUBLICIPV4_HTTP_PROVIDERS=all - PUBLICIPV6_HTTP_PROVIDERS=all - PUBLICIP_DNS_PROVIDERS=all - PUBLICIP_DNS_TIMEOUT=3s - HTTP_TIMEOUT=10s # Web UI - LISTENING_ADDRESS=:${WEBUI_PORT_DDNS_UPDATER:?err} - ROOT_URL=/ # Backup - BACKUP_PERIOD=0 # 0 to disable - BACKUP_DIRECTORY=/updater/data # Other - LOG_LEVEL=info - LOG_CALLER=hidden - SHOUTRRR_ADDRESSES= # NOTE: DDNS-Update MUST NOT connect to the Gluetun VPN container networks: default: name: mediastack external: true ```
Author
Owner

@qdm12 commented on GitHub (Sep 14, 2024):

The docker image has a default 'hardcoded' user and group (see USER instruction in Dockerfile) and runs as that user from start, so it cannot change user/group at runtime. This is for security reasons and also since the docker image is based on scratch (no OS).
What you could do however is set the user field in docker-compose.yml to specify the user id to use to run the container. For example user: "1001" and you should be able to use a variable such as user: "{PUID}", does that solve your issue?

@qdm12 commented on GitHub (Sep 14, 2024): The docker image has a default 'hardcoded' user and group (see `USER` instruction in Dockerfile) and runs as that user from start, so it cannot change user/group at runtime. This is for security reasons and also since the docker image is based on `scratch` (no OS). What you could do however is set the `user` field in docker-compose.yml to specify the user id to use to run the container. For example `user: "1001"` and you should be able to use a variable such as `user: "{PUID}"`, does that solve your issue?
Author
Owner

@qdm12 commented on GitHub (Sep 14, 2024):

See https://docs.docker.com/reference/compose-file/services/#user

@qdm12 commented on GitHub (Sep 14, 2024): See https://docs.docker.com/reference/compose-file/services/#user
Author
Owner

@favonia commented on GitHub (Sep 14, 2024):

@qdm12 Sorry for my uninvited comment, but I think you meant user: "${PUID}" (with a $), and for this case, maybe @geekau wanted user: "${PUID}:${PGID}". Also, you can use docker run --user ... with the same effects of setting user: ... in the compose file. Maybe I misunderstood what you meant by "cannot change user/group at runtime".


My 2 cents: PUID, PGID and UMASK are only a convention made popular by e.g. images produced by LinuxServer.io. They are just regular environment variables read by certain images to set user ID, group ID, or umask by themselves during run time. These variables have no special meanings and the Docker engine provides no special support for them. Even people at LinuxServer.io say the following in their documentation:

We are aware that recent versions of the Docker engine have introduced the --user flag. Our images are not yet compatible with this, so we recommend continuing usage of PUID and PGID.

In my opinion, unless we are talking about the Docker images that explicitly want you to set PUID or PGID, it's safer and more reliable for the Docker engine (instead of individual images/containers) to set the user/group IDs, as recommended by @qdm12. Once https://github.com/moby/moby/issues/19189 is resolved (if ever), we can also set umask directly via the Docker engine. I am hoping that we can move on from the legacy workaround one day.

@favonia commented on GitHub (Sep 14, 2024): @qdm12 Sorry for my uninvited comment, but I think you meant `user: "${PUID}"` (with a `$`), and for this case, maybe @geekau wanted `user: "${PUID}:${PGID}"`. Also, you can use `docker run --user ...` with the same effects of setting `user: ...` in the compose file. Maybe I misunderstood what you meant by "cannot change user/group at runtime". --- My 2 cents: `PUID`, `PGID` and `UMASK` are only a convention made popular by e.g. images produced by [LinuxServer.io](https://www.linuxserver.io/). They are just regular environment variables read by certain images to set user ID, group ID, or umask by themselves during run time. These variables have no special meanings and the Docker engine provides no special support for them. Even people at LinuxServer.io say the following in [their documentation](https://docs.linuxserver.io/general/understanding-puid-and-pgid/): > We are aware that recent versions of the Docker engine have introduced the `--user` flag. Our images are not yet compatible with this, so we recommend continuing usage of `PUID` and `PGID`. In my opinion, unless we are talking about the Docker images that explicitly want you to set `PUID` or `PGID`, it's safer and more reliable for the Docker engine (instead of individual images/containers) to set the user/group IDs, as recommended by @qdm12. Once https://github.com/moby/moby/issues/19189 is resolved (if ever), we can also set umask directly via the Docker engine. I am hoping that we can move on from the legacy workaround one day.
Author
Owner

@qdm12 commented on GitHub (Sep 14, 2024):

@favonia exactly on point, thank you for correcting my comment 👍

@qdm12 commented on GitHub (Sep 14, 2024): @favonia exactly on point, thank you for correcting my comment 👍
Author
Owner

@geekau commented on GitHub (Sep 14, 2024):

@qdm12 Sorry for my uninvited comment, but I think you meant user: "${PUID}" (with a $), and for this case, maybe @geekau wanted user: "${PUID}:${PGID}". Also, you can use docker run --user ... with the same effects of setting user: ... in the compose file. Maybe I misunderstood what you meant by "cannot change user/group at runtime".

My 2 cents: PUID, PGID and UMASK are only a convention made popular by e.g. images produced by LinuxServer.io. They are just regular environment variables read by certain images to set user ID, group ID, or umask by themselves during run time. These variables have no special meanings and the Docker engine provides no special support for them. Even people at LinuxServer.io say the following in their documentation:

We are aware that recent versions of the Docker engine have introduced the --user flag. Our images are not yet compatible with this, so we recommend continuing usage of PUID and PGID.

In my opinion, unless we are talking about the Docker images that explicitly want you to set PUID or PGID, it's safer and more reliable for the Docker engine (instead of individual images/containers) to set the user/group IDs, as recommended by @qdm12. Once moby/moby#19189 is resolved (if ever), we can also set umask directly via the Docker engine. I am hoping that we can move on from the legacy workaround one day.

Thank you so much @qdm12 and @favonia I was incorrect in my thinking that PUID, PGID and UMASK were Docker-wide variables, but do seem to be isolated to the LinuxServer and Hotio image fleet. However, the compose-file link and recommendation of user: have provided some progress.

I've updated the docker-compose-ddns-updater.yaml file to:

services:
  ddns-updater:
    image: qmcgaw/ddns-updater:latest
    container_name: ddns-updater
    restart: always
    user: ${PUID:?err}:${PGID:?err}

This now allows the DDNS-Updater running container to read / write to the local Docker host filesystem as the PUID:PGID values specified in the docker-compose.env file, which is really what we're after - we don't want to change uid/gid internally of any docker images, just the values so they can read / write to the local filesystem, so user: works well here.

I also tested user: ${PUID:?err}:${PGID:?err} in the Gluetun docker-compose, but it failed as Gluetun needs to run as root - And that's OK, I understand why Gluetun needs to run as root, its more mapping the PUID, PGID and UMASK values to the local Docker host filesystem, so the logging / backups can interact with the local host with relevant permissions.

I initially marked this as urgent, as I thought there would be a lot of NAS users not being able to map running users / processes to the local filesystem, however we can drop urgency on this now.

So outstanding questions I have:

  1. Am I correct in understanding user: ${PUID:?err}:${PGID:?err} allows the DDNS-Updater container to "run-as" these IDs, which means the PUID:PGID will be the same inside the container are inherited the same as the local docker host filesystem?

  2. Using user: ${PUID:?err}:${PGID:?err} inside the Gluetun container fails to run, as Gluetun needs to run as root user, therefore Gluetun is not able to read / write as a different PUID:PGID mapped from the local Docker host computer?

  3. Is it possible (as a feature request), that Gluetun and DDNS-Updater are able to be updated, so the containers still run internally as their intended users (root / 1000), but there is a mapping of PUID, PGID and UMASK to the local Docker host values?

Please correct me if I've misunderstood some of Docker's technical capabilities / implementation, you're work on these images are amazing - thank you.

Regards.

@geekau commented on GitHub (Sep 14, 2024): > @qdm12 Sorry for my uninvited comment, but I think you meant `user: "${PUID}"` (with a `$`), and for this case, maybe @geekau wanted `user: "${PUID}:${PGID}"`. Also, you can use `docker run --user ...` with the same effects of setting `user: ...` in the compose file. Maybe I misunderstood what you meant by "cannot change user/group at runtime". > > My 2 cents: `PUID`, `PGID` and `UMASK` are only a convention made popular by e.g. images produced by [LinuxServer.io](https://www.linuxserver.io/). They are just regular environment variables read by certain images to set user ID, group ID, or umask by themselves during run time. These variables have no special meanings and the Docker engine provides no special support for them. Even people at LinuxServer.io say the following in [their documentation](https://docs.linuxserver.io/general/understanding-puid-and-pgid/): > > > We are aware that recent versions of the Docker engine have introduced the `--user` flag. Our images are not yet compatible with this, so we recommend continuing usage of `PUID` and `PGID`. > > In my opinion, unless we are talking about the Docker images that explicitly want you to set `PUID` or `PGID`, it's safer and more reliable for the Docker engine (instead of individual images/containers) to set the user/group IDs, as recommended by @qdm12. Once [moby/moby#19189](https://github.com/moby/moby/issues/19189) is resolved (if ever), we can also set umask directly via the Docker engine. I am hoping that we can move on from the legacy workaround one day. Thank you so much @qdm12 and @favonia I was incorrect in my thinking that `PUID`, `PGID` and `UMASK` were Docker-wide variables, but do seem to be isolated to the LinuxServer and Hotio image fleet. However, the compose-file link and recommendation of `user:` have provided some progress. I've updated the *docker-compose-ddns-updater.yaml* file to: ``` services: ddns-updater: image: qmcgaw/ddns-updater:latest container_name: ddns-updater restart: always user: ${PUID:?err}:${PGID:?err} ``` This now allows the DDNS-Updater running container to read / write to the local Docker host filesystem as the PUID:PGID values specified in the *docker-compose.env* file, which is really what we're after - we don't want to change uid/gid internally of any docker images, just the values so they can read / write to the local filesystem, so `user:` works well here. I also tested `user: ${PUID:?err}:${PGID:?err}` in the Gluetun docker-compose, but it failed as Gluetun needs to run as `root` - And that's OK, I understand why Gluetun needs to run as `root`, its more mapping the `PUID`, `PGID` and `UMASK` values to the local Docker host filesystem, so the logging / backups can interact with the local host with relevant permissions. I initially marked this as urgent, as I thought there would be a lot of NAS users not being able to map running users / processes to the local filesystem, however we can drop urgency on this now. So outstanding questions I have: 1. Am I correct in understanding `user: ${PUID:?err}:${PGID:?err}` allows the DDNS-Updater container to "run-as" these IDs, which means the PUID:PGID will be the same inside the container are inherited the same as the local docker host filesystem? 2. Using `user: ${PUID:?err}:${PGID:?err}` inside the Gluetun container fails to run, as Gluetun needs to run as `root` user, therefore Gluetun is not able to read / write as a different PUID:PGID mapped from the local Docker host computer? 3. Is it possible (as a feature request), that Gluetun and DDNS-Updater are able to be updated, so the containers still run internally as their intended users (root / 1000), but there is a mapping of `PUID`, `PGID` and `UMASK` to the local Docker host values? Please correct me if I've misunderstood some of Docker's technical capabilities / implementation, you're work on these images are amazing - thank you. Regards.
Author
Owner

@qdm12 commented on GitHub (Sep 15, 2024):

  1. It depends on PUID And PGID variables you inject to the user configuration field; if they're set to the current user id and group id, then yes.
  2. Yes gluetun needs root access for various reasons (firewall, routing, bind to privileged port 53 etc.). I think you can set PUID and PGID in order for gluetun to write certain files with the ownership specified by those, but it won't run as PUID/PGID. It was also possible to drop root when running openvpn, but that produced a lot of additional problems and doesn't really help security wise I would say, so that got removed.
  3. I'm not sure what you mean, can you re-clarify (limiting to ddns-updater since gluetun cannot be changed really)
    Thanks!
@qdm12 commented on GitHub (Sep 15, 2024): 1. It depends on PUID And PGID variables you inject to the user configuration field; if they're set to the current user id and group id, then yes. 2. Yes gluetun needs root access for various reasons (firewall, routing, bind to privileged port 53 etc.). I think you can set PUID and PGID in order for gluetun to write certain files with the ownership specified by those, but it won't run as PUID/PGID. It was also possible to drop root when running openvpn, but that produced a lot of additional problems and doesn't really help security wise I would say, so that got removed. 3. I'm not sure what you mean, can you re-clarify (limiting to ddns-updater since gluetun cannot be changed really) Thanks!
Author
Owner

@geekau commented on GitHub (Sep 15, 2024):

Ref 3... Am referring generally if the Gluetun / DDNS-Updater containers can still run as they've been designed, but allow the Docker processes for these containers to interact with the local Docker host, with different PUID, PGID, and UMASK values.

In our downstreat project, we use the PUID, PGID, and UMASK values in an ENV file, which are then used by each of the containers, when Docker Compose deploys each of the YAML configurations - these are mostly LinuxServer and Hotio images.

Then the containers still deploy / run with the build in user values, however they interact with the local filesystem with the values from the Docker Compose ENV file. This allows the container to read / write with a local user account the user may choose from... a bit like a port mapping, but file permissions - excuse the analogy.

Our project currently has 25 Docker containers, and our vision is to make it very easy for new users to be able to deploy Docker images with very little (to no), prior knowledge - hence we put all of the varibles into the one ENV file, which all the containers use when they deploy the Docker Compose YAML files.

We use the PUID, PGID, and UMASK values in all the other containers, the then interact with the local Docker filesystem with a single dedicated account.

Just curious whether this is technically feasible for Gluetun / DDNS-Updater?

Hope this explains it better.

@geekau commented on GitHub (Sep 15, 2024): Ref 3... Am referring generally if the Gluetun / DDNS-Updater containers can still run as they've been designed, but allow the Docker processes for these containers to interact with the local Docker host, with different `PUID`, `PGID`, and `UMASK` values. In our downstreat project, we use the `PUID`, `PGID`, and `UMASK` values in an ENV file, which are then used by each of the containers, when Docker Compose deploys each of the YAML configurations - these are mostly LinuxServer and Hotio images. Then the containers still deploy / run with the build in user values, however they interact with the local filesystem with the values from the Docker Compose ENV file. This allows the container to read / write with a local user account the user may choose from... a bit like a port mapping, but file permissions - excuse the analogy. Our project currently has 25 Docker containers, and our vision is to make it very easy for new users to be able to deploy Docker images with very little (to no), prior knowledge - hence we put all of the varibles into the one ENV file, which all the containers use when they deploy the Docker Compose YAML files. We use the `PUID`, `PGID`, and `UMASK` values in all the other containers, the then interact with the local Docker filesystem with a single dedicated account. Just curious whether this is technically feasible for Gluetun / DDNS-Updater? Hope this explains it better.
Author
Owner

@qdm12 commented on GitHub (Sep 16, 2024):

UMASK is added in 3a6262ef2c and should be effective from here. Although that will obviously depend on the user+group configured to run ddns-updater.
Now PUID and PGID cannot be used for ddns-updater, since the user configured to run ddns-updater (even before the program starts) cannot be changed at runtime (unless it's root, but would there be a use case for this??)... this is as far as I know, maybe I'm missing something. I just learnt all about umask to be honest, so I may be lacking other knowledge!

@qdm12 commented on GitHub (Sep 16, 2024): UMASK is added in 3a6262ef2ce3cf7f77aded5a2102de61db309cd4 and should be effective from here. Although that will obviously depend on the user+group configured to run ddns-updater. Now PUID and PGID cannot be used for ddns-updater, since the user configured to run ddns-updater (even before the program starts) cannot be changed at runtime (unless it's root, but would there be a use case for this??)... this is as far as I know, maybe I'm missing something. I just learnt all about umask to be honest, so I may be lacking other knowledge!
Author
Owner

@favonia commented on GitHub (Sep 16, 2024):

My 2.5 cents: personally I'm against using environment variables along the line of PUID and PGID because they should have been part of Docker and it could be difficult for this project to drop it when Docker (finally) supports that, but this is my project... 🤷 @geekau A possible workaround or alternative approach might be using setfacl to set default permissions for the directory, but I didn't test it carefully.

PS: podman already supports umask, so there's hope that Docker will follow, eventually.

@favonia commented on GitHub (Sep 16, 2024): My 2.5 cents: personally I'm against using environment variables along the line of `PUID` and `PGID` because they should have been part of Docker and it could be difficult for this project to drop it when Docker (finally) supports that, but this is my project... :shrug: @geekau A possible workaround or alternative approach might be using `setfacl` to set default permissions for the directory, but I didn't test it carefully. PS: podman already supports umask, so there's hope that Docker will follow, eventually.
Author
Owner

@qdm12 commented on GitHub (Sep 16, 2024):

PS: podman already supports umask, so there's hope that Docker will follow, eventually.

So that means you could set the system default umask right?

For ddns-updater previous to the UMASK commit I mentioned, it would arbitrarily set permissions on directories (700) and files (600) without taking care of umask. Now it's quite better, since it uses the OS default umask if UMASK is left empty, and uses 777-umask for directories and 666-umask for files, which does make a lot more sense I think, even if the variable UMASK is not set.

Also, fun fact I found, to get the system umask (via syscall) you must set the system umask to something to get the existing old umask. So in ddns-updater, it sets the system umask to 022 to get the system umask, and then restores the old system umask. All this in probably microseconds, but still strange you cannot get the system umask in a read-only fashion.

@qdm12 commented on GitHub (Sep 16, 2024): > PS: podman already supports umask, so there's hope that Docker will follow, eventually. So that means you could set the system default umask right? For ddns-updater previous to the UMASK commit I mentioned, it would arbitrarily set permissions on directories (700) and files (600) without taking care of umask. Now it's quite better, since it uses the OS default umask if UMASK is left empty, and uses 777-umask for directories and 666-umask for files, which does make a lot more sense I think, even if the variable UMASK is not set. Also, fun fact I found, to get the system umask (via syscall) you must set the system umask to something to get the existing old umask. So in ddns-updater, it sets the system umask to 022 to get the system umask, and then restores the old system umask. All this in probably microseconds, but still strange you cannot get the system umask in a read-only fashion.
Author
Owner

@favonia commented on GitHub (Sep 16, 2024):

PS: podman already supports umask, so there's hope that Docker will follow, eventually.

So that means you could set the system default umask right?

Yes, see https://docs.podman.io/en/latest/markdown/podman-run.1.html#umask-umask. You can do podman run --umask=.... Nothing stops Docker from adopting the same thing at any time---at least it's no longer a technical problem.

For ddns-updater previous to the UMASK commit I mentioned, it would arbitrarily set permissions on directories (700) and files (600) without taking care of umask. Now it's quite better, since it uses the OS default umask if UMASK is left empty, and uses 777-umask for directories and 666-umask for files, which does make a lot more sense I think, even if the variable UMASK is not set.

It depends on whether those files should be readable and writable by others. For example, ssh thinks the answer is "absolutely no" for SSH keys and whatnot, which would match how ddns-updater previously treated its data.

Anyway, it seems to me the reason @geekau needs UMASK is because of the mismatched UID and GID? Now that we know --user would work, maybe there's no need to set umask anymore?

And even if we wish to take care of umask, umask will automatically affact open and mkdir, so a simpler solution is to call syscall.Umask(...) once when UMASK is not empty, which will take care of the calculation and avoid the hack to read the current umask.

If we really want to emulate the umask by ourselves, I would do 0o666 & ^umask instead of 0o666 - umask. The latter is wrong when umask = 0o001, for example. (A umask 0o001 should not affect 0o666.) The subtraction only works in special cases.

The other issue is that, if we are emulating umask by ourselves without changing the system umask, then we are effectively using a umask that is the union of our own umask and the system umask. I'm not sure if that's the intended behavior.

Also, fun fact I found, to get the system umask (via syscall) you must set the system umask to something to get the existing old umask. So in ddns-updater, it sets the system umask to 022 to get the system umask, and then restores the old system umask. All this in probably microseconds, but still strange you cannot get the system umask in a read-only fashion.

Yes, or you have to read some Linux-specific files under /proc. The simpler solution (calling Umask when UMASK is not empty) can avoid all the hacks.

@favonia commented on GitHub (Sep 16, 2024): > > PS: podman already supports umask, so there's hope that Docker will follow, eventually. > > So that means you could set the system default umask right? Yes, see https://docs.podman.io/en/latest/markdown/podman-run.1.html#umask-umask. You can do `podman run --umask=...`. Nothing stops Docker from adopting the same thing at any time---at least it's no longer a technical problem. > For ddns-updater previous to the UMASK commit I mentioned, it would arbitrarily set permissions on directories (700) and files (600) without taking care of umask. Now it's quite better, since it uses the OS default umask if UMASK is left empty, and uses 777-umask for directories and 666-umask for files, which does make a lot more sense I think, even if the variable UMASK is not set. It depends on whether those files should be readable and writable by others. For example, `ssh` thinks the answer is "absolutely no" for SSH keys and whatnot, which would match how ddns-updater previously treated its data. Anyway, it seems to me the reason @geekau needs `UMASK` is because of the mismatched UID and GID? Now that we know `--user` would work, maybe there's no need to set umask anymore? And even if we wish to take care of umask, umask will automatically affact `open` and `mkdir`, so a simpler solution is to call `syscall.Umask(...)` once when UMASK is not empty, which will take care of the calculation and avoid the hack to read the current umask. If we really want to emulate the umask by ourselves, I would do `0o666 & ^umask` instead of `0o666 - umask`. The latter is wrong when `umask = 0o001`, for example. (A umask `0o001` should not affect `0o666`.) The subtraction only works in special cases. The other issue is that, if we are emulating `umask` by ourselves without changing the system umask, then we are effectively using a umask that is the _union_ of our own umask and the system umask. I'm not sure if that's the intended behavior. > Also, fun fact I found, to get the system umask (via syscall) you must set the system umask to something to get the existing old umask. So in ddns-updater, it sets the system umask to 022 to get the system umask, and then restores the old system umask. All this in probably microseconds, but still strange you cannot get the system umask in a read-only fashion. Yes, or you have to read some Linux-specific files under `/proc`. The simpler solution (calling `Umask` when `UMASK` is not empty) can avoid all the hacks.
Author
Owner

@qdm12 commented on GitHub (Sep 19, 2024):

@favonia Thank you for all these amazing information and fixes needed, they should all be bundled in c16287e48a

  • remove bad calculation of "our own" umask
  • remove umask "union effect"
  • do not touch system umask unless UMASK is set
  • set system umask only if UMASK is set

Again, thank you 🎖️ !!

EDIT: Also I'll do a v2.8.1. release soon with this, since it seems a bit... urgent-ish.

Anyway, it seems to me the reason @geekau needs UMASK is because of the mismatched UID and GID? Now that we know --user would work, maybe there's no need to set umask anymore?

Good question. Anyway, it doesn't hurt me to support it, especially since it should be compatible with Podman's --umask + empty UMASK variable, but yes it feels redundant a bit. @geekau ?

@qdm12 commented on GitHub (Sep 19, 2024): @favonia Thank you for all these amazing information and fixes needed, they should all be bundled in c16287e48a6080e3693f5373976020df6a116365 - remove bad calculation of "our own" umask - remove umask "union effect" - do not touch system umask unless `UMASK` is set - set system umask only if `UMASK` is set Again, thank you 🎖️ !! EDIT: Also I'll do a v2.8.1. release soon with this, since it seems a bit... urgent-ish. > Anyway, it seems to me the reason @geekau needs UMASK is because of the mismatched UID and GID? Now that we know --user would work, maybe there's no need to set umask anymore? Good question. Anyway, it doesn't hurt me to support it, especially since it *should* be compatible with Podman's `--umask` + empty `UMASK` variable, but yes it feels redundant a bit. @geekau ?
Author
Owner

@qdm12 commented on GitHub (Sep 19, 2024):

One more thing I'm wondering, for sensitive files such as the default empty json config file created, should the permissions be left to 666 (then umasked)? At least the user has the choice of changing this with umask, but I feel like defaults should really be 640 or 600. Thoughts on that? 🤔

@qdm12 commented on GitHub (Sep 19, 2024): One more thing I'm wondering, for sensitive files such as the default empty json config file created, should the permissions be left to 666 (then umasked)? At least the user has the choice of changing this with umask, but I feel like defaults should really be 640 or 600. Thoughts on that? 🤔
Author
Owner

@geekau commented on GitHub (Sep 20, 2024):

Hey Team,

So I've pulled down the latest image Running version latest built on 2024-09-19T14:24:22.858Z (commit c16287e), removed all of the persistant files from the Gluetun and DDNS-Updater volume mounts, and redeployed DDNS-Updater with the following config (shortened version).

services:
  ddns-updater:
    image: qmcgaw/ddns-updater:latest
    container_name: ddns-updater
    restart: always
    user: ${PUID:?err}:${PGID:?err}
  environment:
    - UMASK=${UMASK:?err}

When I deployed the DDNS-Updater container, it successfully used docker:users (PUID:PGID) and created the config.json with correct permissions / umask.

I then updated the config.json with the running config, and it successfully created the updates.json file with correct permissions / umask.

Screenshot 2024-09-20 170432

I'm happy with this advancement, as users can still apply their configurations in the docker-compose.env file, and we now have a way to make DDNS-Updater behave with the filesystem the same as the other apps.

Gluetun still runs as root, and I understand why, but its only generating the server.json as root, and we don't want general users trying to edit this file, so I think its best we leave it as is.

I had a look for the setfacl / getfacl, and they don't appear to be available in Synology OS (they're probably hidden somewhere), but I think this should be suitable for most user's needs. We can always make a mention of the Gluetun root requirement as a note to users.

Very much appreciate your assistance on this... as the work on the apps - TY.

@geekau commented on GitHub (Sep 20, 2024): Hey Team, So I've pulled down the latest image `Running version latest built on 2024-09-19T14:24:22.858Z (commit c16287e)`, removed all of the persistant files from the Gluetun and DDNS-Updater volume mounts, and redeployed DDNS-Updater with the following config (shortened version). ``` services: ddns-updater: image: qmcgaw/ddns-updater:latest container_name: ddns-updater restart: always user: ${PUID:?err}:${PGID:?err} environment: - UMASK=${UMASK:?err} ``` When I deployed the DDNS-Updater container, it successfully used docker:users (PUID:PGID) and created the `config.json` with correct permissions / umask. I then updated the `config.json` with the running config, and it successfully created the `updates.json` file with correct permissions / umask. ![Screenshot 2024-09-20 170432](https://github.com/user-attachments/assets/d05c318c-4084-4c08-90cf-6731008ed91b) I'm happy with this advancement, as users can still apply their configurations in the `docker-compose.env` file, and we now have a way to make DDNS-Updater behave with the filesystem the same as the other apps. Gluetun still runs as root, and I understand why, but its only generating the `server.json` as `root`, and we don't want general users trying to edit this file, so I think its best we leave it as is. I had a look for the setfacl / getfacl, and they don't appear to be available in Synology OS (they're probably hidden somewhere), but I think this should be suitable for most user's needs. We can always make a mention of the Gluetun root requirement as a note to users. Very much appreciate your assistance on this... as the work on the apps - TY.
Author
Owner

@qdm12 commented on GitHub (Sep 25, 2024):

Thanks, everyone's happy to close this issue then?

@qdm12 commented on GitHub (Sep 25, 2024): Thanks, everyone's happy to close this issue then?
Author
Owner

@geekau commented on GitHub (Sep 25, 2024):

Thank you very much, closing now

Regards.

@geekau commented on GitHub (Sep 25, 2024): Thank you very much, closing now Regards.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/ddns-updater#436