"smtp.gmail.com: i/o timeout" but can ping, nslookup and nc to the server just fine from the container

[bverkron]

What are you trying to do?
Send SMTP notifications. They were working before with this exact config but no longer work after rebuilding the host.

What is your current configuration?

  backup:
    # https://github.com/offen/docker-volume-backup#readme
    image: offen/docker-volume-backup:latest
    container_name: $APP_NAME-backup
    depends_on:
      - $APP_NAME
    environment:
      BACKUP_CRON_EXPRESSION: $BACKUP_CRON_EXPRESSION
      BACKUP_RETENTION_DAYS: $BACKUP_RETENTION_DAYS
      BACKUP_EXCLUDE_REGEXP: $BACKUP_EXCLUDE_REGEXP
      BACKUP_FILENAME: $APP_NAME-backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_PRUNING_PREFIX: $APP_NAME-backup-
      NOTIFICATION_URLS: $NOTIFICATION_URLS
      NOTIFICATION_LEVEL: info
      GPG_PASSPHRASE: $GPG_PASSPHRASE
      AWS_S3_BUCKET_NAME: $AWS_S3_BUCKET_NAME
      AWS_S3_PATH: $APP_NAME
      AWS_ENDPOINT: $AWS_ENDPOINT
      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
    volumes:
      # Standard config    
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - /media/ext_storage/backups/$APP_NAME/:/archive/
      - $APP_DATA_PATH/:/backup/:ro
    restart: unless-stopped

Log output

time=2025-10-11T12:59:16.327-07:00 level=ERROR msg="Fatal error running command: main.(*script).runHooks: error running hook: main.(*script).notify: error sending notification: main.(*script).sendNotification: error sending message: error getting SMTP client: error connecting to server: dial tcp: lookup smtp.gmail.com: i/o timeout" error="main.(*command).runAsCommand: error running script: main.runScript.func4: the script ran successfully, but an error occurred calling the registered hooks: main.(*script).runHooks: error running hook: main.(*script).notify: error sending notification: main.(*script).sendNotification: error sending message: error getting SMTP client: error connecting to server: dial tcp: lookup smtp.gmail.com: i/o timeout"

~ # nslookup smtp.gmail.com
Server:         127.0.0.11
Address:        127.0.0.11:53

Non-authoritative answer:
Name:   smtp.gmail.com
Address: 2607:f8b0:400e:c09::6d

Non-authoritative answer:
Name:   smtp.gmail.com
Address: 192.178.163.108

~ # ping smtp.gmail.com
PING smtp.gmail.com (74.125.195.109): 56 data bytes
64 bytes from 74.125.195.109: seq=0 ttl=108 time=17.403 ms
64 bytes from 74.125.195.109: seq=1 ttl=108 time=17.345 ms
64 bytes from 74.125.195.109: seq=2 ttl=108 time=16.507 ms

--- smtp.gmail.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 16.507/17.085/17.403 ms

~ # nc -vz smtp.gmail.com 587
smtp.gmail.com (74.125.195.109:587) open

Additional context
I recently had to rebuild my docker host and went from Ubuntu to Debian. This most likely has something to do with the new host but I cannot figure out what.

DNS works on the new host and in every other container aside from offen backup. My manual tests above show that the offen backup container resolves the Gmail SMTP server and can connect but for some reason offen backup itself cannot? I have another host in the same VLAN that is still working as expected so it's not a broader network issue either.

I'm using the identical compose file on this new host as I as before and it's identical to a compose file on another host on the same VLAN that also works fine and gets the same results from all the checks above (nslookup, ping, nc, etc)

A bit stumped on where to go from here. Are there some go or offen specific commands I can run to debug?

[m90]

Did you upgrade to https://github.com/offen/docker-volume-backup/releases/tag/v2.44.1 by any chance while setting up your new host? The release swaps out the dependency used to send mails, and while email still works for me using the newer version, it would at least be some explanation for what you are seeing. If you did update, could you try downgrading to v2.44.0 and see if it makes a difference?

Other than that I don't really have any idea for further debugging unfortunately.

[PythonGermany]

Looks to me like this issue might be related to #304 and should be fixed once nicholas-fedor/shoutrrr#370 makes it into this project. Until then try adding &timeout=10s to your NOTIFICATION_URLS env variable to overwrite the faulty default timeout.

[m90]

@PythonGermany sweet, I'll update and cut a new version once your fix is released.

[bverkron]

Did you upgrade to https://github.com/offen/docker-volume-backup/releases/tag/v2.44.1 by any chance while setting up your new host? ... could you try downgrading to v2.44.0 and see if it makes a difference?

I was using latest so it looks like it probably pulled v2.44.1 when I setup the new host. I thought my other host was on the same latest version but looks like it was not, hence the discrepancy in behaviour between the two. I forced the v2.44.0 tag and it works as expected so that confirms it.

Adding &timeout=10s (though I went with 5s) to the NOTIFICATION_URLS env var as suggested works, so will go with that for now since it allows me to stay on the latest version and work around the but. TBH having that setting long term isn't terrible either as it adds some resiliency in case there are communication delays with the SMTP server for some reason.

Thanks all!

[nicholas-fedor]

@m90 Shoutrrr v0.10.3 has been released with @PythonGermany's fix (nicholas-fedor/shoutrrr#370).

The release notes can be viewed here: https://github.com/nicholas-fedor/shoutrrr/releases/tag/v0.10.3

[m90]

v2.45.0 now has an updated shoutrrr version that includes nicholas-fedor/shoutrrr#370