Docker swarm error

[zibellon]

Describe the bug
OffenBackup - search labels only on One node. For example Traefik (in swarm mode) search container (deploy) labels on all nodes

To Reproduce
Steps to reproduce the behavior:

1. I have docker swarm cluster: 1 master (MASTER_1) and 2 workers (WORKER_1, WORKER_2).
2. Run 3 different Postgres docker on all nodes (For three different projects).
3. Want to backup all of them.
4. For all Postgres docker add volume: - pg-backup-data:/tmp-backup
5. For all Postgres docker add two labels (In service section, not in deploy section):

• "docker-volume-backup.archive-pre=/bin/sh -c 'pg_dump -U postgres postgres | gzip > /tmp-backup/pg_dump.sql.gz'"
• "docker-volume-backup.exec-label=BackupMain"

6. On each node (MASTER_1, WORKER_1, WORKER_2) start offen-backup docker
7. Mount volume: - pg-backup-data:/backup/pg-backup-data:ro
8. Add labels for each offen-backup (MASTER_1, WORKER_1, WORKER_2)

• "docker-volume-backup.archive-post=/bin/sh -c 'backup'"
• "docker-volume-backup.exec-label=BackupMain"

9. Start one more offen-backup on MASTER_1 = OffenBackupMain
10. For OffenBackupMain mount: /var/run/docker.sock:/var/run/docker.sock
11. For OffenBackupMain add env variable: EXEC_LABEL=BackupMain
12. Go inside OffenBackupMain - docker exec -it ...
13. Run command: backup
14. See: backup only ONE postgres docker - on MASTER_1. Two others - not

Expected behavior
OffenBackup docker in swarm mode should search labels on all nodes

Version (please complete the following information):

• Image Version: v2.39.1
• Docker Version: 25.0.5

Additional context
No additional context

[m90]

This is a limitation of Docker: when listing containers in a multi node swarm setup, you'll only see containers from a single node. This should probably be mentioned in the documentation.

If you want to achieve what you describe, you should place your labels in the deploy section so that services will be stopped and restarted instead of containers, which shall work across nodes.

[zibellon]

I try to replace Postgres docker labels into deploy section - Not working. What I can do - to backup all my Postgres docker with OffenBackup ?

[m90]

A setup that could work for you is:

• for each Postgres instance, deploy a dedicated backup service using the same placement rules you use for your Postgres services
• configure each backup service to back up the postgres instance that will be placed on the same node, and nothing else

See: backup only ONE postgres docker - on MASTER_1. Two others - not

It's important to understand that the mutliple backup services are not connected in any way. If you now trigger a backup for one of the services, either manually or by cron schedule, this will only be backing up a single postgres instance. If you want to take backups of all of them, invoke the command for all of the backup containers.

[m90]

I'm closing this as there's no more conversation, if you require further help at some point, feel free to open a new issue.

[zibellon]

@m90

For each postgres deploy dedicated backup service. Okay. BUT: My postgres instances deploy on worker nodes, before backup I need to exec script (pg_dump ....) and your service cant do it, if it deploy on worker node (In swarm mode).
This solution - not work

---

I spent some time and write .sh script for backup WHOLE swarm cluster, depends on labels and use your open-source project

swarm-backuper.sh

1. GET all volumes from all nodes. Start docker dind container on each node and get volume list
2. GET all services with label volume-list. in this label - we enter volumes, which we want to backup for this service. JOIN this volumes with volumes from step one. Result: MAP<node-name, [volume1, volume2, etc]>
3. RUN all labels: exec-pre. Run separate container on each node, where we need to run exec-pre and run docker exec ...
4. STOP all services with label: stop. Important: stop === scale to 0, Stop only services in replicated mode
5. RUN offen-backup on each node with all volumes from step 2
6. RESTORE all services with label: stop. Important: we need to use the JSON from step 4. Scale to original replicas number, before step 4
7. RUN all labels: exec-post

How we can run it

swarm-backuper-stack.yaml