Skip to content

Commit 94ac45a

Browse files
oleksandr-ncoleksandr-nc
authored
feat: high level wrapper around Docker API (#39)
* feat: high level wrapper around Docker API Signed-off-by: Oleksander Piskun <oleksandr2088@icloud.com>
1 parent 234ce12 commit 94ac45a

File tree

8 files changed

+1375
-79
lines changed

8 files changed

+1375
-79
lines changed

‎README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -346,6 +346,39 @@ docker run \
346346
-d nextcloud-appapi-harp:local
347347
```
348348

349+
#### Debugging HaRP
350+
351+
##### One time initializing steps:
352+
353+
1. Create virtual environment
354+
2. Install `pydantic` (you can look at exact version in the **Dockerfile*) and `git+https://github.com/cloud-py-api/haproxy-python-spoa.git`
355+
3. Set next environment variables for running `haproxy_agent.py` script:
356+
```
357+
HP_LOG_LEVEL=info;NC_INSTANCE_URL=http://nextcloud.local;HP_SHARED_KEY=some_very_secure_password;HP_FRP_DISABLE_TLS=true
358+
```
359+
4. Create folder `dev` at the root of repository, extract there content of the desired archive with the [FRP](https://github.com/fatedier/frp/releases/latest) archive which is located at `exapps_dev` folder of this repo.
360+
5. Edit the `data/nginx/vhost.d/nextcloud.local_location` file from the `nextcloud-docker-dev` to point `/exapps/` web route to the host:
361+
```
362+
proxy_pass http://172.17.0.1:8780;
363+
```
364+
365+
> **Note:** my original content from my dev machine of file `nextcloud.local_location`:
366+
> ```nginx
367+
> location /exapps/ {
368+
> proxy_pass http://172.17.0.1:8780;
369+
> }
370+
> ```
371+
6. Use `docker compose up -d --force-recreate proxy` command from Julius `nextcloud-docker-dev` to recreate the proxy container.
372+
7. Register `HaRP` from the **Host** template. Replace `localhost` with `host.docker.internal` in `HaRP Host` field.
373+
374+
##### Steps to run all parts of HaRP after initializing:
375+
376+
1. Run FRP Server with `./dev/frps -c ./development/debugging/frps.toml` command.
377+
2. Run the FRP Client to connect Docker Engine to the FRP Server with `./dev/frpc -c ./development/debugging/frpc.toml` command.
378+
3. Run `./development/debugging/redeploy_haproxy_host.sh` command to redeploy `appapi-harp` container **with HaProxy only**.
379+
380+
> **Note:** Existing `appapi-harp` container will be removed.
381+
349382
## Contributing
350383
351384
Contributions to HaRP are welcome. Feel free to open issues, discussions or submit pull requests with improvements, bug fixes, or new features.

‎development/debugging/Dockerfile‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
2+
# SPDX-License-Identifier: AGPL-3.0-or-later
3+
4+
FROM haproxy:3.1.2-alpine3.21
5+
6+
USER root
7+
8+
# Bind addresses for 2 frontends (HTTP + HTTPS for exapps) and FRP Server.
9+
# If /certs/cert.pem does not exist, EXAPPS HTTPS frontend are disabled automatically.
10+
ENV HP_EXAPPS_ADDRESS="0.0.0.0:8780" \
11+
HP_EXAPPS_HTTPS_ADDRESS="0.0.0.0:8781" \
12+
HP_FRP_ADDRESS="0.0.0.0:8782" \
13+
HP_FRP_DISABLE_TLS="false" \
14+
HP_TIMEOUT_CONNECT="30s" \
15+
HP_TIMEOUT_CLIENT="30s" \
16+
HP_TIMEOUT_SERVER="1800s" \
17+
NC_INSTANCE_URL="" \
18+
HP_LOG_LEVEL="warning"
19+
20+
RUN set -ex; \
21+
apk add --no-cache \
22+
git \
23+
ca-certificates \
24+
tzdata \
25+
bash \
26+
curl \
27+
openssl \
28+
bind-tools \
29+
nano \
30+
vim \
31+
envsubst \
32+
frp \
33+
python3 \
34+
py3-pip \
35+
py3-aiohttp \
36+
wget \
37+
tar \
38+
netcat-openbsd; \
39+
chmod -R 777 /tmp;
40+
41+
# Main haproxy config template
42+
COPY --chmod=664 ./development/debugging/haproxy.cfg /haproxy.cfg
43+
44+
# SPOE config
45+
COPY --chmod=664 spoe-agent.conf /etc/haproxy/spoe-agent.conf
46+
47+
ENTRYPOINT ["haproxy", "-f", "/haproxy.cfg", "-W", "-db"]
48+
49+
LABEL com.centurylinklabs.watchtower.enable="false"

‎development/debugging/frpc.toml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
2+
# SPDX-License-Identifier: AGPL-3.0-or-later
3+
4+
serverAddr = "127.0.0.1" # Replace with your HP_FRP_ADDRESS host
5+
serverPort = 8782 # Default port for FRP or the port your reverse proxy listens on
6+
loginFailExit = false # If the FRP (HaRP) server is unavailable, continue trying to log in.
7+
8+
metadatas.token = "some_very_secure_password"
9+
10+
log.level = "info"
11+
12+
[[proxies]]
13+
remotePort = 24000 # we set it to 24000 as it is the basic Docker Engine
14+
name = "bundled-deploy-daemon" # Unique name for each Docker Engine
15+
type = "tcp"
16+
[proxies.plugin]
17+
type = "unix_domain_socket"
18+
unixPath = "/var/run/docker.sock"

‎development/debugging/frps.toml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
2+
# SPDX-License-Identifier: AGPL-3.0-or-later
3+
4+
bindAddr = "127.0.0.1"
5+
bindPort = 8782
6+
7+
transport.tls.force = false
8+
9+
log.level = "info"
10+
11+
maxPortsPerClient = 1
12+
allowPorts = [
13+
{ start = 23000, end = 23999 },
14+
{ start = 24000, end = 24099 }
15+
]
16+
17+
[[httpPlugins]]
18+
addr = "127.0.0.1:8200"
19+
path = "/frp_handler"
20+
ops = ["Login"]

‎development/debugging/haproxy.cfg‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
2+
# SPDX-License-Identifier: AGPL-3.0-or-later
3+
4+
global
5+
log stdout local0 info
6+
maxconn 8192
7+
ca-base /etc/ssl/certs
8+
9+
defaults
10+
log global
11+
option httplog
12+
option dontlognull
13+
timeout connect 30s
14+
timeout client 30s
15+
timeout server 1800s
16+
17+
18+
###############################################################################
19+
# FRONTEND: ex_apps (HTTP)
20+
###############################################################################
21+
frontend ex_apps
22+
mode http
23+
bind 0.0.0.0:8780
24+
25+
filter spoe engine exapps-spoe config /etc/haproxy/spoe-agent.conf
26+
http-request silent-drop if { var(txn.exapps.bad_request) -m int eq 1 }
27+
http-request return status 401 content-type text/plain string "401 Unauthorized" if { var(txn.exapps.unauthorized) -m int eq 1 }
28+
http-request return status 403 content-type text/plain string "403 Forbidden" if { var(txn.exapps.forbidden) -m int eq 1 }
29+
http-request return status 404 content-type text/plain string "404 Not Found" if { var(txn.exapps.not_found) -m int eq 1 }
30+
use_backend %[var(txn.exapps.backend)]
31+
32+
###############################################################################
33+
# BACKENDS: ex_apps & ex_apps_backend_w_bruteforce
34+
###############################################################################
35+
backend ex_apps_backend
36+
mode http
37+
server frp_server 0.0.0.0
38+
http-request set-path %[var(txn.exapps.target_path)]
39+
http-request set-dst var(txn.exapps.target_ip)
40+
http-request set-dst-port var(txn.exapps.target_port)
41+
http-request set-header EX-APP-ID %[var(txn.exapps.exapp_id)]
42+
http-request set-header EX-APP-VERSION %[var(txn.exapps.exapp_version)]
43+
http-request set-header AUTHORIZATION-APP-API %[var(txn.exapps.exapp_token)]
44+
http-request set-header AA-VERSION "32" # TO-DO: temporary, remove it after we update all ExApps.
45+
46+
backend ex_apps_backend_w_bruteforce
47+
mode http
48+
server frp_server 0.0.0.0
49+
http-request set-path %[var(txn.exapps.target_path)]
50+
http-request set-dst var(txn.exapps.target_ip)
51+
http-request set-dst-port var(txn.exapps.target_port)
52+
http-request set-header EX-APP-ID %[var(txn.exapps.exapp_id)]
53+
http-request set-header EX-APP-VERSION %[var(txn.exapps.exapp_version)]
54+
http-request set-header AUTHORIZATION-APP-API %[var(txn.exapps.exapp_token)]
55+
http-request set-header AA-VERSION "32" # TO-DO: temporary, remove it after we update all ExApps.
56+
filter spoe engine exapps-bruteforce-protection-spoe config /etc/haproxy/spoe-agent.conf
57+
58+
###############################################################################
59+
# BACKEND: nextcloud_control (HTTP)
60+
###############################################################################
61+
backend nextcloud_control_backend
62+
mode http
63+
server nextcloud_control 127.0.0.1:8200
64+
http-request set-path %[var(txn.exapps.target_path)]
65+
66+
###############################################################################
67+
# BACKEND: docker_engine (HTTP)
68+
###############################################################################
69+
backend docker_engine_backend
70+
mode http
71+
server frp_server 127.0.0.1
72+
http-request set-dst-port var(txn.exapps.target_port)
73+
http-request set-path %[var(txn.exapps.target_path)]
74+
75+
# docker system _ping
76+
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/_ping$ } METH_GET
77+
# docker inspect image
78+
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/.*/json } METH_GET
79+
# container inspect: GET containers/%s/json
80+
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/json } METH_GET
81+
# container inspect: GET containers/%s/logs
82+
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/logs } METH_GET
83+
84+
# image pull: POST images/create?fromImage=%s
85+
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
86+
http-request deny
87+
88+
89+
backend agents
90+
mode tcp
91+
timeout connect 5s
92+
timeout server 3m
93+
option spop-check
94+
server agent1 127.0.0.1:9600 check

‎development/debugging/redeploy_haproxy_host.sh‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/sh
2+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
3+
# SPDX-License-Identifier: AGPL-3.0-or-later
4+
5+
# This file can be used for development for the "manual install" deployment type when FRP is disabled.
6+
# For Julius Docker-Dev, you need to additionally edit the `data/nginx/vhost.d/nextcloud.local_location` file,
7+
# changing `appapi-harp` to `172.17.0.1` and restart the "proxy" container.
8+
9+
docker container remove --force appapi-harp
10+
11+
docker build -f development/debugging/Dockerfile -t nextcloud-appapi-harp:debug .
12+
13+
docker run \
14+
--name appapi-harp -h appapi-harp \
15+
--restart unless-stopped \
16+
--network=host \
17+
-d nextcloud-appapi-harp:debug

‎haproxy.cfg.template‎

Lines changed: 1 addition & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -108,44 +108,7 @@ backend docker_engine_backend
108108
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/json } METH_GET
109109
# container inspect: GET containers/%s/logs
110110
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/logs } METH_GET
111-
# container start/stop: POST containers/%s/start containers/%s/stop
112-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/((start)|(stop)) } METH_POST
113-
# container rm: DELETE containers/%s
114-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+ } METH_DELETE
115-
# container update/exec: POST containers/%s/update containers/%s/exec
116-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/((update)|(exec)) } METH_POST
117-
# container put: PUT containers/%s/archive
118-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/archive } METH_PUT
119-
# run exec instance: POST exec/%s
120-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/exec/[a-zA-Z0-9_.-]+/start } METH_POST
121-
122-
# container create: POST containers/create?name=%s
123-
# ACL to restrict container name to nc_app_[a-zA-Z0-9_.-]+
124-
acl nc_app_container_name url_param(name) -m reg -i "^nc_app_[a-zA-Z0-9_.-]+"
125-
126-
# ACL to restrict the number of Mounts to 1
127-
acl one_mount_volume req.body -m reg -i "\"Mounts\"\s*:\s*\[\s*(?:(?!\"Mounts\"\s*:\s*\[)[^}]*)}[^}]*\]"
128-
# ACL to deny if there are any binds
129-
acl binds_present req.body -m reg -i "\"HostConfig\"\s*:.*\"Binds\"\s*:"
130-
# ACL to restrict the type of Mounts to volume
131-
acl type_not_volume req.body -m reg -i "\"Mounts\":\s*\[[^\]]*(\"Type\":\s*\"(?!volume\b)\w+\"[^\]]*)+\]"
132-
http-request deny if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !one_mount_volume binds_present type_not_volume METH_POST
133-
134-
# ACL to restrict container creation, that it has HostConfig.Privileged(by searching for "Privileged" word in all payload) not set
135-
acl no_privileged_flag req.body -m reg -i "\"Privileged\""
136-
# ACL to allow mount volume with strict pattern for name: nc_app_[a-zA-Z0-9_.-]+_data
137-
acl nc_app_volume_data_only req.body -m reg -i "\"Mounts\":\s?\[\s?{[^}]*\"Source\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
138-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !no_privileged_flag nc_app_volume_data_only METH_POST
139-
# end of container create
140-
141-
# volume create: POST volumes/create
142-
# restrict name
143-
acl nc_app_volume_data req.body -m reg -i "\"Name\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
144-
# do not allow to use "device" word e.g., "--opt device=:/path/to/dir"
145-
acl volume_no_device req.body -m reg -i "\"device\""
146-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/create } nc_app_volume_data !volume_no_device METH_POST
147-
# volume rm: DELETE volumes/%s
148-
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/nc_app_[a-zA-Z0-9_.-]+_data } METH_DELETE
111+
149112
# image pull: POST images/create?fromImage=%s
150113
http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
151114
http-request deny

0 commit comments

Comments
 (0)