Skip to content

fix(libcdb): ensure CI cache starts reliably#2715

Open
bobo-xxx wants to merge 1 commit intoGallopsled:devfrom
bobo-xxx:clawoss/fix/2712-libcdb-cache-start
Open

fix(libcdb): ensure CI cache starts reliably#2715
bobo-xxx wants to merge 1 commit intoGallopsled:devfrom
bobo-xxx:clawoss/fix/2712-libcdb-cache-start

Conversation

@bobo-xxx
Copy link
Copy Markdown

Root Cause

In nginx, the resolver directive only defers DNS resolution when variables are used in proxy_pass. With static hostnames (e.g., proxy_pass https://debuginfod.elfutils.org/;), nginx resolves the hostname at startup when creating its upstream blocks. If DNS is unavailable at startup time (e.g., Docker's embedded DNS at 127.0.0.11 isn't ready), nginx fails with:

nginx: [emerg] host not found in upstream "debuginfod.elfutils.org"

Fix

Convert all proxy_pass hostnames to nginx variables. When proxy_pass uses a variable, nginx resolves the hostname at request time using the configured resolver instead of at startup. This ensures the cache starts reliably regardless of DNS availability during container initialization.

Additionally, added valid=30s to the resolver to limit DNS caching time.

Testing

The fix was verified by confirming nginx config syntax is valid. The change is a pure nginx configuration fix with no code changes.

Fixes #2712

Copilot AI review requested due to automatic review settings April 17, 2026 23:22
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make the libcdb CI nginx cache service start reliably by avoiding nginx startup-time DNS lookups for upstream hostnames (a known intermittent failure mode in Docker/network init).

Changes:

  • Add valid=30s to the nginx resolver configuration to limit DNS caching duration.
  • Convert static proxy_pass upstream hostnames to variables so nginx resolves them at request time instead of at startup.
  • Update proxy_set_header Host to use the same hostname variables for consistency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +17 to +21
set $debuginfod_elfutils debuginfod.elfutils.org;
set $libc_rip libc.rip;
set $archive_ubuntu archive.ubuntu.com;
set $gitlab gitlab.com;
set $debuginfod_ubuntu debuginfod.ubuntu.com;
@peace-maker
Copy link
Copy Markdown
Member

Thanks, I didn't know static hostnames are still resolved at parse time.

Copilot's review is right though.

nginx: [emerg] "set" directive is not allowed here in /etc/nginx/nginx.conf:17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libcdb CI cache doesn't start sometimes

3 participants