[ARO-22145] Bump to Azure Linux 3.0 by rhamitarora · Pull Request #4766 · Azure/ARO-RP

rhamitarora · 2026-04-13T02:51:44Z

Which issue this PR addresses:

Fixes ARO-22145 — Migrate Azure Red Hat OpenShift RP/Gateway VMSS from Azure Linux 2.0 (EOL July 31, 2025) to Azure Linux 3.0.

What this PR does / why we need it:

Switching VMSS base images from CBL-Mariner 2.0 to Azure Linux 3.0 FIPS
Updating Dockerfiles to use azurelinux base images instead of mariner
Adding podman 5.x dependencies (crun, netavark) required on Azure Linux 3
Removing the iptables firewalld backend switch (Azure Linux 3 uses nftables natively)
Updating fluentbit build for Azure Linux 3 compatibility
Bumping e2e and CI pipeline resources to Azure Linux 3
Adding the Azure Linux extended repo for additional package availability

Test plan for issue:

Verify RP and Gateway VMSS boot and run successfully on Azure Linux 3 FIPS images

Is there any documentation that needs to be updated for this PR?

How do you know this will function as expected in production?

INT and Canary Testing
Refer attached screen-shots from https://redhat.atlassian.net/browse/ARO-22197

openshift-ci · 2026-04-13T02:52:56Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

github-actions · 2026-04-13T16:48:01Z

Please rebase pull request.

github-actions · 2026-04-13T16:48:01Z

Please rebase pull request.

Podman 5.x on Azure Linux 3 requires crun (OCI runtime), netavark (network stack), and aardvark-dns explicitly installed. Without these, az acr login fails with "could not find netavark" on RP and gateway VMSS. Made-with: Cursor

aardvark-dns is not a separate package in Azure Linux 3 repos. DNS functionality is bundled with netavark on this platform. Made-with: Cursor

On Azure Linux 3, nftables is the default and native firewall backend. Forcing iptables causes firewalld to crash with a DBus NoReply error because the iptables backend is not functional on this platform. Made-with: Cursor

…e Linux 3) Made-with: Cursor

…ackages - Use block list for nginx command in route/loadbalancer e2e manifests - Rename dnf_*_pkgs to tdnf_*_pkgs and use tdnf consistently with extended repo - Regenerate gateway and rp production deploy assets Made-with: Cursor

Add a file-level comment to util-packages.sh clarifying that the RP and gateway VMSS bootstrap uses tdnf exclusively (extended repo, update, and install), consistent with the dev-env Azure Linux migration in PR #4777. Made-with: Cursor

…red gallery The Mariner 2 FIPS marketplace SKU was absent from the platform-image allowlist for VMSS Automatic OS Upgrades, so ARO used the non-FIPS image and configured FIPS manually at boot. Azure Linux 3 FIPS is referenced via the 1P Shared Gallery, which uses the gallery-based automatic upgrade path and is not subject to that allowlist restriction. Addresses reviewer question from PR #4777. Made-with: Cursor

Made-with: Cursor

Switch configure_repo_azurelinux_extended to use dnf instead of tdnf, and update the default argument fallback from 1 to empty string. Made-with: Cursor

Made-with: Cursor

- Replace dnf with tdnf in configure_repo_azurelinux_extended in util-packages.sh to prevent VMSS bootstrapping failure on Azure Linux 3 where dnf is not present - Replace yum with tdnf in devProxyVMSS.sh weekly cron job to prevent silent failures; rename cron file from yumupdate to tdnfupdate - Regenerate assets after changes Made-with: Cursor

rhamitarora · 2026-04-29T06:59:04Z

devProxyVMSS.sh:67 still runs yum update -y weekly, but Azure Linux 3 uses tdnf. That cron will silently fail.

Yes. Implemented that change.

Copilot

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

pkg/deploy/generator/scripts/util-packages.sh:58

tdnf_install_pkgs builds the command by feeding "${pkgs[@]}" through mapfile with a space delimiter. With a here-string this typically leaves a trailing newline in the last element (and relies on word-splitting semantics), which can produce an invalid package name and make installs flaky. Prefer appending the array directly to cmd (e.g., cmd+=("${pkgs[@]}") ) so each package is passed as its own argv element with no delimiter/newline issues.

    # Reference: https://www.shellcheck.net/wiki/SC2206
    # append pkgs array to cmd
    mapfile -O $(( ${#cmd[@]} + 1 )) -d ' ' cmd <<< "${pkgs[@]}"
    local -r cmd

    log "Attempting to install packages: ${pkgs[*]}"
    retry cmd "$2" "${3:-}"

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-29T16:51:43Z

    log "FIPS mode is enabled"
 }

-# fips_configure
-#
-# Configures VM to run with fips mode enabled
-#
-# Taken and refactored from https://eng.ms/docs/products/azure-linux/features/security/fips
-# TODO remove this once sku cbl-mariner-2-gen2-fips is supported by automatic OS updates
-#   * Reference: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade#supported-os-images
-fips_configure() {
-    # shellcheck disable=SC2034
-    local boot_uuid
-    get_boot_dev_uuid boot_uuid
-
-    local grub2_env
-    if grub2_env="$(grub2-editenv - list | grep kernelopts)"; then
-        grub2-editenv - set "$grub2_env fips=1 $boot_uuid"
-    else
-        grubby --update-kernel=ALL --args="fips=1 $boot_uuid"
-    fi
-
-    # fips mode verification will fail until after the vm has been rebooted
-    # fips_verify
-}
-
 # configure_sshd
 #


With fips_configure removed, get_boot_dev_uuid appears to be unused (no remaining references found). Consider removing it (and the related outdated comment block) to reduce dead code and avoid future confusion about how FIPS is expected to be configured on these images.

Copilot · 2026-04-29T16:51:43Z

 configure_rpm_repos() {
    log "starting"

-    configure_repo_mariner_extended "$1" "${2:-1}"
+    configure_repo_azurelinux_extended "$1" "${2:-}"
 }


configure_rpm_repos now forwards the caller-provided retry count into configure_repo_azurelinux_extended. Since rpVMSS/gatewayVMSS pass pkg_retry_count=60, this can make enabling the extended repo retry for up to ~30 minutes (60 * 30s) before failing. Consider keeping a small fixed retry budget for repo enablement (e.g., default to 1–5) and reserving the larger retry count for package update/install operations.

rhamitarora force-pushed the rhamitarora/ARO-22145-azure-linux3 branch from 273c839 to d499bc1 Compare April 13, 2026 02:54

rhamitarora changed the title ~~azure linux3~~ [ARO-22145] Bump to Azure Linux 3.0 Apr 13, 2026

github-actions Bot added the needs-rebase branch needs a rebase label Apr 13, 2026

rhamitarora force-pushed the rhamitarora/ARO-22145-azure-linux3 branch from d499bc1 to 454c1c8 Compare April 20, 2026 12:40

github-actions Bot removed the needs-rebase branch needs a rebase label Apr 20, 2026

rhamitarora marked this pull request as ready for review April 20, 2026 16:05

Copilot AI review requested due to automatic review settings April 20, 2026 16:05

rhamitarora requested review from alcasim, bennerv, cadenmarchese, hawkowl, hlipsig, jharrington22, kevinobriendotca, kimorris27, mociarain, mrWinston, rogbas, sankur-codes, tiguelu, tsatam, tuxerrante, ventifus, wanghaoran1988 and yjst2012 as code owners April 20, 2026 16:05

Copilot started reviewing on behalf of rhamitarora April 20, 2026 16:06 View session

rhamitarora added the priority-medium Medium priority issue or pull request label Apr 20, 2026

hawkowl and others added 20 commits April 29, 2026 09:08

bump dev to azlinux3

29d4ce4

bump to azure linux 3 FIPS for the VMs

36216f0

move to fips-by-default on azurelinux3

7bf311c

add the extended repo as per docs

4d8294b

whitespace

184de55

bump e2e resources to azurelinux

2f68bed

bump onebranch for validate-roledef

a2d6db5

bump changes for fluentbit to build on azurelinux3

dde2229

use 3.12 for ci pipeline too?

e99a25d

Remove aardvark-dns from install_pkgs (not available on Azure Linux 3)

5b8fede

aardvark-dns is not a separate package in Azure Linux 3 repos. DNS functionality is bundled with netavark on this platform. Made-with: Cursor

Stop switching firewalld backend to iptables on Azure Linux 3

4a89db1

On Azure Linux 3, nftables is the default and native firewall backend. Forcing iptables causes firewalld to crash with a DBus NoReply error because the iptables backend is not functional on this platform. Made-with: Cursor

Restore fluentbit Dockerfile and Makefile to CBL Mariner (revert Azur…

9910313

…e Linux 3) Made-with: Cursor

fix indentation of StorageProfile block in gateway and RP VMSS

a09022e

Made-with: Cursor

use dnf instead of tdnf for Azure Linux 3 extended repo install

ec5b824

Switch configure_repo_azurelinux_extended to use dnf instead of tdnf, and update the default argument fallback from 1 to empty string. Made-with: Cursor

Regenerate gateway and rp production assets after rebase

b5b7ea0

Made-with: Cursor

rhamitarora force-pushed the rhamitarora/ARO-22145-azure-linux3 branch from a40c5fb to f31a8ea Compare April 29, 2026 03:51

rhamitarora requested a review from Copilot April 29, 2026 06:58

Copilot started reviewing on behalf of rhamitarora April 29, 2026 06:59 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Comment thread pkg/deploy/generator/scripts/util-system.sh

rhamitarora requested a review from Copilot April 29, 2026 11:02

Copilot started reviewing on behalf of rhamitarora April 29, 2026 11:03 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

rhamitarora requested a review from Copilot April 29, 2026 16:41

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ARO-22145] Bump to Azure Linux 3.0#4766

[ARO-22145] Bump to Azure Linux 3.0#4766
rhamitarora wants to merge 22 commits intomasterfrom
rhamitarora/ARO-22145-azure-linux3

rhamitarora commented Apr 13, 2026 •

edited

Loading

Uh oh!

openshift-ci Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

rhamitarora commented Apr 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

rhamitarora commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue this PR addresses:

What this PR does / why we need it:

Test plan for issue:

Is there any documentation that needs to be updated for this PR?

How do you know this will function as expected in production?

Uh oh!

openshift-ci Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

rhamitarora commented Apr 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rhamitarora commented Apr 13, 2026 •

edited

Loading