Unable to build a new custom image - problem with repo fsx-lustre-client #7297
Description
Required Info:
- AWS ParallelCluster version : 3.14 and 3.14.2
- OS Version: Rocky 9.1
Bug description and how to reproduce:
I am trying to build a new parallelcluster using a custom AMI
During the build process, there is a hook to pull down a repository called aws-fsx
As the instance boots, I can login and monitor the logs - in particular /var/log/dnf.log
This shows
2026-03-17T15:02:25+0000 DEBUG YUM version: 4.12.0
2026-03-17T15:02:25+0000 DDEBUG Command: yum -q -y makecache --disablerepo=* --enablerepo=aws-fsx
2026-03-17T15:02:25+0000 DDEBUG Installroot: /
2026-03-17T15:02:25+0000 DDEBUG Releasever: 9.1
2026-03-17T15:02:25+0000 DEBUG cachedir: /var/cache/yum/x86_64/9.1
2026-03-17T15:02:25+0000 DDEBUG Base command: makecache
2026-03-17T15:02:25+0000 DDEBUG Extra commands: ['-q', '-y', 'makecache', '--disablerepo=*', '--enablerepo=aws-fsx']
2026-03-17T15:02:25+0000 DEBUG Unknown configuration option: fastestmirror_enabled = 0 in /etc/yum.repos.d/aws-fsx.repo
2026-03-17T15:02:25+0000 DEBUG Unknown configuration option: fastestmirror_enabled = 0 in /etc/yum.repos.d/epel.repo
2026-03-17T15:02:25+0000 DEBUG Making cache files for all metadata files.
2026-03-17T15:02:25+0000 DEBUG aws-fsx: has expired and will be refreshed.
2026-03-17T15:02:25+0000 DEBUG repo: downloading from remote: aws-fsx
2026-03-17T15:02:25+0000 DEBUG error: Status code: 404 for https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml (IP: 16.15.207.36) (https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml).
2026-03-17T15:02:25+0000 DEBUG error: Status code: 404 for https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml (IP: 16.15.207.36) (https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml).
2026-03-17T15:02:25+0000 DEBUG error: Status code: 404 for https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml (IP: 16.15.207.36) (https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml).
2026-03-17T15:02:25+0000 DEBUG error: Status code: 404 for https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml (IP: 16.15.207.36) (https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml).
2026-03-17T15:02:25+0000 WARNING Errors during downloading metadata for repository 'aws-fsx':
- Status code: 404 for https://fsx-lustre-client-repo.s3.amazonaws.com/el/9.1/x86_64/repodata/repomd.xml (IP: 16.15.207.36)
2026-03-17T15:02:25+0000 DDEBUG Cleaning up.
2026-03-17T15:02:25+0000 SUBDEBUG
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 573, in load
ret = self._repo.load()
File "/usr/lib64/python3.9/site-packages/libdnf/repo.py", line 331, in load
return _repo.Repo_load(self)
libdnf._error.Error: Failed to download metadata for repo 'aws-fsx': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 67, in main
return _main(base, args, cli_class, option_parser_class)
File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 106, in _main
return cli_run(cli, base)
File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 122, in cli_run
cli.run()
File "/usr/lib/python3.9/site-packages/dnf/cli/cli.py", line 1057, in run
return self.command.run()
File "/usr/lib/python3.9/site-packages/dnf/cli/commands/makecache.py", line 50, in run
return self.base.update_cache(timer)
File "/usr/lib/python3.9/site-packages/dnf/base.py", line 372, in update_cache
self.fill_sack(load_system_repo=False, load_available_repos=True) # performs the md sync
File "/usr/lib/python3.9/site-packages/dnf/base.py", line 401, in fill_sack
self._add_repo_to_sack(r)
File "/usr/lib/python3.9/site-packages/dnf/base.py", line 141, in _add_repo_to_sack
repo.load()
File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 580, in load
raise dnf.exceptions.RepoError(str(e))
dnf.exceptions.RepoError: Failed to download metadata for repo 'aws-fsx': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
2026-03-17T15:02:25+0000 CRITICAL Error: Failed to download metadata for repo 'aws-fsx': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
Looking at the URL, I only see data for
centos/7.8
centos/7.9
centos/7
el/7.7
el/7.8
el/7.9
el/7
Given this, I'm surprised that this hasnt been triggered for builds like Rocky8 etc.