Updating WHDBooter files fails/local WHDBoot directory never created/populated

[mrandreastoth]

Updating WHDBooter files fails with the log file showing the following failures...

Downloading /home/andreas/.local/share/amiberry/whdboot/WHDLoad ...
2025-01-26 19:57:59 URL:https://raw.githubusercontent.com/BlitterStudio/amiberry/refs/heads/master/whdboot/WHDLoad [156867/156867] -> "/home/andreas/.local/share/amiberry/whdboot/WHDLoad.tmp" [1]

Renaming downloaded temporary file /home/andreas/.local/share/amiberry/whdboot/WHDLoad.tmp to final destination
Downloading /home/andreas/.local/share/amiberry/whdboot/JST ...
2025-01-26 19:57:59 URL:https://raw.githubusercontent.com/BlitterStudio/amiberry/refs/heads/master/whdboot/JST [89108/89108] -> "/home/andreas/.local/share/amiberry/whdboot/JST.tmp" [1]

Renaming downloaded temporary file /home/andreas/.local/share/amiberry/whdboot/JST.tmp to final destination
Downloading /home/andreas/.local/share/amiberry/whdboot/AmiQuit ...
2025-01-26 19:58:00 URL:https://raw.githubusercontent.com/BlitterStudio/amiberry/refs/heads/master/whdboot/AmiQuit [76/76] -> "/home/andreas/.local/share/amiberry/whdboot/AmiQuit.tmp" [1]

Renaming downloaded temporary file /home/andreas/.local/share/amiberry/whdboot/AmiQuit.tmp to final destination
Downloading /home/andreas/.local/share/amiberry/whdboot/boot-data.zip ...
2025-01-26 19:58:00 URL:https://raw.githubusercontent.com/BlitterStudio/amiberry/refs/heads/master/whdboot/boot-data.zip [703471/703471] -> "/home/andreas/.local/share/amiberry/whdboot/boot-data.zip.tmp" [1]

Renaming downloaded temporary file /home/andreas/.local/share/amiberry/whdboot/boot-data.zip.tmp to final destination
Downloading /home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick33180.A500.RTB ...
/home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick33180.A500.RTB.tmp: No such file or directory

Downloading /home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick34005.A500.RTB ...
/home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick34005.A500.RTB.tmp: No such file or directory

Downloading /home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40063.A600.RTB ...
/home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40063.A600.RTB.tmp: No such file or directory

Downloading /home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40068.A1200.RTB ...
/home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40068.A1200.RTB.tmp: No such file or directory

Downloading /home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40068.A4000.RTB ...
/home/andreas/.local/share/amiberry/whdboot/save-data/Kickstarts/kick40068.A4000.RTB.tmp: No such file or directory

Downloading /home/andreas/.local/share/amiberry/whdboot/game-data/whdload_db.xml ...
/home/andreas/.local/share/amiberry/whdboot/game-data/whdload_db.xml.tmp: No such file or directory

Notes

• On starting up Amiberry, Amiberry never creates and populates "/home/andreas/.local/share/amiberry/whdboot" with the data from "/usr/local/share/amiberry/whdboot"

• The directory "/home/andreas/.local/share/amiberry/whdboot" does not exist at the start of the WHDBooter updating process nor does it exist after the process has failed.

To Reproduce

1. Run up Amiberry

2. Go to paths

3. Click on "Update WHDBooter Files" (optionally enable logging beforehand)

Expected behaviour
The process completes successfully with the "/home/andreas/.local/share/amiberry/whdboot" populated with all the required files.

Actual behaviour
The process fails displaying an error message. The "/home/andreas/.local/share/amiberry/whdboot" isn't even created.

Screenshots
The error message...

Desktop (please complete the following information):

• OS: EndeavourOS

• Version 2024.09.22

Additional context
Built from source pulled some hours ago.

[giantclambake]

Cannot recreate this here, using the usual build process (logfile attached)

I do note however the build prefix=/usr/local is being used... (perms?)

@mrandreastoth ~ can you detail your build & install commands?

amiberry.log.gz

[giantclambake]

Additional:

Retested this on CachyOS (Arch based), but still can't recreate the issue ~ FTR ... this was a clean install (previously installed amiberry stuffs removed).....at cmdline ;

git clone https://github.com/BlitterStudio/amiberry
cd amiberry
cmake -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/usr && cmake --build build
sudo cmake --install build --prefix /usr

Seems to work as expected, whdbooter files update as expected, cannot fault the installation ...hmm...

....I'll do it again using cmake defaults, just to check that...

edit: cmake defaults (prefix=/usr/local) also works as expected including updating whdbooter files...

Cannot recreate ; need more details wrt build/install process undertaken

[mrandreastoth]

I had also cleaned up from the command-line before I rebuilt and reinstalled, all using cmake, and all using defaults. Everything installed to the shares as expected including the whdboot files. I accessed and verified these as myself (user andreas), no special permissions required, i.e., permissions are not an issue.

The issue seems to stem from the fact that when I then ran up Amiberry, Amiberry did not properly create the home environment as it did not copy the whdboot files from "/usr/local/share/amiberry/whdboot" into "$XDG_CONFIG_HOME/amiberry", which is what it seems it should have done as the whdboot_path path, in amiberry.conf, defaults to "XDG_CONFIG_HOME/amiberry/whdboot".

After I discovered this issue, I blew away the "$HOME/Amiberry" and the "$XDG_CONFIG_HOME/amiberry" directories, created by running Amiberry, and I recreated these using my personal home directory preparation script, running as myself, for Amiberry, the script can be seen here #1612 (comment). This script copies the whdboot directory from "/usr/local/share/amiberry/whdboot" into "$HOME/Amiberry" as whdnoot and modifies the amiberry.conf file to point at this location. Interestingly running Amiberry in this state, i.e., with all the WHDBoot files present, and hitting the "Update HDBooter Files" had the update process succeed.

[giantclambake]

I've got no real idea why you're using scripts...

//if I really want to exorcise amiberry from the current system makeup (debian example)

rm -fr /home/gcb/.config/amiberry
rm -fr /home/gcb/.local/share/amiberry
rm -fr /home/gcb/Amiberry
sudo rm -fr /usr/share/amiberry
sudo rm /usr/share/applications/Amiberry.desktop
sudo rm /usr/share/mime/packages/amiberry.xml
sudo rm -fr /usr/lib/x86_64-linux-gnu/amiberry

There is no need to 'prepare' (make directories) before installation ~ the cmake install command does that automatically.

I take it you have read -> https://github.com/BlitterStudio/amiberry/wiki/Compile-from-source ??

Nothing more than what's documented there is required ;)

[midwan]

Your steps are missing an important bit: how did you install Amiberry in your system in the first place?
If you follow the instructions in the wiki, you will use CMake to install it, along with an optional PREFIX parameter. Doing it that way, I cannot recreate the problem here.

If you followed another method, perhaps something is missing there.

There shouldn't be any need for custom scripts to get Amiberry installed/running, normally.

[mrandreastoth]

@midwan: the very first time it was a manual install but there after I used CMake to install it. The issue reported herein was following the CMake procedure and did NOT involve my script. I always test my script after I have tested the CMake installation first.

[mrandreastoth]

I made a bit of a boo-boo: I looked in the wrong place for the WHDBooter files, i looked in ~/.config/amiberry, not ~/.local/share/amiberry! Doh!

However, having just rebuilt and reinstalled (using CMake), I can confirm that, although I looked in the wrong place, only the files from ~/.local/share/amiberry! have copied across BUT, as can be seen below, the folders and the data in them have not, and, as such, Amiberry fails to update the WHDBooter files. Copying across all of the missing data, as per my script, is what sees the WHDBooter update working.

Question: why are the missing files that important? Why can't it just download it nevertheless? In fact, isn't this how controller data works, i.e., if the non-user file is missing, isn't it simply downloaded and all is honky-dorey?

$ ls -lh ~/.local/share/amiberry/
total 936K
-rw-r--r-- 1 andreas andreas   76 jan   28 20.07 AmiQuit
-rw-r--r-- 1 andreas andreas 687K jan   28 20.07 boot-data.zip
-rw-r--r-- 1 andreas andreas  88K jan   28 20.07 JST
-rw-r--r-- 1 andreas andreas 154K jan   28 20.07 WHDLoad
ls -lh /usr/local/share/amiberry/whdboot/
total 944K
-rw-r--r-- 1 root root   76 jan   20 16.10 AmiQuit
-rw-r--r-- 1 root root 687K jan   20 16.10 boot-data.zip
drwxr-xr-x 2 root root 4,0K jan   26 18.58 game-data
-rw-r--r-- 1 root root  88K jan   20 16.10 JST
drwxr-xr-x 6 root root 4,0K jan   26 18.58 save-data
-rw-r--r-- 1 root root 154K jan   20 16.10 WHDLoad

[midwan]

New commit to try and improve this further.
It will now also look under /usr/local/share/amiberry/ for the bundled files, if not found in the first two paths.
As a fallback, if all else fails, it will try to recreate the structure from scratch, then download the files from the repositories. This way, no matter how you installed it, it should still create things eventually.

I still cannot recreate what you've done there, but at least this should help :)

[giantclambake]

I can't/couldn't recreate this either (even when using the default cmake $prefix of /usr/local ), but that commit doesn't break anything AFAICT ;) On debian, XDG_DATA_DIRS="/usr/share/xfce4:/usr/local/share/:/usr/share/" and so amiberry always seemed to find things in $XDG_DATA_DIRS ...

For mine, prefix=/usr/local ... wrt a mainstream distro ... has to be considered as a 'custom' path, relatively speaking, as prefix=/usr is pretty much standard for a linux distro (even my LFS builds) ....like I say though, doesn't seem to break existing behavior with the extra check, so status-quo =)