Improving community builds

[letsfindaway]

The current qmake based build system of OpenBoard is driven by the needs of the project in Geneva schools:

• Build packages for Windows, MacOS and Ubuntu LTE releases.
• Decouple software updates from any operating system release cycles.
• Optionally bundle a specific Qt version if the target platform's default version of the target platform has some bugs that affect OpenBoard.

The last two requirements, however, contradict the rules for providing packages to a distribution. Here, the release cycles must be consistent with the distribution's release strategy. Some may release a new package version quite quickly, others may take months.

These requirements also led to the decision to install OpenBoard in /opt instead of providing an FHS-compliant file system layout, in order to decouple the software installation from the distribution as much as possible.

There have been several issues and pull requests from the community trying to improve the situation for community builds. But none of them have actually been merged. So my idea is to provide an additional cmake based build system, instead of trying to improve the qmake build, so that we can meet the community's needs without compromising the original goals of the project.

I think this can be a parallel effort with a different focus:

• Focus on Linux in the first place.
• Work on as many distributions as possible with minimal customization.
• Provide an FHS-compliant file system layout.

Keeping this effort separate from the main project allows us to be much freer and put the needs of Linux distributions first. Over time, this build system will evolve and improve, and may eventually be good enough to replace the original qmake build system for Windows and MacOS as well.

Please share your thoughts and ideas!

Links

• Draft pull request for current state of cmake build system
  chore: add cmake build system OpenBoard-org/OpenBoard#698
• Issue and discussion about FHS-confoming filesystem
  Correct installation of files according to FHS OpenBoard-org/OpenBoard#638
• Issue and discussion about a cmake build system
  Create a cmake build system #99
• Branch [WIP] providing cmake based community builds
  master...chore-cmake-community-builds

CMake community builds now available!

After merging OpenBoard-org#830 the cmake based build system is now available in the dev branch. See https://github.com/letsfindaway/OpenBoard/wiki/CMake-builds for usage instructions.

[Vekhir]

Are there plans / Do you consider it likely that the cmake based build system will be merged?

Arch packages intentionally follow upstream very closely with minimal (read: necessary) patches to keep delays during updates at a minimum. We tested the PKGBUILDS with the patches, and I can attest that they work, but no Arch packager will consider a 2000 line patch "close to upstream". I would consider it a fork for the time being. Add to that that qmake is the official way to package openboard - cmake needs to be mentioned as a valid alternative for Linux once it is merged (and get maintained when changes happen or issues arise).

In short, until this is merged, Arch can't use it, but once it is, it very likely will be.

[letsfindaway]

Currently, I don't see it very likely that this will be merged soon. However your arguments are valid and I will discuss with @kaamui whether and how this could be merged, probably as an addition to qmake. However as my patch is also making some code changes in the way directories are located, this might need further work.

[Vekhir]

To keep development centralised and easy to find, it might be useful to create a repo OpenBoard-cmake as a soft-fork of OpenBoard, where you regularly rebase the changes against OpenBoard master.
Then, one might consider you upstream for a package openboard-cmake, which would make packaging as a PKGBUILD possible.
Is that something you are considering?

[letsfindaway]

This is not my preferred solution, but it gives me another idea:

I will create another branch/PR which focuses on cmake for Linux only and which is explicitly meant as addition for community builds and not as replacement to qmake. The code changes will be reworked to be compatible with the current qmake. I will then promote this PR over the existing one with the argument, that it does not affect the way how OpenBoard is currently built for Genova, but helps the community. I think this is then something where @kaamui can much better agree with and where I see a substantial chance that this could be merged.

Second best solution would be to have a master-cmake branch in the main repository, which is regularly rebased against master when a new release is created.

Having this in a separate repo would only be a last resort.

BTW: Thanks for your ideas and valuable input!!!!

[letsfindaway]

I have now created a branch with two commits according to this plan:

• The first commit provides the necessary code changes for an FHS-compliant package layout. For merging it is essential that these changes will not affect the existing qmake builds. As far as I can see, this is the case.
• The second commit only adds files needed for cmake and therefore also does not affect qmake builds.
• References to Windows or MacOS builds are not included - it is purely about Linux.

See master...chore-cmake-community-builds for the branch diff. I think this approach should have a change to be merged. I will create the PR as soon as it is clear what the base branch should be (master or 1.7.1-dev).

[letsfindaway]

The cmake build system is now merged to the dev branch!

[Vekhir]

Optionally bundle a specific Qt version if the target platform's default version of the target platform has some bugs that affect OpenBoard.

Out of interest, has this ever occurred?

[letsfindaway]

Yes, it has occured. And OpenBoard still contains some workarounds for unresolved Qt issues. See OpenBoard-org#706 for a (not completely up-to-date) collection of issues with different Qt versions on various platforms.

[Vekhir]

Bugs sure, but is a specific Qt version being bundled to avoid these issues? That's the benefit of qmake, isn't it?

[letsfindaway]

Just look at the last release: https://github.com/OpenBoard-org/OpenBoard/releases/tag/v1.6.4. The package for Ubuntu 20.04 is so much bigger than that for 22.04 because it contains a set of Qt libraries. Same happened for 1.6.2 and 1.6.3.

But I would not phrase that a benefit of qmake. You could do the same with cmake. It however causes a build system structure with manually configured paths instead of automatic recognition everywhere.

[Vekhir]

Alright, I see

[Vekhir]

There is currently the SYSCONF_INSTALL_DIR switch which has to be provided. In any case I can imagine, it is going to be /etc. It would simplify the build process if per default the variable would be set to that - and it would be fine with me if it was hardcoded.

On a related note, are the files widgets.md5sum and OpenBoard.css supposed to be configured, i.e. changed, by the user? Perhaps a better place for them would be in /usr/share/openboard. I don't think they necessarily need a folder of their own, they can be put there as plain files.

[letsfindaway]

About SYSCONF_INSTALL_DIR: There have been several reasons why I have chosen to do it this way.

The first one comes from my openSUSE background. Here it is required that you use macros in your spec file to denote system paths. For the configuration directory this macro is called %_sysconfdir and is typically set to /etc by the build system. You find this macro in the %files section in my spec file. Here you might also see that I'm using a macro for cmake, too. %cmake is also something you have to use for building a package, and it expands to a cmake call with many additional parameters. One of them is -DSYSCONF_INSTALL_DIR:PATH=%{_sysconfdir}. So I have chosen to use that variable in my cmake files. This guarantees to be consistent between building and packaging with no additional effort.

The second reason was that I did not get the handling described here working: Even ig CMAKE_INSTALL_PREFIX is set to /usr, the default for the CMAKE_INSTALL_SYSCONFDIR was /usr/etc and not /etc. Perhaps you can shed some light in here so that indeed /etc would get the default.

And as a third argument: I would never hardcode that. You know, there are some distros on the way with an immutable core system. Here a package must only own files in the immutable part of the OS. /etc is however mutable, as it contains files which are supposed to be edited by the administrator. Package provided configuration files go to /usr/lib or similar, and /etc will just contain administrator overrides. Even if I think that for an immutable OS OpenBoard is likely to be provided as Flatpak and not as a package: /etc is not and should not be carved in stone.

About widgets.md5sum and OpenBoard.css: There are reasons why these files might be edited by an administrator.

widgets.md5sum contains checksums and ids for widgets which can profit from the automatic upgrade mechanism. If somebody has a custom or third-party widget and wants to enable the automatic upgrade for that, they can add a line to this file.

OpenBoard.css defines some styling and can be adapted by the user. E.g. here OpenBoard-org#339 changing the css was proposed to solve an issue on special resolution displays.

Therefore I considered both files to be candidates for configuration files. I have placed the other files from resources/etc into /usr/share/openboard/template, as they are more or less just templates. I have even omitted two of them, as they are only referenced in dead code (old stuff for Adobe Flash).

[Vekhir]

So many system configurations to deal with, everyone does it slightly differently 😄 And hardcoding isn't a good idea, that's true.
The choice for widgets.md5sum and OpenBoard.css makes sense. I thought so at first, but then got doubts, so thanks for laying their purpose out so clearly.

With regard to GNUInstallDirs, I encountered the same issue. After reading the section for the millionth time, I realized that it talks about CMAKE_INSTALL_FULL_<dir>, notice the _FULL_ part...
Sure enough, replacing CMAKE_INSTALL_SYSCONFDIR with CMAKE_INSTALL_FULL_SYSCONFDIR turns etc into /etc. This probably works for OPENSUSE too.
I used this command in my PKGBUILD for the replacement:

sed -i "s/CMAKE_INSTALL_SYSCONFDIR/CMAKE_INSTALL_FULL_SYSCONFDIR/" CMakeLists.txt

CMAKE_INSTALL_SYSCONFDIR only occurs once.

[Vekhir]

More as a reminder than anything else, I want to talk about the version calculation for OpenBoard between cmake and qmake. The cmake computation is essentially the same as the one in qmake.
The VERSION_RC is related to Windows .rc files, not release candidates, so it's more useful to think of that as VERSION_WIN or something like that. As such, it is currently unused in cmake (which is Linux-only), but can be retained for easier future integration of Windows.
The second thing is that the versions in Openboard.pro and CMakeLists.txt need to be in sync with each other. E.g. currently, the cmake version has VERSION_TYPE set to r instead of rc. I don't think there is a practical way to combine those two, most approaches use a third file version.txt or something to read the version from. But the parsing would likely cause more effort than just updating the version in two places.
The OpenBoard version needs to be updated in both Openboard.pro and CMakeLists.txt.

[letsfindaway]

More as a reminder than anything else, I want to talk about the version calculation for OpenBoard between cmake and qmake.

This is really a point that needs harmonization!

The cmake computation is essentially the same as the one in qmake. The VERSION_RC is related to Windows .rc files, not release candidates, so it's more useful to think of that as VERSION_WIN or something like that. As such, it is currently unused in cmake (which is Linux-only), but can be retained for easier future integration of Windows.

I have removed it for the time being.

The second thing is that the versions in Openboard.pro and CMakeLists.txt need to be in sync with each other. E.g. currently, the cmake version has VERSION_TYPE set to r instead of rc. I don't think there is a practical way to combine those two, most approaches use a third file version.txt or something to read the version from.

Very good idea!

But the parsing would likely cause more effort than just updating the version in two places. The OpenBoard version needs to be updated in both Openboard.pro and CMakeLists.txt.

Parsing is not so difficult. I wrote version.txt as simple key=value file, which can immediately be included in the .pro file and added parsing to Version.cmake using regular expressions.

[Vekhir]

The update works nicely, thanks!

[jbruechert]

Hey, I hope this is the right place to ask.

Since 1.7.0 is now released and works on both Qt5 and Qt6, would you recommend updating community packages to Qt6 already, or are there major issues you are aware of?
For the flatpak builds both is easily possible.

[letsfindaway]

Hi Jonah,

I do my tests basically with Qt5. But the released MacOS version of OpenBoard compiled for the Apple ARM chips is using Qt6 and went through the testing at Geneva schools. The only thing mentioned in the release notes is "Some .mp4 videos can be read, some can't.".

When I look at the changes introduced in OpenBoard with Qt6, then most are just small API changes. The only bigger thing is the media framework, which looks quite different in Qt6 than in Qt5. This affects the media player (audio and video), but also the microphone input for the podcast recording. I did basic testing for those, but may be here are some more subtle bugs.

Baseline: There are no major issues with Qt6. From my "Bauchgefühl" it is good enough to base a community build also on Qt6, as the official and well tested MacOS build also uses that.

[jbruechert]

The flatpak build is now Qt6 based, so far there have been no complaints :)
Thanks for the help on deciding.