Thunderbird & Home Manger an ill-fated alliance.

[AliSot2000]

Hey

Two disclaimers:

1. I'm rather new to nix. I might have missed something absolutely obvious.
2. I'm not far enough down the nix rabbit hole where I just patch whatever I'm describing here.

As far as I understand, Home Manager excels at dropping files in place that are usually read only (even conditionally populating these files, so based on hostname [...] you configure your system differently. A good use case I'd imagine would be auto scaling infrastructure and easily configuring them with nix). Similar to NixOS which takes /etc under its umbrella, Home Manager attempts to do the same for all config files for the user's home. And any program or service that reads these configurations but never writes to them, is perfectly compatible with this setup. In the system environment, you rarely encounter applications that expose a config editor with which you edit the actual files in /etc. An example for one that does would be timeshift.

However in user land, there's a bunch of different kinds of programs. We have the KISS programs like ssh or bash that read the ~/.bashrc or ~/.ssh/config but that don't really write to themselves (well discounting known_hosts). I'm sure there's more, I couldn't be bothered to compile an extensive list of these programs. Then we have the programs that allow you to write to config (I'm thinking of something like git: git config set user.name AliSot2000 which would write to ~/.config/git/config.

Similarly, we could also write to these config files by doing something like vim ~/.config/git/config. And again, this is rather close to vim nix/myconfig.nix so not that much of a hassle.

But Thunderbird is extra annoying.

1. There's a sort of deadlock when you use profiles. You have the config option StartWithLastProfile=ProfileNumberwhich is a result of setting a Default=1 in ~/.thunderbird/profiles.ini. As this file is ro in home-manager-files, I cannot open any other profile than the default profile. Even disabling use this profile for next start is not something I can disable because the file is read only. There are two problems here. A) The Home Manager config options don't correctly map to the capabilities of Thunderbird. B) Home manager is enforcing ro on a file which subsequently causes crashes of Thunderbird.
2. Also account settings are locked. (From home-manager-files,ro). So basically the entire Account Settings tab is useless in Thunderbird because your changes cannot be saved to disk. Interestingly enough, this does not cause Thunderbird to crash. Not even emit an error. The UI updates the changes but the file obviously cannot.
3. Also credentials don't work properly. As can be seen in this thread, the option accounts.email.accounts.<name>.imap.authentication and accounts.email.accounts.<name>.smpt.authentication were added but not mapped for Thunderbird. To map these options you do this (thanks to LLMs who helped me with this)

accounts.email.accounts."outlook-account".thunderbird.settings = id : {
    "mail.server.server_${id}.authMethod" = 10;  
    "mail.smtpserver.smtp_${id}.authMethod" = 10; 
};

3. Continued: Additionally the way Home Manager is set up at the moment, authentication is also wonky at best. Plain Password works but nothing else really. You are relying on Thunderbird realizing "I don't have credentials for account X" and then prompting you with the password prompt or the OAuth2 flow. But also this is not certain. In my testing, the OAuth2 flow was only started for one of three mail accounts. And the other two accounts were stuck with:

The current operation on 'Inbox' did not succeed. The mail server for account ethz-main responded: Command Error. 12.

What we can observe in the case of Thunderbird: Having Home Manager watch over these files and also setting them with ro seems to cause more issues than it solves.

Any changes to the settings need to be done via the nix config which is lacking in the Thunderbird settings. Thanks to ChatGPT et al I was able to do that quickly but you need to know the config structure of the user.js file for Home Manager to its job effectively. I couldn't find a first party source for documentation of the users.js file. There's a lot of documentation but by what I can tell, it's reverse engineered which makes it a lot less reliable than a first party source with documentation.
Changes from Thunderbird are block. This causes both application crashes as for the profiles and it causes broken UIs with the state of the UI being out of sync with the config and possible knock-on effects that I didn't find because I didn't stick around long enough.

All I think there are three options for how one could handle this issue:

1. Remove Thunderbird from Home Manager. It is almost unusable when it comes to settings and updating them is a non trivial task. The task of changing settings is a very unreliable tasks as knowledge of setting changes cannot be source from the first party but one is forced to either reverse engineer it yourself or you need to wait for third parties to reverse engineer it. It is not very good experience for new comers (I've wasted all my evenings for a week to find out this integration is just broken) and it very much a non-trivial task to untangle Home Manager, Thunderbird and any errors you made yourself.
2. Home Manager as a bootstrapping assistant. Have Home Manager only work as a one-shot configurator. It creates the config files, copies them into the target directory and marks the directory as "done". Any subsequent changes to the Home Manager config will be written to home-manager-files but the actual Thunderbird config is left untouched. I think this is a rather sensible approach. Blocking any updates to the files in home-manager-files seems to be going to far, as it prevents you from updating your config for future / other systems. And mindlessly writing to the config also seems sketchy as the writes by Home Manager might break the state that Thunderbird created.
3. Full Home Manager integration to Thunderbird with rw. This like the most invasive and most complex endeavor: On one hand, add the ability to expose files in rw to applications to allow Thunderbird to write to its config files. And on the other there's a big development effort needed to be able to fully capture the state of the Thunderbird config, attempt to inject the changes or abort the update in case the changes would break the config as it is. So things like adding an account would work but deleting for example an account that in Home Manager is only responsible for email but in Thunderbird is now also tied to a calendar would be stopped. Or alternatively, allow Home Manager to distill the full config from the config files present in ~/.thunderbird/. Also a very hard task.

I added my suggestions in order of work that it requires to implement them. I personally see option 1 as the most realistic.

Option 2 is feasible, but requires the accounts and Thunderbird file to be updated for that.

Option 3 prohibitively time consuming. Dealing parallel edits in a config from different processes that don't know the other exists sounds like a research question that has conferences dedicated to it, not a challenge that's just kinda solved on the side of a OSS project.

But both option 2 and 3 require a lot broader discussion inside Home Manager and maybe also NixOS. How do we deal with config files that another program needs to write to (e.g. Timeshift -> NixOS, ..., Thunderbird -> home manger, ...)?
Do we implement a boot strapping logic and create a different class of config for these type of programs? Or do we attempt to capture the state and update it without creating conflicts.

I'm admittedly at a loss for what's a good solution. I've already spent about a week on this project and I don't feel like spending another, whipping up a bootstrapping system only for it to be rejected, scrapped or deferred because it does not fit with the vision of the nix community as a whole.

So yeah. I'm here for - well this issue - but also to spark and have a discussion.

Best

AliSot20000