Inconsistent results with MSBuild and dotnet build

[xecrets]

Hi,

(Edit: See comments below, the actual difference is when building with MSBuild vs. dotnet build)

I have an annoying situation where builds on macOS and Windows produce slightly different output, causing source control to consider it a change... Both versions are "correct", but it's very annoying to have to revert changes all the time.

I have not completely analyzed it, but the differene is that some items are written in a different order on Windows and macOS respectively.

After a brief glance at the source code, a possible culprit could be the enumeration of the 'DocItems' dictionary. The order of the keys in the dictionary is unspecified, so it could certainly differ between platforms.

Has this behavior been noticed before? Does it seem like a reasonable cause for the divergence of the results?

Notably, the difference only occurs in one file, with one block of items moved on macOS compared to building on Windows. All other documentation generated appears stable across platoform builds. But I have a hard time finding other likely causes.

Comments or ideas from the community or the maintainer?

[Doraku]

Can't say I have ever seen this behavior but I have only run it on one OS 🤔 I guess the order is an undefined result because of the unordered dictionary as you said, although the result in memory should be deterministic as there is not randomization in there but could explain the different order on different OS (and why it's always the same on the same OS).
One method that comes to mind would be this one that is used in the ChildrenSection base type where you are observing the differences? But there is a OrderBy that should produce the same order everywhere. Do you have exemple of the section where you get different ordering so I can pinpoint where the issue occurs?

[xecrets]

Hi,

Actually it turns out I misidentified the issue. It's not running on different OS's, it's running from MSBuild or dotnet that causes the issue. I though I had eliminated that, but apparently I made a mistake when verifying. Anyway... It's not where it's built, it's how it's built.

A repo where this occurs is: https://github.com/xecrets/xecrets-slip39 .

If you cd down to the 'src' directory where the solution file is, you can build the project from a Visual Studio Developer command prompt like so:

msbuild /t:rebuild

This will leave everything in source control unmodified.

However, if you build it with dotnet instead, like this:

dotnet build

or even

dotnet msbuild

The file Xecrets.Slip39/docs/Xecrets.Slip39.Extensions.md is modified. One block of documentation, starting with <a name='Xecrets.Slip39.Extensions.GenerateShares(thisXecrets.Slip39.IShamirsSecretSharing,int,int,string)'></a> gets moved down a bit in the file.

From what I gather, the only obvious difference is how MSBuild is used and launched. In the first case, it's the MSBuild.exe that comes with Visual Studio, in the second case it's integrated into dotnet as a DLL. Maybe I'm barking up the wrong tree, and something entirely different is causing it, but something is! The produced documentation file is different depending on how the project is built!

(The reason I misidentified it as a platform issue is because on Windows I use Visual Studio / MSBuild, on macOS and Linux I use VS Code / dotnet. I thought I had eliminated that, but as mentioned, I made a mistake. So it's as above.)

[Doraku]

Interesting, I wonder if the task is not running via the old framework when using msbuild and the newer dotnet when using dotnet cmd 🤔, and from your doc page I guess the comparison between , and ) chars does not yield the same result between the two (if it's changing place with the Extensions.GenerateShares(this IShamirsSecretSharing, int, int, string, StringEncoding) overload?)? Maybe an issue with the default culture info between the two or something else. Thanks for the informations I'll try to look more into it but it probably won't be until the end of the week sorry.

[xecrets]

Yes, interesting.... Super if you'd have a look at it when you can spare the time. I'll try to look also a little bit and see what I can determine about the behavior.

[Doraku]

I had little time to look into it I managed to reproduce your issue with your repo and confirmed that the msbuild use net framework runtime while dotnet use the net runtime. I've looked around and found some people complaining of behavior change between the two, apparently because of this. I tried a quick fix that did not work but it seems I was on the right track. I'll try some more when I have time!

[xecrets]

Hi!

Sounds great. I've encountered the issue in another scenario as well, building from Rider on macOS/Apple silicon caused a different set of "movements".

behavior change between the two, apparently because of this.

Interesting. I was not aware of the sorting differences there. Is the output in fact sorted (I didn't find this when I did my very quick peek)? If it is, I guess that could cause problems. But the situation I discovered when building from Rider should not happen if that's the case, since I now have three different results. If the ICU/NLS diff is the root cause, that should only be two different scenarios, right?

I'm still more thinking it could be the undefined order of keys in a dictionary that's the root cause - but you know your code better! Your conclusion that in once case it's using .NET Framework and in the other .NET at least supports the theory that the behavior differs due to this. I do know that the implementation of GetHashCode() has changed between earlier implementations of .NET, and that it can differ on different architectures, and this is used in collections of various kinds. I seem to recall that dictionaries use a HashTable under the hood.

Anyway, it would be awesome if it got fixed. I am currently working around it by disabling documentation building in all but one environment, but that may cause me to forget to update the docs as I'm currently switching between a number of environments to build, test and debug my code. I'm running Rider and VS Code on macOS Apple silicon (ARM64), Visual Studio 2022 on Windows ARM64 and x64, VS Code on Ubuntu Linux on ARM64 and x64 and maybe some more as well ;-) .

There could possibly be some difference in the .NET version as well, I'm not sure if the build process actually runs on .NET 8 or .NET 9 in all cases. It should be .NET 9, but they are both installed in these environments. Normally .NET 9 should be used, but it depends.

[Doraku]

Sorry for the long delay, I managed to replicate the ordering issue with a simpler code. It should be fixed in the incoming 1.2.1 version. It seems the string comparers behave differently between net framework and net (core). The only one that produce the same result for , and ) is the ordinal. I cannot get differences between documentation produced by dotnet and msbuild anymore, don't hesitate if you encounter more issues 👍