xmake3 config 1k target so slow

[star-hengxing]

Xmake Version

dev

Operating System Version and Architecture

windows 10

Describe Bug

$ xrepo env -b "msvc,cmake,ninja" python test_performance.py
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.44.35215
1000 target
==================================================

=== Testing XMake ===
Running: xmake config

=== Testing CMake ===
Running: cmake .. -G Ninja

=== Testing Meson ===
Running: meson setup build_meson

==================================================
==================================================
--------------------------------------------------
XMake      87.78      87.78      87.78
CMake      2.73       2.73       2.73
Meson      3.89       3.89       3.89

CMake (2.73s)

Expected Behavior

N/A

Project Configuration

for i = 1, 1000 do
    target("test" .. i)
        set_kind("binary")
        add_files("src/main.cpp")
end

Additional Information and Error Logs

xmake 2.9.9

XMake      30.63      30.63      30.63
CMake      2.86       2.86       2.86
Meson      4.13       4.13       4.13

[waruqi]

修这个 #6659 时候，切到 jobgraph 去 config 支持调整 rule order 导致。

目前 jobgraph 里面 add_orders 以及为了支持 group ，目前的实现比较慢，后面可以看看优化下

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

When fixing this #6659, cut to jobgraph and config to support the adjustment of rule order.

Currently, add_orders and in order to support group in jobgraph, the current implementation is relatively slow. You can take a look at the optimization later.

[waruqi]

我先 revert 了之前的提交，用另外一种方式修复之前 #6659 的问题。

至于 jobgraph 问题，jobgraph 是 v3.0 新加的实现，目前还没怎么压测过，后面有空我会优化下，随着内置 rules 越来越多，现在每个 target 都会去生效几十个 rule ，每个 rule 都会有一个 job ，也就是说 如果有 1000 个 target ，每个 target 有 100 个 rule 的 on_config ，那么就要构造 100*1000 个 job 进 jobgraph 里。

然后 jobgraph 为了支持 group ，每次 add job/orders，都会去遍历更新 group 的 job，所以还翻了一个数量级导致。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I reverted the previous submission first and used another way to fix the previous #6659 issue.

As for the jobgraph issue, I will optimize it later when I have time. With more and more built-in rules, each target will take effect dozens of rules, and each rule will have a job. That is to say, if there are 1,000 targets and 100 rules on_config for each target, then 100*1,000 jobs must be constructed into the jobgraph.

Then, in order to support group, jobgraph will traverse the jobs that update group every time they add jobs/orders, so they also go through an order of magnitude.

[waruqi]

再试下，revert 后，应该正常了。后面会继续优化下。

#6780

ruki:xmake ruki$ xmake l ./tests/run.lua config_targets
> 1 test(s) found
>> [1/1]: testing tests/benchmarks/config_targets ...
config targets/1k: xmake: 2372 ms
config targets/1k: cmake/ninja: 2187 ms
config targets/1k: cmake/makefile: 6229 ms
config targets/1k: meson: 3484 ms
> 1 test(s) succeed

[waruqi]

windows 下 config 慢是因为一些 windows 相关的 rule/config 里去检测了 link

但每次检测 会根据不同 target 重新检测，缓存没生效导致重复耗时。

warning: the config script of rule(platform.windows.manifest) for target(test1) takes too long time: 156 ms!
warning: the config script of rule(platform.windows.manifest) for target(test145) takes too long time: 110 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test147) takes too long time: 109 ms!
warning: the config script of rule(platform.windows.manifest) for target(test186) takes too long time: 125 ms!
warning: the config script of rule(platform.windows.manifest) for target(test192) takes too long time: 109 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test210) takes too long time: 140 ms!
warning: the config script of rule(platform.windows.manifest) for target(test23) takes too long time: 109 ms!
warning: the config script of rule(platform.windows.manifest) for target(test231) takes too long time: 110 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test233) takes too long time: 110 ms!
warning: the config script of rule(platform.windows.manifest) for target(test25) takes too long time: 141 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test251) takes too long time: 110 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test254) takes too long time: 125 ms!
warning: the config script of rule(platform.windows.manifest) for target(test258) takes too long time: 140 ms!
warning: the config script of rule(platform.windows.manifest) for target(test270) takes too long time: 109 ms!
warning: the config script of rule(platform.windows.manifest) for target(test285) takes too long time: 110 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test29) takes too long time: 140 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test290) takes too long time: 109 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test292) takes too long time: 125 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test293) takes too long time: 125 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test294) takes too long time: 109 ms!
warning: the config script of rule(utils.compiler.runtime) for target(test295) takes too long time: 141 ms!
warning: the config script of rule(platform.windows.manifest) for target(test297) takes too long time: 110 ms!
warning: the config script of rule(platform.windows.manifest) for target(test298) takes too long time: 125 ms!
warning: the config script of rule(platform.windows.manifest) for target(test299) takes too long time: 110 ms!
warning: the config script of rule(platform.windows.manifest) for target(test3) takes too long time: 156 ms!

[waruqi]

是由于 toolchain 的检测后更新的配置，每个 target 都会去 flush cache 到文件，在 win 上 io 耗时很明显。

目前我改成了，全部检测完，仅仅到 config/build 结束时候，才去一次性 flush toolchain cache，效果还是很明显的，目前测下来 xmake config 以及是最快的了。

后面会尝试继续优化下 jobgraph 和 vs 等 toolchain 检测的耗时，不过会开个新的 pr。

macOS

ruki:xmake ruki$ xrepo env -b "cmake,ninja,meson" xmake l ./tests/run.lua config_targets
> 1 test(s) found
>> [1/1]: testing tests/benchmarks/config_targets ...
config targets/1k: xmake: 2159 ms
config targets/1k: cmake/default: 5883 ms
config targets/1k: cmake/ninja: 2222 ms
config targets/1k: cmake/makefile: 6097 ms
config targets/1k: meson: 3524 ms
> 1 test(s) succeed

Windows

PS D:\projects\personal\xmake> xrepo env -b "msvc,cmake,ninja,meson" xmake l .\tests\run.lua config_targets
> 1 test(s) found
>> [1/1]: testing tests\benchmarks\config_targets ...
config targets/1k: xmake: 4047 ms
config targets/1k: cmake/default: 22907 ms
config targets/1k: cmake/ninja: 5859 ms
config targets/1k: meson: 7469 ms
> 1 test(s) succeed

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

This is due to the updated configuration of toolchain after detection. Each target will flush cache to the file. It is obvious that io time-consuming on win.

At present, I have changed it to it. After all the tests are completed, I only go to the flush toolchain cache at one time until the end of config/build. The effect is still very obvious. At present, xmake config and it is the fastest.

Later, I will try to continue to optimize the time-consuming detection of toolchain such as jobgraph and vs, but a new PR will be opened.

macOS

ruki:xmake ruki$ xrepo env -b "cmake,ninja,meson" xmake l ./tests/run.lua config_targets
> 1 test(s) found
>> [1/1]: testing tests/benchmarks/config_targets ...
config targets/1k: xmake: 2159 ms
config targets/1k: cmake/default: 5883 ms
config targets/1k: cmake/ninja: 2222 ms
config targets/1k: cmake/makefile: 6097 ms
config targets/1k: meson: 3524 ms
> 1 test(s) succeed

Windows

PS D:\projects\personal\xmake> xrepo env -b "msvc,cmake,ninja,meson" xmake l .\tests\run.lua config_targets
> 1 test(s) found
>> [1/1]: testing tests\benchmarks\config_targets ...
config targets/1k: xmake: 4047 ms
config targets/1k: cmake/default: 22907 ms
config targets/1k: cmake/ninja: 5859 ms
config targets/1k: meson: 7469 ms
> 1 test(s) succeed

[waruqi]

Build targets (30 targets)

macOS

ruki:xmake ruki$ xrepo env -b "cmake,ninja,meson" xmake l ./tests/run.lua build_targets
> 1 test(s) found
>> [1/1]: testing tests/benchmarks/build_targets ...
build targets/30: xmake: 4690 ms
build targets/30: cmake/default: 31461 ms
build targets/30: cmake/ninja: 7490 ms
build targets/30: cmake/makefile: 32648 ms
build targets/30: meson: 16120 ms
> 1 test(s) succeed

Windows

PS D:\projects\personal\xmake> xrepo env -b "msvc,cmake,ninja,meson" xmake l .\tests\run.lua build_targets
> 1 test(s) found
>> [1/1]: testing tests\benchmarks\build_targets ...
build targets/30: xmake: 7875 ms
build targets/30: cmake/default: 77016 ms
build targets/30: cmake/ninja: 15594 ms
build targets/30: meson: 15016 ms
> 1 test(s) succeed