Feedback on --workspace

[davidmorgan]

This is a place to put feedback on the new --workspace flag.

I'd like to hear

• if it works for you! If so what seems to work well about it, e.g. speed, simplicity; maybe give an idea how many packages you have / how big they are
• if it is not doing something you expect it to do, or something it does is surprising
• if you can get it to break / give wrong output
• any suggestions for related features or improvements; for example right now it only works for build and watch, how high priority is adding support for serve?

[rrousselGit]

One thing that's unclear to me:

Does it solve ordering issues?
Say a package A and B use codegen, where B depends on A. Does build_runner correctly wait for A to complete before B is ran?

I've tried it only a limited number of times, so it's not super clear to me yet.

[davidmorgan]

A very good question :)

No, it doesn't order by package: rather it runs as if it's one big package.

So builders will not see their own output in a dep package, just like they don't see their own output in the same package.

Builders will see the output of other builders that run before them, and not of builders that run after them, just as in one package.

I figured that this would work for most use cases, since builders are usually already dealing with ordering constraints in a way that works in a single package. But it does mean there is a subtle difference between doing a workspace build and doing a package-by-package build within the workspace.

If it turns out to be needed there could be extra configuration possible here. Automatically building package by package is going to get awkward if there are circular dependencies between packages :)

[rrousselGit]

So yesn't :)

I'll keep playing around and see if that's good enough for my uses. Thanks for the quick reply!

[rrousselGit]

I'll keep looking into it, but at a glance it seems significantly slower than running build_runner on all the related folders.

Probably because of multithreading?

[rrousselGit]

I think my comparison was a little bit unfair.

TL;DR
Parallel running, but only on a limited set of packages.
--workspace ended-up spending lots of time on other packages in the workspace, because "codegen maybe needs a rerun".
But I knew for sure that the output of those packages couldn't change, so I didn't even bother running build_runner there.

[rrousselGit]

Maybe a --only path path2 path3 flag could be helpful? As in, run build_runner only on those paths.

So we could do:

dart run build_runner watch -d --workspace --only packages/a  --only packages/b

We could also support folders within a package:

dart run build_runner watch -d --workspace --only lib/src/feature_a

[rrousselGit]

Ah wait that might already exist: --build-filter

I'll try using that and see if that improves my experience

edit had no success trying to use the flag. It's not very intuitive to use or not doing what I hope it'd do.

[davidmorgan]

Hmmmm it's supposed to be the case, particularly with watch since it reuses analysis results, that those unchanged folders are fast.

Is it a public workspace I can try running the same thing on? I'd be interested to see what's slowing it down.

Thanks.

[rrousselGit]

Riverpod. Specifically on the devtool branch
The issue is that packages/riverpod depends on a code-generator. And the rest of the repo depends on riverpod.
So changing said code-generator reapplies it to the whole repo I think

[homeserve-lsaudon]

I have this error when i launch dart run build_runner build --workspace

Running build hooks...Running build hooks...build_runner
W The --workspace flag is experimental and subject to change based on feedback. Consider adding
  your own feedback at: https://github.com/dart-lang/build/discussions/4349
Unhandled exception:
InvalidOutputException: app|router.g.dart
Tried to delete from package not in the build. Packages in the build are: list of packages
#0      BuildPackages.pathFor (package:build_runner/src/build_plan/build_packages.dart:228:11)
#1      ReaderWriter._pathFor (package:build_runner/src/io/reader_writer.dart:75:25)
#2      ReaderWriter.delete.<anonymous closure> (package:build_runner/src/io/reader_writer.dart:182:20)
#3      TimedActivities._run (package:build_runner/src/logging/timed_activities.dart:123:22)
#4      TimedActivity.run (package:build_runner/src/logging/timed_activities.dart:57:58)
#5      ReaderWriter.delete (package:build_runner/src/io/reader_writer.dart:180:25)
#6      BuildPlan.deleteFilesAndFolders (package:build_runner/src/build_plan/build_plan.dart:360:35)
<asynchronous suspension>
#7      BuildCommand.build (package:build_runner/src/commands/build_command.dart:58:5)
<asynchronous suspension>
#8      BuildCommand._run (package:build_runner/src/commands/build_command.dart:34:20)
<asynchronous suspension>
#9      BuildRunner._runOrThrow (package:build_runner/src/build_runner.dart:164:12)
<asynchronous suspension>
#10     BuildRunner.run (package:build_runner/src/build_runner.dart:47:14)
<asynchronous suspension>
#11     ChildProcess.run (package:build_runner/src/bootstrap/processes.dart:207:9)
<asynchronous suspension>
#12     main (file:///.dart_tool/build/entrypoint/build.dart:50:19)
<asynchronous suspension>

[davidmorgan]

@homeserve-lsaudon thanks for reporting, I filed an issue to follow up #4359 with some questions for you, please take a look and hopefully we can figure out what's going on :) thanks!

[davidmorgan]

Fix released in 2.11.1

[martin-ideeri]

First of all - I'd like to say thanks, centralizing build_runner at the workspace level is huge for us, we have multiple apps and packages and relying on melos to spawn n instances of build_runner has proven quite unreliable at times.

From initial testing, it seems to work out of the box without any hiccups, and for us at least it performs better than the "one invoke per package" setup we had.
I'm switching our non-vital CI to use it - will report if something breaks but so far this looks very, very good !

[martin-ideeri]

For some context in terms of codebase size, we have about ~150 generated files, using freezed, json_serializable and riverpod

[martin-ideeri]

Quick follow-up question : can we expect dart run build_runner clean to work for the whole workspace ? Asking since this sub-command does not have a --workspace flag

[davidmorgan]

Oh, good question, thanks. I forgot about clean.

Actually all clean does right now is delete .dart_tool/build if it's present.

When using --workspace that folder is placed in the workspace root.

So dart run build_runner clean will do what you want if you run it in the workspace root.

[martin-ideeri]

Perfect, thanks :)

[ChaserVasya]

FYI some generators are not ready for --workspace

https://codeberg.org/Tienisto/slang/issues/348
FlutterGen/flutter_gen#731

[davidmorgan]

Thanks! Subscribed to / commented on the GitHub issue.