Deprecate automatic creation of empty `__init__.py` files

[limdor]

Description of the problem / feature request:

Bazel is creating init.py files automatically. This is an issue in several situations (see bazel-contrib/rules_python#55).

This behavior can be suppressed using:
https://docs.bazel.build/versions/master/be/python.html#py_test.legacy_create_init

But in case that you have a large code base with a lot of python rules this is not a good solution. It would mean that this flag needs to be set in every single rule in case that you have a folder in the root of the workspace with the same name as a python package.
When this happens, it is not so easy to figure out what is going on. For modern Python developers init.py files are like source, these are not generated files. Thus, is not intuitive that the error could be because some init files are automatically generated.

Feature requests: what underlying problem are you trying to solve with this feature?

In a first step: There should be an option to decide the behavior of legacy_create_init globally.
In a second step: The default option should be not to generate init.py files by default.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Same like bazel-contrib/rules_python#55 or any setup that at the root of the workspace you have a folder called like a builtin python module (time, logging, platform, queue, etc.)

What operating system are you running Bazel on?

Ubuntu 18.04

What's the output of bazel info release?

release 0.22.0

[brandjon]

The end state shouldn't be a global option but rather removing the legacy create init behavior altogether. This might look like the following.

1. Go through an incompatible change migration to set the default value of legacy_create_init to false.

2. Then go through an incompatible change migration to force the value to false for everyone, effectively making the attribute a no-op.

3. Then go through an incompatible change migration to delete the redundant attribute.

[brandjon]

The first step of the above is implemented in @FaQA's PR #9271, which is in the process of being merged. It introduces an incompatible change flag tracked by #10076.

Steps 2 and 3 above can come after that. They can actually be merged into a single step. But that'll come down the line, since I don't anticipate flipping the first flag for a while. (We need to see how difficult the migration is, and if it requires new tooling, that'll be hard to prioritize.)

We can use this bug to track the overall deprecation effort.

[brandjon]

A concern came up in internal review of #9271: How will a user know that a failure at execution time (from missing __init__.py) is due to flipping this flag?

We already have some logic in the stub script to give users a more informative error message when a failure may be due to a different incompatible change. We can adopt a similar strategy for this change, so that when the incompatible flag is enabled, the exit code is 1, and the stderr matches ^(ModuleNotFoundError|ImportError): , we print an additional message linking to the incompatible change docs. (The second incompatible change, to remove legacy_create_init altogether, won't need any special diagnostic because it'll be a straightforward analysis-time error.)

The downside of this approach is that it means Bazel injects stderr spam into failing py_binarys. But since it would only trigger when legacy_create_init is set to the default value, auto, the user could opt out of the spam on a per-target basis by explicitly setting this attr to true or false -- which works until the second incompatible change to delete the attr. We'd remove the special message from the stub script once the first incompatible flag is flipped.