Compiled methods: keep the source

[privat]

Losing source on uninstalled methods causes a lot of problems.

When the source code of a method is absent but required (for a lot of things, including transformation or debugging), the disassembler is invoked, and it has the following (anti)-features: 1. lose information (names of temps, for instance), 2. do not always give a source with equivalent AST (causing issue when associating AST node with pc), 3. Sometimes it just gives garbage.

Despite these issues, the lack of source also causes potential different behaviors on installed and uninstalled methods.
Moreover, most tests that rely on compiled methods never install them (because installing has system-wide implications).

There is a compiler option optionEmbeddSources that is automatically used it on faulty source code and scripts. If present, the compiler encodes the source code at the end of the compiled method (quite ugly, but is works). This was done because the decompiler cannot work on scripts or on faulty methods. But has the drawback of increasing the size of the method object by A LOT. It was considered acceptable because scripts and faulty methods are expected to be short living and garbage collected soon enough.

Anyway, this added another hack in the presence or not of source code, and a lot of confusion.

The situation what painful enough when working on the AST translators (cf #13352) that I propose the current PR:

• Source code is always preserved as a CompiledMethod property #source.
• Source code is not encoded, so if the existing string is still around and referenced, it does not add any memory cost.
• When installed in a class, the property #source is dropped (and will eventually be garbage collected).
• Previous clients of optionEmbeddSources are removed, because it is now unneeded.

The option optionEmbeddSources is still present. Maybe we can deprecate it.

Note: The commit 'TOREMOVE' is intended to FAIL on tests that still need source but cannot find it. I want to see the result of the CI. I will remove it after.

[privat]

Hum, disassembler is required at bootstrap. This is weird, but out of scope of the PR.

CompiledMethod>>codeForNoSource
CompiledMethod>>sourceCode
CompiledMethod>>getSourceReplacingSelectorWith:
TaCompositionElement(TaAbstractComposition)>>getSourceCodeOf:usingSelector:
TaCompositionElement(TaAbstractComposition)>>copyMethod:into:replacing:
[...]

[privat]

I checked the image size to see if there is some massive unexpected #source leak

• Pharo12 Image file (93f8afa base of the PR): 63.5MB
• PR image file (24d1228): 59.8MB ... that is lower? Nice! (unexpected but nice!)
• Source (both) 40.7 Mo (the source diff is negligible +67/-32 lines)

[MarcusDenker]

Nice!

The properties implementation for CompiledMethod is very arcane, it allocates an AdditionalMethodState Object, we should check if removing the last propery removes this. (memory use seems to imply that this is done ?)

[privat]

Yes, I looked at AdditionalMethodState and it is basically an array of associations with extra steps. Adding and removing just replace it with larger or smaller array.
It could be improved but I think it's out of scope of the PR

[MarcusDenker]

So I guess that all compiledMethods then have an empty AdditionalMethodState.

I am travelling so and internet is so bad that I can not download the build artefact.

[privat]

So I guess that all compiledMethods then have an empty AdditionalMethodState.

Good remark.
It appears that when you add a method property to a method without one, a new AdditionalMethodState is created, and takes the place of the selector (at the penultimate literal slot).
But if you remove the last property of a method, then an empty AdditionalMethodState remain.
We could replace it back with the selector. I did it in another PR #13386.