doc: Zephyr out-of-tree module.yml examples

[gregshue]

This commit adds module.yaml examples which support the
zephyr directory structure beneath /zephyr,
and describes optional (out-of-tree) module scope.

Signed-off-by: Gregory Shue gregory.shue@legrand.us

[gregshue]

@carlescufi , please also review this (or invite a delegate).

[tejlmand]

Thanks for this. Good to get this described.

I think the out-of-tree term is a bit confusing, as the Zephyr manifest described modules are also out-of-tree, when comparing to the Zephyr repository, but all are still inside the Zephyr workspace.

[tejlmand]

This out-of-tree is no more, no less out of tree of the zephyr workspace when comparing with Zephyr manifest described modules.

Why not use the term: Zephyr extra modules as we already describe how those can be specified:
https://docs.zephyrproject.org/latest/guides/modules.html#integrate-modules-in-zephyr-build-system
and then describe that those extra modules can be provided by a custom / downstream manifest file ?

[gregshue]

@tejlmand As I recall, the Zephyr documentation consistently(?):

• Refers to the modules that the zephyr repository depends upon as external, and these modules have a modules/Kconfig.* file within the zephyr repository (due to limits of Kconfig)

• Refers to the modules that the zephyr repository does not depend upon as out of tree, and all configuration, source, and build instructions are in the external module itself.

I have been keeping the following questions in mind as I have wrestled with the concept and implementation of modules:

• The definition of 'module' is currently independent of 'manifest'. Does that matter? (N)

• IF the Zephyr Build System team decided it would be much easier if the build system discovered the zephyr repository in the same way it discovers modules, would the definition of 'module' still fit? (Y)

• IF the zephyr repository had an alternate manifest file which also included all downstream modules hosted at zephyrproject-rtos, do all the terms and definitions still fit? (?)

I really want the term 'module' == zephyr/module.yml. The mechanism is already independent of dependence from zephyr repository, location of config/glue code, hosting service, open source vs. commercial vs. private. I want the definition of the term to reflect that scope. This will avoid confusion about scope and will track capabilities as the module.yml schema is extended. I think this will greatly simplify communication now and in the future. It does mean we need distinct terms for each of the above sub-cases.

I have been trying to stick with the existing terms, but it may be time for them to be overhauled. Unfortunately that means some files get renamed. I would prefer to not use the phrase 'zephyr extra modules' so we don't imply an artificial and unnecessary coupling between the CMake variables and the amount of dependence the zephyr repository has on the module.

Perhaps we should adopt the following?

• 'upstream module' for those modules the zephyr repository depends upon
• 'downstream module' for those modules the zephyr repository does not depend upon
• 'Zephyr Project module' for those modules managed/maintained at zephyrproject-rtos

[jackrosenthal]

FWIW, our team uses zephyr modules without west entirely. So for us, module really does mean zephyr/module.yml.

[tejlmand]

@tejlmand As I recall, the Zephyr documentation consistently(?):

@gregshue Thanks for the answer, but i'm not addressing the module term.
What you describe as module is correct, what I'm addressing, is the term out-of-tree.

Refers to the modules that the zephyr repository depends upon as external, and these modules have a modules/Kconfig.* file within the zephyr repository (due to limits of Kconfig)

The reason for moving the Kconfig files back to Zephyr, was to avoid warnings: #16512.

Refers to the modules that the zephyr repository does not depend upon as out of tree

If you notice #16512, then those modules are optional, so you cannot say Zephyr depends upon the modules referenced in module/Kocnfig.*. Hence, saying that an out-of-tree module is a module Zephyr does not depend upon may lead to the false impression that Zephyr depends on all the other modules (which Zephyr doesn't, as some of them are optional).

Certain features of Zephyr or hw might depend on them, but if not using those features or hw, you can easily leave them out.
Example, if I don't need stm, then I can leave out https://github.com/zephyrproject-rtos/hal_stm32, or if not using Nordic, I can leave out: https://github.com/zephyrproject-rtos/hal_nordic.

Also, the build code of the external module are for now, still located in the external module.

I really want the term 'module' == zephyr/module.yml.

which is correct. You seem to have missed the intention of my comment.

I would prefer to not use the phrase 'zephyr extra modules' so we don't imply an artificial and unnecessary coupling between the CMake variables and the amount of dependence the zephyr repository has on the module.

So the fact that the guide describes:

The build system variable ZEPHYR_MODULES is a CMake list of absolute paths to the directories containing Zephyr modules.

means we cannot use the term Zephyr modules because it is also used by CMake ?

You seem to be under the impression that CMake uses the argument ZEPHYR_EXTRA_MODULES.
It doesn't, it simply passes on that piece of information to: https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/zephyr_module.py
here:

zephyr/cmake/zephyr_module.cmake

Lines 33 to 36 in e7df25c

	${PYTHON_EXECUTABLE} ${ZEPHYR_BASE}/scripts/zephyr_module.py
	${WEST_ARG}
	${ZEPHYR_MODULES_ARG}
	${ZEPHYR_EXTRA_MODULES_ARG}

and then zephyr_module.py processes the information in module.yml and determine if that module.yml contains additional Kconfig, CMake, SoC / Board / DTS roots to include in the build system.
Just like it uses west to fetch manifest modules.
I therefore think Zephyr extra modules is the closest term that actually describes your intention.

Maybe if we cleaned up the module description (as you are already in process of doing 👍 ), then we could describe:

• Zephyr Modules (The projects defined in zephyr/west.yml and containing a zephyr/module.yml file)
• Zephyr Extra Modules (Additional projects defined in a custom manifest file and containing a zephyr/module.yml file)

and then describe how to specify Zephyr Extra Modules in a custom / downstream manifest, or providing those with CMake.

Zephyr Module is not exclusive for CMake, so why should the term Zephyr Extra Module be that ?

• 'upstream module' for those modules the zephyr repository depends upon
• 'downstream module' for those modules the zephyr repository does not depend upon

I would refrain from using upstream / downstream as some users might be experiencing Zephyr for the first time when using a downstream Zephyr based SDK, like nRF Connect SDK.
In such cases it can become less clear what upstream / downstream refers to.

Finally, upstream module can still be optional, so I would not say that Zephyr depends on them.

[gregshue]

@tejlmand Thank you for your detailed feedback. I'm glad to read that my description of module is correct.

Please note that I didn't originate the term out-of-tree. It seems from your proposal that cleaning up usages and moving/renaming files/samples/etc is an acceptable result.

Actually, I was looking at the interface to the build system independent of the current implementation. Whether the build system currently implements logic around ZEPHYR_EXTRA_MODULES in CMake statements, python, ruby, or ksh does not matter to the users of the build system. Similarly, since the build system does not require west to be installed, the manifest concept (however integrators choose to manage it) and locations for optional west manifest files are critical.

Reading the current modules.rst documentation and looking into cmake/zephyr_module.cmake and scripts/zephyr_module.py, it seems ZEPHYR_EXTRA_MODULES is reserved for a specific use case described in modules.rst, while ZEPHYR_MODULES is the standard way of passing any/all module paths into the Zephyr build system. I think we need to not conflict with the ZEPHYR_EXTRA_MODULES use case either directly or through inference and avoid using 'extra' adjective elsewhere.

Here is a list of the various conditions for which we need (hopefully unambiguous) phrases:

1. 'Zephyr module'
   Any repository having zephyr/module.yml (agreeable)
2. 'Zephyr manifest module'
   Any Zephyr module holding one or more west manifest files.
3. 'Zephyr Project module' ?
   Any Zephyr module hosted at zephyrproject-rtos
4. 'Zephyr Project manifest module' ?
   Any Zephyr manifest module hosted at zephyrproject-rtos
5. ?
   Any Zephyr non-manifest module hosted at zephyrproject-rtos
6. 'Zephyr default module' ?
   Any Zephyr module in zephyr repository default manifest
7. ?
   Any Zephyr module with zephyr/manifest/Kconfig.<module>

I like "optional" for any module that doesn't have to be cloned to build the minimal Zephyr system (e.g. samples/hello_world for native_posix). Given that criteria, are there any modules which are NOT optional?

[gregshue]

Meaning, we still don't know if this work will need the ability to distinguish between the type of module, in order to load the files correctly.

By extension of the existing pattern, any metadata needed by the build system about a module is to exist in zephyr/module.yml. The schema for that will grow over time as we identify needed metadata, but the scope of the content is already defined.

Or how to handle the use case where a downstream project forks a Zephor hosted module, and then patch it.
Would users have to patch BOTH zephyr (with gluecode) and the module, or will it be enough to patch just the module, examples of such modules:
https://github.com/nrfconnect/sdk-mcuboot (making it a Zephyr module)
https://github.com/nrfconnect/sdk-mcumgr (just patches)

I have already experienced this when Nordic maintained a public fork of Zephyr (driving the need for my application repository to have alternate manifest files. The interfaces between the repositories doesn't move. If the issue is in the interface, then all repositories using that interface get patched. If the issue is not in the interface, then only the repositories with the issues get patched. This holds true no matter where the glue code is. In both cases the user will need want to create a clone of the repository and apply the patch to their own long-term branch rather than the upstream main development branch. When the upstream repository has a fix for the issue then the user chooses which implementation to use. (This is just standard-operating-procedure. :-))

Not sure. when moving build logic into Zephyr repository, there could be a need for distinguish between the type of modules, so that downstream users may still use their own fork of a Zephyr hosted module.
Either by patching the module, or by having code in their manifest repository having higher precedence than the Zephyr repository code.
Exactly what is needed / how to handle it / implement, is yet to be decided.

Once a user forks a zephyrproject-rtos repository they can do whatever they want. All we can do is provide recommendations. I have already been working against forks of all the Zephyr Project hosted repositories on other Git servers. I have already updated and privately patched the forks to work around issues. This helped me recognize the need for policies/guidance includes prefixes on branch names and tags (to avoid collisions).

So the fact that this repository may also function as a manifest repository seems irrelevant when describing the module functionality.

The difference is relevant to:

• Module release processes (which is now independent of zephyr)
• Zephyr Project documentation
• Zephyr Project CI pipeline (since the module build needs to use a different manifest file)
• Where issues are tracked(?)

The build system use ZEPHYR_MODULES or west list as a list of candidates to parse as Zephyr modules, besides that, there is no link to manifest or west.

There is no link to the manifest file or west, but there is still an upstream/downstream relationship between the repositories that affects how the downstream module is managed.

...
So the fact that this repository may also function as a Zephyr module seems irrelevant when describing the manifest functionality.

That is true with respect to the source build system. That is no longer true for document generation along with the other points mentioned above.

So why describe manifest and Zephyr module together ?
Keep them independent, cause they are independent.

When I created a downstream module I needed to develop it in isolation against a known, tracked version of Zephyr. This meant the downstream module development environment had one or more distinct manifest that needed to be tracked and maintained. (I actually simultaneously had three in the same branch: current Zephyr master, latest Zephyr release, prior Zephyr release.)

When I integrated this downstream module into an internal device, the prj.conf for the device was in another repository with its own superset manifest. This one is independent of the module.

...
but those Zephyr modules will still be in a Zephyr hosted manifest file.

You just changed from "the manifest" to "a manifest". Take it the next step and identify which manifest file in which repository. :-)

Today we only have zephyrproject-rtos/zephyr as manifest project.
With #28438 there will be one more, and in future, there could even be more, as example, an incubated manifest repository, for projects not fully allowed into Zephyr mainline.

Exactly. We will likely end up with a manifest-only repository to verify that the #28438 repository content works as a module with an external manifest. In addition, we may choose to move one or more of our existing upstream modules to be a downstream module (e.g. Civetweb).

BUT the phrase "not allowed into Zephyr mainline" is misleading. These will be repositories that are part of the Zephyr Project mainline functionality. They are purposefully kept separated from the zephyr (RTOS) repository to help verify the essential functionality of extensibility.

We may refer to those manifest repositories differently, but seen from Zephyr module perspective, they should all follow the same naming conventions and they may move from one manifest file to another, or be present in multiple.

Absolutely! The question here is how they are named. We have already evolved to have HAL modules begin with the hal_ prefix. Remember, the purpose of the naming convention is to aid automation and convey meaning to people. It may be useful for Zephyr Project CI automation to have a rule that says "All zpmm_* repositories are to be run separately with each manifest matching zephyr/manifests/zp_ci/*/west.yml"

Do we similarly want a distinct prefix for downstream modules? For example modules?

A Zephyr module is not required to have a Kconfig or CMakeLists.txt file, that is optional.

Oh! I missed that and assumed we needed to use empty files instead. 😆

When enhancing the Zephyr module handling, so that module glue code can be located outside the repo containing the module itself, then this should be cleaned up, in order to be consistent.

I have already demonstrated that and am actively depending on the Zephyr module handling currently supports glue code outside the zephyr repository. No enhancement of the build system is needed. (I did need west 0.8.01a though.) Ticket #28438 is primarily about enhancing the Zephyr Project CI coverage to verify the functionality is maintained for each of the extension hooks.

This seems to turn into a bigger discussion, than what I initially anticipated when addressing the out-of-tree term

The good news is the vast majority of the things we are discussing have a natural solution that follows the existing patterns. (Sometimes the existing patterns take some work to recognize though e.g. ZEPHYR_MODULES carries all types of modules, not just the upstream ones.)

Once we settle on module terms then the name of the repository for #28438 is determined. The rest follows the pattern of existing contribution processes and should flow smoothly and quickly.

[tejlmand]

Meaning, we still don't know if this work will need the ability to distinguish between the type of module, in order to load the files correctly.

By extension of the existing pattern, any metadata needed by the build system about a module is to exist in zephyr/module.yml. The schema for that will grow over time as we identify needed metadata, but the scope of the content is already defined.

I'm not talking about the zephyr/module.yml.
But even today, with #16512, the design of Zephyr modules actually broke partly.
Cause all the code in modules/Kconfig.* is not really Zephyr modukle glue code, it's just regular Kconfig code.
What still lacks is correct integration into Zephyr module handling.

There is no proper handling / safe guarding of whether a module is actually available.

Or how to handle the use case where a downstream project forks a Zephor hosted module, and then patch it.
Would users have to patch BOTH zephyr (with gluecode) and the module, or will it be enough to patch just the module, examples of such modules:
https://github.com/nrfconnect/sdk-mcuboot (making it a Zephyr module)
https://github.com/nrfconnect/sdk-mcumgr (just patches)

I have already experienced this when Nordic maintained a public fork of Zephyr (driving the need for my application repository to have alternate manifest files. The interfaces between the repositories doesn't move. If the issue is in the interface, then all repositories using that interface get patched. If the issue is not in the interface, then only the repositories with the issues get patched. This holds true no matter where the glue code is. In both cases the user will need want to create a clone of the repository and apply the patch to their own long-term branch rather than the upstream main development branch. When the upstream repository has a fix for the issue then the user chooses which implementation to use. (

This is just standard-operating-procedure. :-)

For the simple case, yes.
But let's say I have an older Zephyr, then I want to update a module to Module-latest, because it fixes some critical bug.
I cannot simply update the module, cause the glue code has changed, so now I need to decide if I want to also start patching Zephyr, because glue code changes are needed.
In this situation, it might be beneficial to be able to simply take the latest glue code and place it directly in the module, and even if outdated glue code is also available in Zephyr-old release, then the code in the module takes precedence.
Or maybe even be able to place glue code in alternative repo, to avoid patching zephyr project repo completely.

Not sure. when moving build logic into Zephyr repository, there could be a need for distinguish between the type of modules, so that downstream users may still use their own fork of a Zephyr hosted module.
Either by patching the module, or by having code in their manifest repository having higher precedence than the Zephyr repository code.
Exactly what is needed / how to handle it / implement, is yet to be decided.

Once a user forks a zephyrproject-rtos repository they can do whatever they want.

Yes, but we should try to create the infrastructure in such a way so user can hook into the system, without having to fork and patch.
If a module is identified as downstream hosted, then the build infrastructure could allow for looking up glue code in another repo.
That would be an example of where distiguishing between Zephyr hosted / non-hosted module could be valuable.

So the fact that this repository may also function as a manifest repository seems irrelevant when describing the module functionality.

The difference is relevant to:

• Module release processes (which is now independent of zephyr)

Not sure what you mean by Module release processes ?
Zephyr is not releasing modules, so are you refering to upmerging of Externally handled modules, such as mbedtls, openamp, etc. ?
If we talk about upmerges and hence PRs to a Zephyr hosted module, then it should be stated:

• All PRs to a Zephyr module must be accompanied by corresponding PRs to all Zephyr hosted manifest repositories that includes the Zephyr module, where revision is updated tp the form pull/<no>head.

That way it doesn't matter if the module is also a manifest repository.
If the module is used by another manifest repo, then it must have a corresponding PR, regardless whether it is also a manifest repo itself.

Zephyr Project documentation

Project documentation is generated from the manifest repository.
And if that manifest repository include docs from Zephyr module(s), then I see no problem in doing so.
I still fail to see why the fact that a module repo can also be a manifest repo is special.

Zephyr Project CI pipeline (since the module build needs to use a different manifest file)

That would be covered by the requirement to have a manifest update.

Where issues are tracked(?)

The manifest project where the issue was identified.
So if some finds an issue when using zephyr, the report the issue towards Zephyr, which then might spawn additional issues elsewhere.
Like this chain: #26114 --> OpenAMP/open-amp#213 and OpenAMP/libmetal#119
and then waiting for later upmerge.

With this approach, then a module doesn't need to care.
The manifest project where the issue was identified, is where the issue should be reported.
If multiple projects are affected, the issue can be cloned.
When a fix has been merged to the module, and manifest files in various projects has been updated to include latest version of the module, then the issue can be closed.
Just like today. I see no reason to change that, even if Zephyr starts to have multiple manifest repositories.

Similar to what is done at Nordic, where might be reported against our local Zephyr fork. If it is identified to also exist in Zephyr, then an issue is also created in Zephyr, and fixed there.

When I created a downstream module I needed to develop it in isolation against a known, tracked version of Zephyr. This meant the downstream module development environment had one or more distinct manifest that needed to be tracked and maintained. (I actually simultaneously had three in the same branch: current Zephyr master, latest Zephyr release, prior Zephyr release.)

When I integrated this downstream module into an internal device, the prj.conf for the device was in another repository with its own superset manifest. This one is independent of the module.

To me, this seems to be overcomplicating the initial work of a new module.
You could have used -DZEPHYR_EXTRA_MODULE=/path/to/my/new/module together with CMake or west.
https://docs.zephyrproject.org/latest/guides/modules.html#integrate-modules-in-zephyr-build-system

and then just switched between Zephyr vX.Y.Z / vO.P.Q/ master and tested the module.
I think that if the intention is to just test a given module at revision A with n-versions of Zephyr, then the manifest approach you just described is not the best way.
It could be you have found new use case that should be considered as improvement, but I still don't see why Zephyr module should be described together with manifest.

We have already evolved to have HAL modules begin with the hal_ prefix.

And such a naming pattern is good, cause it describe the purpose of the repo, that it contains hal code.
But prefixing the repo because it contains a manifest file is not good, cause that would mean repos would change name if at later point it should not hold a manifest file (or the other way around).

It may be useful for Zephyr Project CI automation to have a rule that says "All zpmm_* repositories are to be run separately with each manifest matching zephyr/manifests/zp_ci/*/west.yml"

I think that if we start to have so many repositories that we would benefit from automation here, then I'm concerned people will loose track of where to update manifest files.

When enhancing the Zephyr module handling, so that module glue code can be located outside the repo containing the module itself, then this should be cleaned up, in order to be consistent.

I have already demonstrated that and am actively depending on the Zephyr module handling currently supports glue code outside the zephyr repository. No enhancement of the build system is needed.

I assume you mean: outside the module repository ?
That code won't work as expected. Of course you can place CMake code anywhere in Zephyr you like, and have it to refer to the module repo, but that won't integrate the module correctly into Zephyr.
It will just be regular CMake code, so can we agree to reserve Zephyr module glue code term to what is actually part of the Zephyr module infrastructure ?
A couple of caveats:

• ZEPHYR_CURRENT_MODULE_DIR won't be set when processing the CMake code, so code inside a module depending on this variable will not work.
  Also the Zephyr extension command: zephyr_library_amend will not work.
• Same for ZEPHYR_CURRENT_CMAKE_DIR
• ZEPHYR_<module_name>_CMAKE_DIR will not be set. (This could be the reason you needed an empty CMakeLists.txt file, if other parts of code depended on this variable)
• The depends keyword will not work properly.

But the original comment I made regarding out-of-tree term has still not been addressed.

[gregshue]

@tejlmand Thanks for the insights. :-)

There is no proper handling / safe guarding of whether a module is actually available.

True, for modules that are upstream of zephyr. We already do have one for those that are downstream of zephyr (which is the main focus of this issue).

But let's say I have an older Zephyr, then I want to update a module to Module-latest, because it fixes some critical bug.
I cannot simply update the module, cause the glue code has changed, so now I need to decide if I want to also start patching Zephyr, because glue code changes are needed.

In the case above I'll assume the interface between the two repositories changed. Otherwise the repository(s) can be changed independently to address the issue(s). Addressing processes for managing changes in upstream modules is out-of-scope for this issue.

In this situation, it might be beneficial to be able to simply take the latest glue code and place it directly in the module, and even if outdated glue code is also available in Zephyr-old release, then the code in the module takes precedence.

How do we control the library precedence used by the Zephyr build system?

Yes, but we should try to create the infrastructure in such a way so user can hook into the system, without having to fork and patch.

Absolutely, which is why I am focusing on downstream modules.

If a module is identified as downstream hosted, then the build infrastructure could allow for looking up glue code in another repo.

There is no need to change the current build infrastructure. It is already finding the glue code in the downstream module.

Not sure what you mean by Module release processes ?
Zephyr is not releasing modules, so are you refering to upmerging of Externally handled modules, such as mbedtls, openamp, etc. ?

Tying the release of Zephyr to the release of Zephyr Project-managed downstream modules seems unnecessary, so I assumed Zephyr Project-managed downstream modules would eventually have their own release numbers and release processes.

If we talk about upmerges and hence PRs to a Zephyr hosted module, then it should be stated:

We are primarily talking about changes to Zephyr hosted downstream modules. In either case, your statements about needing to update mainfests is true, and is already addressed in many cases through the 'import' keyword in west manifest files.

To me, this seems to be overcomplicating the initial work of a new module.

When I created a module out of an existing open source project I found there were three things I cared about.

• Could I use it in another manifest?
• How do I develop this with the minimal dependencies?
• What is the easiest way to tie into the existing Zephyr build system that is consistent with the design?

The last question led me to avoid having other contributors to my module require special configurations with ZEPHYR_EXTRA_MODULES.

I have already demonstrated that and am actively depending on the Zephyr module handling currently supports glue code outside the zephyr repository. No enhancement of the build system is needed.

I assume you mean: outside the module repository ?

No. I have not tried to address upstream modules with glue code in the zephyr repository. That is out-of-scope for this issue.

Pulling this back to the scope of this issue/ticket, it seems that you prefer:

• using a term for downstream modules other than the already existing term "out-of-tree".
• not having a naming pattern for Zephyr Project repositories that indicates which are also manifest repositories

(I ignored the ZEPHYR_MODULES vs ZEPHYR_EXTRA_MODULES because all we can provide is a recommendation on usage.)

Have I missed anything else related to this ticket? What do you see as the pending questions?

[mbolivar-nordic]

This discussion is too long to read, but here are my two cents:

"Out of tree" has always, without exception, meant "outside the zephyr repository" to me when used in this context.

It has never had anything to do with whether or not there is an official blessing from the Zephyr Project itself for any functionality in the module.

So "out-of-tree modules" is simply redundant, and "in-tree module" would be an oxymoron.

[gregshue]

@mbolivar-nordic Thank you for your comments. You will want to read the discussion.

BTW, the zephyr repository subdirectory samples/application_development/out_of_tree_driver/hello_world_module is technically an in-tree module as you define it.

[mbolivar-nordic]

Nice examples!

I do have a few whinges about what I perceive as improper usage and editorializing, but I'm not putting a -1 on this to avoid blocking.

[mbolivar-nordic]

This discussion is too long to read, but here are my two cents:

"Out of tree" has always, without exception, meant "outside the zephyr repository" to me when used in this context.

It has never had anything to do with whether or not there is an official blessing from the Zephyr Project itself for any functionality in the module.

So "out-of-tree modules" is simply redundant, and "in-tree module" would be an oxymoron.

[tejlmand]

This discussion is too long to read, but here are my two cents:

@mbolivar-nordic No worries, I commented simply on the out-of-tree usage myself, and for reasons that are little unclear, even to me, turned into a longer discussion that had nothing to do with the out-of-tree term.
So regarding the out-of-tree term, you didn't miss anything.

[gregshue]

@tejlmand @mbolivar-nordic Agreed, No worries.

... for reasons that are little unclear, even to me, turned into a longer discussion that had nothing to do with the out-of-tree term.

Perhaps I can clear that up:

This thread is about choosing terms and definitions that clarify the breadth of extensibility already enabled by `zephyr/module.yml` and supported by the Zephyr build system. 

This may require a paradigm shift, as evidenced by the "But what about ...?" and "But that means ..." type of Q&A above. :-)

[gregshue]

I have extended the glossary with Zephyr module related terms and definitions to work against. Once we settle on the replacement phrase for 'out-of-tree' then I will update modules.rst.

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[dleach02]

This PR has gone quiet but seems like a useful PR to complete. I've removed the stale tag to prevent it from being closed but the author should consider resolving the conflicts and change requests to see if this can get merged... or close it if they don't want to pursue it

[gregshue]

@dleach02, thank you for noticing this. #28438 is working through alignment on the scope of a module, whether up-stream of Zephyr or down-stream of it. I expect to get back to this once a repository is created for that issue. (Any documentation edits before that are likely to change and lead to confusion.)

Do you think it is beneficial to make these documentations before then?

[tejlmand]

@gregshue thanks for the update, looks good.

A couple of minor suggestions when re-reading the updated docs.

[tejlmand]

imo we consider the whole project a Zephyr module when it contains a zephyr/module.yml file, and not just the file itself.

The file is just used to identify that this project is a Zephyr module, and the content of the file defines how it should be integrated, as example, is it a DTS, board, soc root ? should it be included in the build flow (CMakeLists/Kconfig) ? should it integrate into twister.

[gregshue]

@tejlmand I started with your impression, then asked if it still made sense for repositories like project_chip and bacnet-stack, which have substantial content for non-Zephyr platforms and environments. It seems the content not reachable by the Zephyr glue code is beyond any Zephyr-module policies. I just didn't want developers in other Open Source projects to think all of their repository needed to comply with Zephyr policies.

Would you still like this changed?

[tejlmand]

I think we need something in between. Cause I find the current description too limiting.

Personally, I would still refer to the project as a Zephyr module, although not all of the code in there may be used by Zephyr.
Just like https://github.com/zephyrproject-rtos/mbedtls is a Zephyr module, but as the code is developed by ARM, then the code itself will not comply with Zephyr policies.

[gregshue]

@tejlmand Suggestions welcome, and I can align with a more generic definition. Since we can imply a valid zephyr/module.yml from repos with both zephyr/CMakeLists.txt and zephyr/Kconfig (and/or deprecate it), how about:

"A repository with a valid zephyr/module.yml" ?

[tejlmand]

or it can be used in combination to west, where west knows about the manifest defined modules, but you have additional local modules or private repos (as example for testing or confidential code) that you want to include in a build.

Could be, as example, when working with the source for a closed source stack.

[gregshue]

Good to know. Please suggest replacement text if you feel that needs to be captured.

[tejlmand]

will do some thinking.

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.