Community Component "Marketplace" & Separate Plugins

[jem-davies]

On this issue "Reconsider 1:1 fork" - a few ideas are raised:

• having Bento under a shared governance model @gregfurman
• a "marketplace" for plugins @sagikazarmark
• infrastructure that helps building custom binaries @sagikazarmark
• maintaining a repo of plugins before the license changed @nickchomey

Which all sounds good - but in a way is offered as an alternative to what we have with Bento at the current time. That it's a dilemma between having a "hard fork" and the above. But now I am currently thinking that this is a false dilemma - it could be possible to do both.

The vast majority of plugins in the Bento project will work with the Benthos engine - providing that you use the appropriate service package. This has lead to plugin authors to create two PRs on two projects with exactly the same code but with a different import. I know that I have seen @sapk & @aronchick do that ( there are others too ).

Perhaps it would be possible (if plugin authors wished) to use something like https://github.com/jem-davies/poc-benthos-service-shim which enables switching between the two service packages using go build tags. Write the code once - use build tags to control which engine you are building for.

Then if plugin authors do use this "service-shim" - they will create their own repo for the plugin - so users of both benthos & bento could import it. Maybe this is where the "marketplace" idea comes in - for discoverability. I have made an extremely crude github pages deployment for this, using the old bento doc site before the rebrand. https://jem-davies.github.io/poc-marketplace/ - Using the old doc site makes sense, because we will want to display the doc page. So you have the landing page - Component discovery page - and then the doc page for individual components. That would display additional info such as: repo url, engine compatibility, license info etc. ...

I am then thinking that we would then break up ./internal/impl into separate git repos - and list them in the marketplace. But we still provide a 'batteries included' binary of Bento - it's just the imports for the plugins will change.

There is an example of a plugin package that has been moved to a separate repo - and uses the shim here: https://github.com/jem-davies/poc-couchbase

[jem-davies]

POA would be:

• create a marketplace
• provide a "service-shim"
• separate Bento's ./internal/impl into separate git repos