Design PipelineResource extensibility

[bobcatfish]

Expected Behavior

We should have clear extension points in the Pipeline CRD system. These points should be well documented, clear, and tested.

Ideally it will be possible to extend the system without having to modify the existing controller binaries or CRD definitions.

Actual Behavior

At the moment, if a user wanted to extend the Pipeline they have these options:

1. They want to create their own kind of Resource (e.g. they use something like mercurial, but we currently only support git) = They have to modify and re-deploy the TaskRun controller @_@ (and maybe the PipelineRun controller also)
2. They want to create their own kind of Task = Design alternative Task Implementations inside a pipeline #215 + ducktyping should allow for alternative Task implementation
3. They want to run xyz specific logic = They can put any logic they want inside a task

In the worst case, the last option should allow a user to do pretty much anything they want, outside of the functionality provided by this system.

Additional Info

n/a

[bobcatfish]

Here is one proposal, which I am unreasonably attached to:

Similar to how consourse uses Resources as an extension point, we could also use Resources as an extension point.

1. We add a Resource controller, which only pays attention to Resource types it knows about and ignores the rest
2. When the Resource controller sees a Resource it knows about, it will make that Resource available via a PVC (e.g. if it's a Git resource check out the repo into a volume mount) or via some string data in the Status of the resource (e.g. for an image we just need the url + digest)
3. If a user wants to add their own kind of Resource, they can add a controller that knows how to handle that Resource type
4. When the Resource is used as an output, maybe this updated via some fields in the spec, which the controller can react to?

Cons:

• This interferes with the most appealing option for Ability to use same Pipeline with different Resources #200, which is to stop using CRDs for Resources
• This requires the system we deploy to support PVCs, not sure if this is a problem or not, have been trying with @aaron-prindle to avoid solutions to Explore moving away from init containers #224 that require PVC

[abayer]

Seems reasonable at first glance - I'm putting together some thoughts on extensibility based on some design work here (i.e., we're establishing what we want to be able to accomplish to do something like automatic quality gates on Tasks), and will try to get that written up to figure out how that would fit in with this approach.

[nader-ziada]

We will probably need to have a PVC anyways to share outputs from one taskRun (in a pod) with another taskRun in the same pipeline (in a different pod)

[cppforlife]

This requires the system we deploy to support PVCs

last time i was playing with azure's kubernetes service attaching volume to a node was taking ~2min whereas on gke it was only 10s. i think if persistent volumes are used then we should figure out how to deal with long attachment times (or how to avoid them entirely if that's possible).

[tejal29]

@pivotal-nader-ziada would downloading the output from a result store be a viable option ?

[nader-ziada]

@tejal29 you mean something like gcs or s3 for sharing artifacts between tasks? yes, this would be an option, but its one extra thing users have to setup and have ready. Based on @cppforlife comment and my own investigations, I want to avoid using PVC, but still investigating best options to discuss with everyone

[abayer]

Something worth discussing - what exactly do we want the extensions to be able to do? Some things definitely map directly to Resources - different SCM systems, different artifact storage, etc. But is that actually the only thing we want to be able to extend? For example, there's notifications (i.e., #49) - obviously, we can't hardcode in every notification system someone might want to use, so we need there to be a way to have extensions provide additional notification systems. But are those Resources?

[nader-ziada]

I think of resources as the input and outputs of the task, so notifications can be expressed as output of the task. If we design the extension point with enough flexibility, the resource should be responsible for taking the result of the task and doing what it wants with it including sending it to slack for example.

[abayer]

Ok, so in cases like notifications, we wouldn't be using PVC, right?

[nader-ziada]

Yes, we would not be using PVC for the notifications, we are trying to minimize any use of PVC in general for performance reasons

[abayer]

It would be helpful to lay out some more examples of what we want to be extensible in the first place - notification mechanisms and SCM checkout processes are pretty clear to me, but I'm not sure what else is being considered here?