Skip to content

[feature request] Support Compose models #367

@mathieu-benoit

Description

@mathieu-benoit

Docker Compose now supports at the root level models: https://docs.docker.com/ai/compose/models-and-compose/.

Basic example:

services:
  chat-app:
    image: my-chat-app
    models:
      - llm

models:
  llm:
    model: ai/smollm2

It seems that the goal will be to add models support in the provisioners: https://github.com/score-spec/score-compose/blob/main/internal/provisioners/core.go#L85.

Open question: for patch-templates too? Not sure...

As a reference, today we can do that via a service's provider like illustrated below, but this will be deprecated soon:
https://github.com/score-spec/community-provisioners/blob/main/llm-model/score-compose/10-dmr-llm-model-via-service-provider.provisioners.yaml

Metadata

Metadata

Assignees

No one assigned

    Labels

    dockerPull requests that update Docker codeenhancementNew feature or requestgoPull requests that update Go codegood first issueGood for newcomershelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions