allow `!machine` in x-machines?

[miekg]

Should we allow

services:
  web:
    image: nginx
    x-machines:
      - !machine-1

to mean "not on machine-1"?

Shouldn't be too hard to implement.

[miekg]

this also hooks in the labels discussion. Might be more generic to allow labels here, tbf machine names are already some kind of labels (without values)

[psviderski]

Compose spec has placement constraints: https://github.com/compose-spec/compose-spec/blob/main/deploy.md#placement. See relevant issue: #3

x-machines is just a shortcut we implemented first as

deploy:
  placement:
    constraints:
      - machine.name==machine-1

is quite cumbersome and it still doesn't support OR.

x-machines got us very far but I agree we might want a bit more advanced way to specify a set of machines, including support for labels when they're implemented.

Some kind of a super basic logical expression would be nice. Any good examples from relevant tools like k8s, AWS ECS?

[miekg]

k8s goes overboard with https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/, but Affinity and Anti-afinity based on labels might be enough.

Oring or Anding them might be useful, foro which k8s has operator (-ish). Might copy some of that.

[miekg]

ok, so k8s first thing was nodeSelector which does the same as x-machines, and does not allow for negative matching. For affinity (k8s 2nd thing) that use a dsl spread over yaml. I think it makes make more sense to use a mini-dsl to express this. This alslo allows you to use the same dsl on the command line, for listing, updating and what not.

With some tiny llm help, this is what I came up with, which doesn't look to bad. I don't like Notin, btw, prefer (!key in (a,b)).

DSL	K8s MatchExpression Operator
key == val	In ["val"]
key != val	NotIn ["val"]
key in (a, b)	In ["a", "b"]
key notin (a, b)	NotIn ["a", "b"]
!key in (a, b)	NotIn ["a", "b"] (flagged)
key > val	Gt val
key < val	Lt val
key	Exists
!key	DoesNotExist

So you get stuff like:

x-machines:
  selector:
   - "zone == us-east-1"
   - "!gpu == a100"
   - "instance-type in (m5, c5)"
   - "!disk-type in (ssd, nvme)"
   - "master"
   - "!worker"
   - "capacity > 16"

• Rules within a single array element are ANDed together. Elements are ORed in the final matchExpressions list.

Which we could do: !worker, master is ANDed, but split in two fields they are ORed.

TL;DR: this is node-affinity, node-anti-affinity, but in a DSL, not in YAML. And allows re-use in the cli for filtering. And the entire expression results in a set of machines where we deploy.

update

played a bit with an implementation and that code is super short. fwiw

[miekg]

doing it as templates might make more sense and doesn't add a dsl to compose:

x-machines:
  selector:
   - {{"zone" eq "us-east-1"}}
   - {{"gpu" ne "a100"}}
   - {{"instance-type" (in "m5 "c5")
   - {{"disk-type" not (in "ssd"  nvme")
   - {{"master"}}
   - {{not "worker"}}
   - {{"capacity" gt 16}}

you even get access to all text/template functions, the only one added would be in

update

templating sucks, "zone" does not resolv to true by itself so you need custom parse everything and even then it becomes ugly fast