Set inequality syntax suggestion

[odow]

See #3769 (comment)

Problem

Consider:

model = Model();
@variable(model, x[1:2, 1:2])
@constraint(model, x >= 0)

Does this mean:

@constraint(model, x >= 0, Nonnegatives())

or

@constraint(model, x >= 0, PSDCone())

Because of this ambiguity, we currently disallow the x >= 0 syntax for matrix sets.

Suggestion

@LebedevRI suggests that we introduce the ⪰ operator with the following behavior:

@constraint(model, x ⪰ y, set)
# lowers to
@constraint(model, x - y in set)

Examples

@constraint(model, x >= 0)
# lowers to
@constraint(model, x ⪰ 0, Nonnegatives())

@constraint(model, x <= 0)
# lowers to
@constraint(model, x ⪰ 0, Nonpositives())

@constraint(model, x ⪰ 0)   
# ERROR: missing a set that defines the partial order

@constraint(model, x >= 0, set)  
# lowers to
@constraint(model, x ⪰ 0, set)

Tradeoffs

The biggest pro is that it clarifies the distinction between ≥ and ⪰.

This is also the biggest con: I worry that users will not make the distinction, and people will write x >= 0 expecting that to be a PSD constraint when it is really a nonnegative constraint.

Currenntly, they need to opt-in by passing a set to avoid the ambiguity.

[LebedevRI]

@odow thank you for posting this for disscussion!

My main concern is syntax divergence. With the current behavior,
given current JuliaLang/julia#3766 + JuliaLang/julia#3769, while @constraint(model, x >= y) will work for scalars/vectors,
but if you've got matrices, you will need to use a completely different syntax:
@constraint(model, x >= y, Nonnegatives) or @constraint(model, x - y, Nonnegatives),
which requires actual reasoning about which of the sets should be used there,
wherein with @constraint(model, x >= y) syntax it just works.

I agree that this is very much not a clear-cut choice,
though it would seem to me that making such a change
would allow for a much more natural-looking models.

[odow]

My answer to:

while @constraint(model, x >= y) will work for scalars/vectors, but if you've got matrices, you will need to use a completely different syntax

is "yes, because of the ambiguity that users will experience, particularly those coming from YALMIP, where >= defaults to PSDCone: https://jump.dev/JuMP.jl/dev/tutorials/transitioning/transitioning_from_matlab"

There is no ambiguity for scalars and vectors.

[LebedevRI]

My answer to:

while @constraint(model, x >= y) will work for scalars/vectors, but if you've got matrices, you will need to use a completely different syntax

is "yes, because of the ambiguity that users will experience, particularly those coming from YALMIP, where >= defaults to PSDCone: https://jump.dev/JuMP.jl/dev/tutorials/transitioning/transitioning_from_matlab"

Ah, i did not know that. But then, is it not reasonable to expect them to read the documentation?
They'll also be able to notice the problem by actually looking at the built constraint (by printing it/model).
It seems unfortunate that said short transitonary/learning period expirience
should make better experience for everyone else be unachievable.

[blegat]

I think x >= y between scalars is clear, x >= y between vectors is slightly ambiguous because it does not say what is the order between vectors but I don't think there is too much risk that someone would misinterpret this and think that another cone than the positive orthant would be used by default.
For matrix cones, it's an inequality with no cone specified could be interpreted differently by different people so I don't think it would be helpful. Even for users that have read the docs, using this syntax would make their model have the potential of being misinterpreted.
For many people, >= is the only way to compare things in ascii so adding subtle difference with unicode character is going to be a source of confusion.

[LebedevRI]

Copying more of #3769 (comment) here, for clarity:
The proposed behavior is:

@constraint(model, x \succeq y)
# error: specify one of the conic sets

@constraint(model, x >= y, <... Some conic set ...>)
# warning: use \succeq instead

@constraint(model, x \succeq y, <... Some non-conic set ...>)
# warning: use >= instead

@constraint(model, x >= y)
# becomes @constraint(model, x - y, Nonnegatives)

@constraint(model, x <= y)
# becomes @constraint(model, x - y, Nonpositives)

I.e. if you use >=/<=, and don't specify a set, or specify a non-conic set, you get vector behavior,
if you use >=/<=, and specify a conic set, you get a warning to use \succeq instead,
and if you use \succeq, you must specify a set, and if it's not a conic one, you get a warning.

So really, the only thing one needs to remember is to use \succeq when meaning conic constraint.
Is this really too much to ask of users?

[odow]

1. We could do this
2. We can't do this. That is breaking. We don't want to start warning for commonly used existing code.
3. We can't do this. There is no "conic"/"non-conic" distinction in MOI
4. We could do this, but there is the ambiguity question
5. We could do this, but there is the ambiguity question

Is this really too much to ask of users?

Experience has told me: yes. Especially for novice users who are most at risk of making this mistake. (Not to pick on one example, but here is a recent one that I remember: https://discourse.julialang.org/t/lmi-optimization-problem/112218. They ended up trying .<= 0 instead of <= 0, PSDCone() just to get something that didn't error.)

I'm okay adding the \succeq operator as a new opt-in feature. I don't want to change existing behavior.

[blegat]

What do you mean by @constraint(model, x \succeq y, <... Some non-conic set ...>) ? What is an example of non-conic set ? Do you mean Nonnegatives is not a cone ? The issue is you can also see Nonnegatives as just one cone among the others and it's not really the first one you would think of when thinking of matrices.

[jd-foster]

This discussion is reminiscent of JuliaLang/julia#44358 where a breaking change in Base Julia was made to avoid element vs. vector division confusion.

[LebedevRI]

What do you mean by @constraint(model, x \succeq y, <... Some non-conic set ...>) ?

I specifically mean PSDCone/HermmitianPSDCone, yes, which are, per previous comments,
what newcomers may think what the matrix x >= y may mean.

What is an example of non-conic set ?

I only mean Zeros/Nonnegatives/Nonpositives by that.

This discussion is reminiscent of JuliaLang/julia#44358 where a breaking change in Base Julia was made to avoid element vs. vector division confusion.

Right, though here the behavior would be consistent (and in fact,
remove an inconsistency that the matrix and non-matrix arguments
require different syntax), and well-documented. The fear really is that
newcomers may not spare time to actually read what the docs say
and write one thing thinking it does something completely different...

[odow]

Thinking on this over the last few days, I don't even particularly want to add \succeq as an opt-in syntactic sugar for what we already have. It opens up the question of "how is it different from >=" and our answer will be that it isn't.

What we currently have is Pareto optimal on a set of tradeoffs. We could make it better for some people at the expense of others, and it isn't obvious that moving one particular way is preferable.

[LebedevRI]

@blegat

For matrix cones, it's an inequality with no cone specified could be interpreted differently by different people so I don't think it would be helpful. Even for users that have read the docs, using this syntax would make their model have the potential of being misinterpreted.

1. That's why i'm suggesting \succeq as the "only" method to introduce conic constraints
   (as per user-facing documentation, i get the part about not breaking existing models).
   With that disambiguation >= becomes available to be used for the $subj syntax sugar.
2. Isn't that confusion essentially already there? Given just x >= y, you can't know if they are matrices or not?
   I don't get this argument.

@odow

Thinking on this over the last few days, I don't even particularly want to add \succeq as an opt-in syntactic sugar for what we already have. It opens up the question of "how is it different from >=" and our answer will be that it isn't.

Oh come on...

">= syntax is used to perform element-wise comparisons, regardless the type of the variable,
as long as the shapes match, while the \succeq syntax is used when
a) you are operating on a matrix, and b) you are adding a explicitly specified conic constraint.
While >= syntax can also be used to add such constraints, that is a legacy interface
that is retained only as a stop-gap measure to support legacy models,
it may be dropped in future major version."

What we currently have is Pareto optimal on a set of tradeoffs. We could make it better for some people at the expense of others, and it isn't obvious that moving one particular way is preferable.

Perhaps i'm missing some context that was communicated somewhere elsewhere,
but i'm still a bit confused what those exact groups are.

From where i'm sitting, i wouldn't expect those who have previously encountered
this issue (using/thinking >= 0 to mean `>= 0, PSDCone()``) before, to completely forget it the next day.
Is that an unreasonable expectation? That's the delimiter i thought is in play,
and then i would say one group is disproportionally larger than the other one.

Experience has told me: yes. Especially for novice users who are most at risk of making this mistake. (Not to pick on one example, but here is a recent one that I remember: https://discourse.julialang.org/t/lmi-optimization-problem/112218. They ended up trying .<= 0 instead of <= 0, PSDCone() just to get something that didn't error.)

As someone who had to deal with users before, to me that tells that even with the current behavior
(a diagnostic "did you mean to use PSDCone()?"), there will always be cases where a completely wrong thing
will be done, falsely hoping it does something else. Trying to avoid that at all cost seems impractical, IMO.
It will always happen regardless.

[blegat]

The current design is pareto optimal with respect to the readability of models. If you see x >= y alone and you know they are vectors (ok if you don't know their shape it could be ambigious, that's why we didn't directly add it). If you see A >= B alone and A and B are symmetric square matrices, some people will see this a matrices, because it is a much more natural order between symmetric square matrices. No amount of documentation can solve that. Some people may loose hours debugging before finally figuring out that this actually means elementwise inequality.

The fear really is that newcomers may not spare time to actually read what the docs say and write one thing thinking it does something completely different...

Yes, or use JuMP again after some time so don't read the doc again, or know the distinction but made a mistake by using the wrong unicode character and don't notice it because you need to zoom in to see the difference.

With the cone specified (including Nonnegatives as a cone here), it's always explicit and clear. You need more keystrokes but also increase the readability of your model so each of these keystrokes is a win IMO. JuMP tries to be a modeling language that works for Constraint Programming, Conic Programming, Linear Programming, Non-Linear Programming, ... so we can't use a syntax that will feel natural for LP but that will make people from other fields struggle

[LebedevRI]

Okay, let's backtrack.

My syntax complaint is: it seems like having to manually go between the x >= y / x <= y syntax, and
@constraint(model, x - y in Nonnegatives) / @constraint(model, x - y in Nonpositives) is lacking sugar.
While, clearly, the suggestion of allowing @constraint(model, x >= y) to just work for matrices
is not receiving a warm welcome, there is at least one alternative:

Introduce a 'phony' JuMP set, ElementWiseComparison (name TBD),
which is immediately lowered into either Nonnegatives or Nonpositives, depending on the comparison:

@constraint(model, x >= y, ElementWiseComparison)
   =>
@constraint(model, x - y in Nonnegatives)

@constraint(model, x <= y, ElementWiseComparison)
   =>
@constraint(model, x - y in Nonpositives)

@constraint(model, x >= y)
# error: <...>

@constraint(model, x <= y)
# error: <...>

@constraint(model, x <= y, PSDCone)
# error: use `>=`

Hopefully surely that is a reasonable acceptable middle ground solution? :)

[odow]

First, note that if you want element wise inequality, you have always been able to write:

@constraint(model, x .<= y)

This is perhaps the simplest and most explicit way of writing it down. The only downside is that in the symmetric matrix case, it does not exploit symmetry. (But perhaps we could make it.)

Nonnegatives is pretty much your ElementWiseComparison, except that we would have:

@constraint(model, x <= y, Nonnegatives())
   =>
@constraint(model, y - x in Nonnegatives())

Because of the slight mis-match, we currently error:
https://github.com/jump-dev/JuMP.jl/pull/3766/files#diff-8cd8322515b4f61e46a3296d7ca78445d1eb335270416ee1dec6dd831b5e8ad7R686-R695

We can't do this because it breaks existing syntax:

@constraint(model, x <= y, PSDCone)
# error: use `>=`