DECSDM (private mode 80) is interpreted incorrectly

[j4james]

I'm fairly certain that you've got the DECSDM mode backwards. I think it is XTerm that has implemented it incorrectly, and MLterm that has it the right way around. CSI ?80l should enable sixel scrolling, and CSI ?80h should disable it.

Other terminals that match MLterm's behaviour include RLogin (you can see the source here), Tanasinn (see here), and WRQ Reflection (it's a commercial app, but you can see the documentation describing the behaviour of DECSDM here).

I suspect the misunderstanding came about because the VT330/340 manual documented DECSDM the way XTerm has currently implemented it, but I believe that was a mistake. The various VT382 manuals have it the other way around. See here, and here.

The author of Tera Term, who has a collection of DEC terminals, actually tested this on a VT330 and VT382 and reported the results here: https://twitter.com/ttdoda/status/479053314412126208

According to Google translate:

The actual machine and manual of VT382 and the actual machine of VT330 are DECSDM reset and sixel scrolling mode is enabled.

Which I understand to mean that sixel scrolling mode is enabled when DECSDM is reset.

Now I guess it's possible that everyone else is wrong about this, and XTerm is right, but all the evidence appears to point in the other direction.

[hackerb9]

Thank you for gathering all that evidence. How do you explain the "mistake" in the DEC documentation? It's not a simple transposition as it discusses both behaviors.

[j4james]

Reading the VT3xx manual here, my thought was that the documentation writer probably got confused between the concept of the "Sixel Scrolling" feature in the Graphics Set-Up screen, and "Sixel Display Mode", which are really the opposite of each other.

When you enable "Sixel Display Mode", you're requesting Sixel images be displayed full-screen, starting in the top left corner. You're not asking to enable scrolling. If DECSDM was intended to enable scrolling you would think they would have called it something like "Sixel Scrolling Mode".

Anyway, I don't want to claim that I'm absolutely certain about this, but it just seems very likely, given the evidence, that XTerm got it wrong.

[j4james]

The other thing I meant to ask, was do you really need to set this mode at all? You'll probably have tested this on more terminals than I have, but all the ones I've tried have scrolling enabled by default, regardless of how they interpret DECSDM. So I would have thought the safest thing to do was just to leave the setting alone.

[hackerb9]

Please share with me the list of terminals you've seen that have scrolling enabled by default. It makes sense that lsix should default to whatever people most commonly use, whether it is "right" or "wrong". I believe the main reason I added the escape sequence to enable scrolling in the first place was because some terminals (mlterm?) had it off by default. If that's no longer an issue, then removing the control sequence may be the best option.

I see how you could think the DEC VT3xx manual is in error. I have been working off of a more thorough reference from DEC, EL-SM070-00 DEC STD 070 Video Systems Reference Manual Dec91 .

[j4james]

Please share with me the list of terminals you've seen that have scrolling enabled by default.

XTerm (360), MLterm (3.8.4), Contour (0.1.1), WezTerm (20210502), Mintty (3.5.0), RLogin (2.25.3), and Reflection Desktop (16.2.0).

I added the escape sequence to enable scrolling in the first place was because some terminals (mlterm?) had it off by default.

I figured that might have been the case, but hopefully whoever that was will have fixed the issue by now.

I have been working off of a more thorough reference from DEC, EL-SM070-00 DEC STD 070 Video Systems Reference Manual Dec91 .

Yeah I've got the STD 070 manual, but that has no mention of DECSDM mode as far as I'm aware. It just points out that the VT125 and VT240 don't support scrolling, but that doesn't really help us. Or is there something in there that I've missed?

[hackerb9]

Thank you for that research. I think you're right that if all modern terminals start with scrolling on, we might be able to skip the sequence. [Reminder to self: if we do this, check TERM and send it for those terminals that start with scrolling off (vt125?).]

However, there is no way I know of to poll the current setting, so lsix cannot know what state the last sixel application has left the terminal in. Maybe this is worrying about nothing, but I notice other projects, like notcurses seem to think it is important to always enable sixel scrolling.

[hackerb9]

By the way, as I reread the specs, I became less sure I read them correctly. I'd like to know what the actual hardware behaviour is, so I have purchased a VT340+. Hopefully we'll get this resolved in two weeks when it arrives.

[j4james]

I'd like to know what the actual hardware behaviour is, so I have purchased a VT340+. Hopefully we'll get this resolved in two weeks when it arrives.

This is brilliant news! Once you've got it set up, do you think you might be interested in running some other Sixel compatibility tests too? I've been putting together a small set of tests covering edge cases in the spec that I was hoping to be able to run on a VT340 one day. The idea being to create a repository of examples with expected output which terminal developers could refer to if they want to be VT340 compatible.

Each test is just a small text file that can be output to the terminal, and the result then photographed as a reference. For example, this is my current DECSDM test case:

sixel_display_mode.txt

And this is what I would expect as output (assuming I'm correct in my understanding of the VT340 behaviour).

Click to view

[j4james]

I notice other projects, like notcurses seem to think it is important to always enable sixel scrolling.

Yeah, I've been slowing raising issues with other apps that I've seen doing this. So far I've yet to find anyone that had a legitimate reason for it. I haven't got around to notcurses yet because I wanted to be able to actually test their code first, and I couldn't get it to build. I should probably just raise the issue anyway.

check TERM and send it for those terminals that start with scrolling off (vt125?)

The VT125 (and the VT240) don't support Sixel scrolling at all. They're effectively always in "display mode", so the image will always start in the top left corner of the screen, and will be clipped when it reaches the bottom.

[wez]

@dankamongmen seems like the kind of person that would happily join this discussion even without a formal issue being filed :)
I'm no expert on this, and I'm just idly perusing this issue from the one that links here in wezterm, but this is the relevant section from notcurses:

https://github.com/dankamongmen/notcurses/blob/fe61082c5cc3439083ea80fad19c0811cfd2683b/src/lib/sixel.c#L863-L872

[dankamongmen]

i'm familiar with my valued comrade @j4james 's DECSDM march:

dankamongmen/notcurses#1782

i think he has the right idea, but i think in practice it would turn out flawed, as the (over)eager merging of my PR to mlterm kinda supports IMHO:

arakiken/mlterm#23

in the end, pretty much no one got everything they wanted, and all parties seemed a bit less happy than they were before. i apologize for my role in this expansion of sadness, but have high hopes for a brighter tomorrow.

[hackerb9]

Yes, I was thinking of asking @dankamongmen his opinion about omitting the scroll initialization. I have a guess, though, given his frequently expletive laden comments about other software packages. 😏 Like XTerm author @ThomasDickey, he clearly cares about good engineering.

If @j4james is correct, his suggestion would work as long as no other application had changed the default setting. lsix might be able to get away with it as simplicity is one of its design goals. My guess is Nick Black would reject it out of hand since notcurses has to be more robust than that.

Given that different terminals (XTerm and mlterm) interpret the same sequence in opposite ways, a reasonable solution might be a TERMCAP entry for "enable sixel scrolling". But I would want to get @ThomasDickey's opinion first.

[dankamongmen]

Given that different terminals (XTerm and mlterm) interpret the same sequence in opposite ways, a reasonable solution might be a TERMCAP entry for "enable sixel scrolling". But I would want to get @ThomasDickey's opinion first.

a terminfo entry definitely would be the preferred way to get this information (aside from issues with TERM possibly being incorrect, le sigh).

Yes, I was thinking of asking @dankamongmen his opinion about omitting the scroll initialization. I have a guess, though, given his frequently expletive laden comments about other software packages

=D i'm not sure i've ever spoken with you, @hackerb9 , but i dig your style

[ThomasDickey]

I can wait til you've investigated the VT340. The existing documentation is too sketchy to say whether one was wrong or whether DEC happened to implement the feature differently: The PDF for the first editing of the VT330/VT340 graphics manual doesn't mention the feature, and the VT382 manual is undated.