Line numbers in ssh debug output and changes triggered by ESC[K sequence

[J-cztery]

• I have read and understood docs/Issues.md.

Describe the problem

Oxidized reports spurious changes in the config

When ssh debug is enabled in the log i am seeing lines and console control characters :
^[[7mlines 24-46 ^[[m^[[K^M^[[K interface loopback 0 ip address 10.2.1.1/32 primary
Similar to #3453 but for console control characters

From git i see:

[root@hcf01 devices.git]#  git --bare --git-dir="/root/.config/oxidized/devices.git" show 4a8fdce34d2ad19dec409b137857c21dcd52b1e1
commit 4a8fdce34d2ad19dec409b137857c21dcd52b1e1
Author: oxidized <oxidized@github.com>
Date:   Mon Mar 24 20:14:43 2025 -0400

    update default/hs-25

diff --git a/default/hs-25 b/default/hs-25
index 30a7ed1..a0d2164 100644
--- a/default/hs-25
+++ b/default/hs-25
@@ -69,7 +69,7 @@ no cli default prefix-modes enable
    logging vrf default source-interface loopback0
    tacacs-server vrf default source-interface loopback0
    
-[K##
+##
 ## QoS switch configuration
 ##

Expected behavior
No changes triggered by spurious change detections.
No line "lines 24-46" visible and no control characters visible as they depend on the size of terminal?

Running environment (please complete the following information):

Oxidized v.0.28.0

Device:
Product name: Onyx
Product release: 3.9.2504
Build ID: #1-dev
Build date: 2021-04-29 06:31:55
Target arch: x86_64
Target hw: x86_64
Built by: sw-r2d2-bot@26dc75704811
Version summary: X86_64 3.9.2504 2021-04-29 06:31:55 x86_64

Product model: x86onie

[robertcheramy]

x86onie is not an official model in oxidized.
If this is a custom model, you should share it here, so we can see what is going on. Usually, Escape codes and pager lines are stripped down from the model.

[J-cztery]

x86onie is not an official model in oxidized.

Sorry this is Mellanox onyx. So the model i am using is mlnxos.rb

I have since upgraded onyx to 32.1 but the problem persist. On top of that there is a new problem in 32.1: /x00 characters and files are deemed binary (on some other switch).
I have added
cfg.gsub! /#{"(\x00.*$)".b}/, ''
so the relevant section in mlnxos.rb is now:

 cmd :all do |cfg|
    cfg.gsub! /^[\[\?1h^[=\r/, '' # Pager Handling
    cfg.gsub! /^[\[24;1H/, '' # Pager Handling
    cfg.gsub! /\r^[\[K/, '' # Pager Handling
    cfg.gsub! /^[\[K/, '' # Pager Handling
    cfg.gsub! /\s^H/, '' # Linebreak Handling
    cfg.gsub! /#{"(\x00.*$)".b}/, ''
    cfg.gsub! /^CPU load averages:\s.+/, '' # Omit constantly changing CPU info
    cfg.gsub! /^System memory:\s.+/, '' # Omit constantly changing memory info
    cfg.gsub! /^Uptime:\s.+/, '' # Omit constantly changing uptime info
    cfg.gsub! /.+Generated at\s\d+.+/, '' # Omit constantly changing generation time info
    cfg.lines.to_a[2..-3].join
  end

but the issue with /x00 does not go away.

[J-cztery]

TL;DR; Would it be possible to read entire input from ssh and only then process it by regexps?

I think the issue (with [K left behind) is this:

received "(...)   interface ethernet 1/1-1/2 shutdown\r\n   interface ethernet 1/4-1/6 shutdown\r\n\e[7mlines 1-23 \e[m\e[K"
received "\r\e[K"

i.e. when above is separated by callbacks this way, mlnxos.rb:expect produces different results than when the callbacks are broken up differently.

# Pager Handling
  expect /.+lines\s\d+-\d+([\s]|\/\d+\s\(END\)\s).+$/ do |data, re|
    send ' '
    data.sub re, ''
  end

I have since changed :

cfg.gsub! /#{"(\x00.*$)".b}/, ''

to

cfg.gsub! /#{"(\x00.*)".b}/, ''

and

cfg.gsub! /\r^[\[K/, '' # Pager Handling

to

cfg.gsub! /\r\033\[K/, '' # Pager Handling

because irb complained about it but the problem is not yet resolved (well, /x00 issue might be gone, need to double check that).

[robertcheramy]

Please submit a YAML simulation file.

extra/device2yaml.rb oxidized@r61 -c "show version
show inventory
enable
show running-config
exit" -o spec/model/data/mlnxos:Onyx_32.1:simulation.yaml

Interact with the device (Press [space] when the pager asks to, enable will probably ask you to enter your password) and clean afterwards data you don't want to share, but leave the escape codes unchanged.

[J-cztery]

Here:
mlnxos:Onyx_32.1_3-30:simulation.yaml.txt

Again, i do not think the problem is in the contents of the output, just how it gets split up by channel callbacks in ssh.rb

[J-cztery]

@robertcheramy do you think it would be feasible to collect all the config data unchanged in one string and just call @output = @node.model.expects @output once the data is collected?

Would eof be a good callback to do it?

[robertcheramy]

I'm working on it.

The solution will be a combination of https://github.com/ytti/oxidized/blob/master/docs/Creating-Models.md#common-task-remove-ansi-escape-codes and more cleaning regexps.

I'll produce a PR you'll have to test because I don't have a device for it.

[J-cztery]

I'm working on it.

Thanks!

I'll produce a PR you'll have to test because I don't have a device for it.

Not a problem.

[robertcheramy]

Could you try if https://github.com/ytti/oxidized/blob/fix/3469-mlnxos/lib/oxidized/model/mlnxos.rb fixes the problem and produces the expected output?

Since the model simulation does not process pager requests and your simulation file send "exit" to early, I had to tinker and guess, so we may need to change things for it to work ;-)