[Beats Monitoring] Monitoring data payload has breaking changes in schema from 6.3 to 7.0

[tsullivan]

In 6.3.x, the beats_stats field in .monitoring-beats data looks like:

"beats_stats": {
  "timestamp": ...
    "beat": {
      "type": "filebeat",
      "version": "6.3.1",
      ...
    },
    "metrics": {
      "libbeat": {
        "config": {
          ...
        },
        "output": {
          "events": {
            ...
          },
          "write": {
            ...
          },
          "read": {
            ...
          },
          "type": ...
        },
        "pipeline": {
          "clients": ...,
          "events": {
            ...
          },
          "queue": ...
        }
      },
      "system": {
        ...
      },
      "beat": {
        ...
      },
      ...
    }
}

In 7.0/master branch, the beats_stats schema has changed and now looks like:

"beats_stats": {
  "timestamp": ...,
    "beat": {
      "type": "filebeat",
      "version": "7.0.0-alpha1",
      ...
    },
    "stats": {     # WAS: metrics
      "libbeat": {
        "config": {
          ...
        },
        "output": {
          "events": {
            ...
          },
          "write": {
            ...
          },
          "read": {
            ...
          },
          "type": ...
        },
        "pipeline": {
          "clients": ...,
          "events": {
            ...
          },
          "queue": ...
        }
      },
      "system": {
        ...
      },
      "beat": {
        ...
      },
      ...
    }
}

(Ellipses added for length)

This is a breaking change for the Monitoring UI, since beats_stats.metrics is used for a lot of info about Beats in the cluster.

[tsullivan]

If the name change for the metrics field happened in 6.4, then this would be a blocker for 6.4.0

[ruflin]

I opened a PR with a fix here: #7563 Thanks for bringing this up.