[EMCAL-675] Handling of pages without trailer

[mfasDa]

In the new version of the raw format the trailer will be only
on the last page. In order to be compatible with both version
the last trailer word is checked for bit 30 (trailer marker), in
case it found the trailer is decoded and chopped from the page
as it is re-encoded when the multi-page payload is combined.
Otherwise the full payload is added to the multi-page buffer. This
works for single and multi-page payload and therefore
consequently for the old and new raw format.

[mfasDa]

Errors unrelated @shahor02 could this be merged?

[shahor02]

@mfasDa just to be sure: what is the size of the trailer? If it is > than 16B, the default splitting algo. may split the trailer. Do you handle this case?

[mfasDa]

@shahor02 The trailer consists of 9 32 bit words, so the total trailer size is 36 Byte. I think we don't handle that the trailer might be split into two pages, I was not aware of this. The firmware always put the trailer at the end of a page, so the decoder expects the trailer at the end of a page. But this case becomes obsolete as we will remove the trailers in intermediate pages and only add the trailer on the last page. Also in this case the traIler must not be split.

[shahor02]

So, potentially, it can be split. Is it ok?

[mfasDa]

I need to check this with Martin but according to my understanding the firmware never splits the trailer - it expects it at the end of a page.

[shahor02]

But then, it is better to implement the carry-over which makes sure it is never split, i.e. split the payload little bit before the trailer.

[mfasDa]

Trailer marker fixed in the last iteration (both bits 30 and 31 need to be checked - 31 alone is intermediate trailer marker, 30 alone is channel header marker).

[mfasDa]

But then, it is better to implement the carry-over which makes sure it is never split, i.e. split the payload little bit before the trailer.

@shahor02 How could I do this in the carry-over method? This would mean chopping the stream and filling the words on the page which would be used or the trailer with 0. How does the RawFileReader know in this case where to start the next page from? Furthermore I would need to determine that the last words on the page are intermediate trailer words (there are special markers for this, but how do I know what are the last words on the page?). And in addition the payload size on the last page in the trailer would need to be adapted for 0-words I added.

[mfasDa]

@shahor02 This commit is independent from the simulation - the raw parsing can handle both versions of the raw format, because it use trailer and channel markers in order to determine consecutive payload, which are set both by the hardware and in the simulation. Can you please merge it when the CI passes? We need this to be part of the next FLP suite so that it becomes available on the FLPs for our vertical slice test.

[shahor02]

@mfasDa

@shahor02 How could I do this in the carry-over method? This would mean chopping the stream and filling the words on the page which would be used or the trailer with 0. How does the RawFileReader know in this case where to start the next page from? Furthermore I would need to determine that the last words on the page are intermediate trailer words (there are special markers for this, but how do I know what are the last words on the page?). And in addition the payload size on the last page in the trailer would need to be adapted for 0-words I added.

RawFileReader looks only into the RDH, which is adapted according to what carryOverMethod returns.
Basically, your carryOverMethod should simply return the maxSize value suggested by the RawFileWriter (for automatic splitting) unless you see that the residual carried over to the next page is the sole trailer (I assume you cannot have payload made of trailer only) or its part. In this case you should enforce splitting earlier. Something like this:

carryOverMethod(const header::RDHAny* rdh, const gsl::span<char> data,
                                            const char* ptr, int maxSize, int splitID,
                                            std::vector<char>& trailer, std::vector<char>& header) const
{
 constexpr int YOUR_TRAILER_SIZE = 36;
  int offs = ptr - &data[0]; // offset wrt the head of the payload
  assert(offs >= 0 && size_t(offs + maxSize) <= data.size());   // make sure ptr and end of the suggested block are within the payload
  int leftBefore = data.size() - offs; // payload left before this splitting
  int leftAfter = leftBefore - maxSize; // what would be left after the suggested splitting
  int actualSize = maxSize;
  if (leftAfter && leftAfter <= YOUR_TRAILER_SIZE) { // avoid splitting the trailer or writing only it. 
     actualSize -= (YOUR_TRAILER_SIZE - leftAfter) + 1; // (or +16, in case you payload is aligned to GBT word, as far as I remember, it is not)
  }
  return actualSize;
}