PCX format issue (incorrect palette) #210
Description
Hey there.
This is a bit of an odd one, but the PCX plugin is incorrectly handling certain malformed PCXes. Essentially, there's a bit of a weirdness with the spec, because even if you specify the palette to be grayscale (the 16-bit palette type is "2"), the palette at the end of the file is still an RGB palette. Currently, SAIL is incorrectly handling this combination of parameters (8-bit image but marked as 'grayscale'), and interpreting the image to be shades of red.
From looking at other places that implement PCX, it seems like palette 1 and 2 are functionally identical (at least in the case of 8bpp): the only difference is that, for palette 2, the palette is usually grayscale colors, but in the wild this isn't always the case (see Quake II, which uses PCX for its images and has an RGB palette at the end of each PCX). I can provide a test file if necessary.
GIMP's implementation: https://github.com/GNOME/gimp/blob/1b29e77c17407adab208e1718525b8eba0baabac/plug-ins/common/file-pcx.c#L815
The bug is on this line: https://github.com/HappySeaFox/sail/blob/master/src/sail-codecs/pcx/helpers.c#L70 in both cases, the pixel format should be SAIL_PIXEL_FORMAT_BPP8_INDEXED and should be processed by pcx_private_build_palette (note that currently, BPP8_GRAYSCALE isn't even handled by that function even though it should be the same code). In theory, though, one could process the palette and then mark it as grayscale if the palette is actually grayscale but I don't know if that's premature or not.