XML encoding is not taken into account

[DavidNemeskey]

While jusText extracts the page encoding for a HTML page correctly from the meta attribute, it does not for XHTML, which uses an XML header:

<?xml version="1.0" encoding="iso-8859-2"?>

[miso-belica]

You are right. Do you know any real website using XML serialization of the HTML?

[DavidNemeskey]

Unfortunately not, it was quite some time ago... I encountered this issue while processing Common Crawl data, but I do understand that having to download & parse a billion pages to find one that uses XHTML is a bit too much to ask 😄

[miso-belica]

OK, thank you. I guess it's not that hard to add. Maybe I am wrong but I don't think there are plenty of XHTML documents left out there. We will see if anyone else writes here.

Have a nice day 🙂