Output embeddable HTML directly

[lorenzoh]

I've run into two use cases where I'd like to show an image to text/html directly to get embeddable output:

• to render code block results in Pollen.jl
  

• for rich HTML display of images inside other HTML structures, like a table
  For example, inside a PrettyTables.jl table using HTML output:
  

Is there an implementation of show(::IO, ::MIME"text/html", ::ColorantMatrix) somewhere?

---

Workaround

For now, I've been using the below functions to render images (and other data that can be rendered to image mimetypes like plots) to HTML. However, it would be nice to have a method, at least for images (ColorantMatrixes) for showing to text/html that falls back on png/jpg/svg output.

function Base.show(io::IO, mime::MIME"text/html", x::ShowAsHTML)
    if showable(mime, x.val)
        show(io, mime, x.val)
    elseif showable(MIME("image/jpg"), x.val)
        buf = IOBuffer()
        show(buf, MIME("image/jpg"), x.val)
        print(io, """<img src="data:image/png;base64,$(Base64.base64encode(take!(buf)))"/>""")
    elseif showable(MIME("image/png"), x.val)
        buf = IOBuffer()
        show(buf, MIME("image/png"), x.val)
        print(io, """<img src="data:image/png;base64,$(Base64.base64encode(take!(buf)))"/>""")
    elseif showable(MIME("image/svg+xml"), x.val)
        buf = IOBuffer()
        show(buf, MIME("image/svg+xml"), x.val)
        print(io, replace(String(take!(buf)), "\n" => ""))
    else
        print(io, x)
    end
end

[johnnychen94]

This looks a good improvement to me, do you mind opening a PR for this?

[lorenzoh]

Sure, will do!

The PR would include primarly a method show(::IO, ::MIME"text/html", ::ColorantMatrix). Is the precedence jpg > png > svg fine? I was going for smallest size if possible.

using ImageShow

show(io, MIME("text/html"), image)

Additionally, I would like to add a wrapper type that can wrap any type with some image mimes (not just ColorantMatrixes) to force it to try using image mimes to display HTML. This wrapper type could also accept as optional argument a different precedence of preferred image encodings. Maybe HTMLImage is a good name?

So you could do

using ImageShow, CairoMakie

fig = scatter(rand(1:10))

show(io, MIME("text/html"), HTMLImage(fig))  # would  result in a png
show(io, MIME("text/html"), HTMLImage(fig; mimes = ["image/svg", "image/png"]))  # would result in an svg

Does this design make sense?

[johnnychen94]

I think it only makes sense for ImageShow to support show(io, MIME("text/html"), image).

The HTMLImage seems more like a wrapper type that you can define anywhere you want since it can be only a few lines:

struct HTMLImage{T}
    x::T
    mimes::Vector{MIME}
end
Base.show(io::IO, fig::HTMLImage) = show(io, findfirst(mime->showable(mime, fig.x), fig.mimes), fig)
Base.show(io::IO, ::MIME("text/html"), fig::HTMLImage) = show(io, MIME("text/html"), fig.x) # delegate the actual rendering to its owner package

For type stability, it's better to be mimes = [MIME("image/svg"), MIME("image/png")] with an optional friendly depwarn for mimes = ["image/svg", "image/png"].

Then you can add show(io::IO, ::MIME("text/html"), x) method for any concrete data type, e.g., AbstractArray{<:Colorant} to its owner package.

---

P.S. it's okay to put HTMLImage in ImageShow.jl, but it is not okay to add, say, CarioMakie.jl only for its figure type to ImageShow.jl.

[lorenzoh]

Will take these into account!

Re the HTMLImage: I don't think your definition, since the html tags and base64 encoding have to be created somewhere. The wrapper is meant specifically for when you want to say: if this object does not have an html mime, please fall back to the image mimes and convert them to HTML.

In any case, if that is too specific a thing to be included in the library, I completely understand!

P.S. it's okay to put HTMLImage in ImageShow.jl, but it is not okay to add, say, CarioMakie.jl only for its figure type to ImageShow.jl.

Yeah, of course, that was just an example to illustrate. The idea is that a package (say Pollen.jl) that wants to embed images into HTML can do so by explicitly calling HTMLImage(x), without ImageShow.jl needing to know what x is.

[johnnychen94]

A second thought, it seems that you're suggesting a PollenHTMLDisplay stuff:

1. iterate the Multimedia.displays until displayable returns true --> This is all handled automatically as long as you push your display to the global displays
2. call display(d, mime, fig), which calls
3. show(d.io, mime, fig)

[lorenzoh]

That still doesn't define an html display for image-like things?

[lorenzoh]

I was just checking how Documenter.jl writes image-like things to HTML and found it here: https://github.com/JuliaDocs/Documenter.jl/blob/master/src/Writers/HTMLWriter.jl#L1958

Seems like they have higher precedence for mime "text/html" than the image mimes, so adding the proposed code to ImageShow.jl would mean that (for images only) ImageShow's method would be called. Only difference in Documenter.jl is that they have some extra handling for "image/svg+xml" to make sure it is valid.
Now my concern is that without porting this svg processing, some documentation that outputs svgs somewhere would silently fail. But "svg/xml" will not be used for images anyway, correct? Judging from

ImageShow.jl/src/showmime.jl

Line 18 in 324e8a4

showable(::MIME"image/svg+xml", img::AbstractMatrix{C}) where {C<:Color} = false

Just to be on the safe side, I would set the mimes that show(io, ::MIME"text/html", ::ColorantMatrix) checks to just [MIME("image/jpg"), MIME("image/png")] for now.