Enhancement: images handling

[azaozz]

This is a "mini-proposal" on how to handle images in the editor and the front-end (follow-up from #4914).

Currently:

• Images are inserted in the editor at full size, i.e. we force the user to download 2MB-5MB files in most cases.
• When the user saves the post without changing image source from the inspector, it is possible that front-end visitors will have to also download the huge files. WP adds proper srcset attribute, but the sizes attribute is pretty useless as it only refers to the full size image file: sizes="(max-width: 6000px) 100vw, 6000px".

To fix this in the editor:

• Always insert the "large" size (1024px) images . If it doesn't exist (if the uploaded image is smaller), insert the "full" size.
• Add srcset attribute that will also list 2x large size (so images are retina ready). This is a new size and will have to be added to the default WP sizes, see below.
• Add another attribute to the tag for Gutenberg images so we can recognize them easily in PHP. This is necessary as we will have to calculate the front-end srcset and sizes attributes a bit differently. Ideally that should have the attachment ID, something like data-wp-attachment-id="1234". Then perhaps we will be able to drop the "old" way of passing the ID, class="wp-image-1234".
• Special handling of the width="123" attribute. In HTML 5.0 is has to be in pixels, but since the width of the editor is different than the width of the theme, the results would often be unexpected. This impacts mostly images that are resized when the user drags the corners or when image dimensions are set directly (see Enhancement: Inspector interface for setting pixel dimensions of image #4914). We will need better solution for these cases, perhaps recalculating the number when displaying the image on the front-end according to the theme's width.

To fix this for the front-end:

• Add new default size 2x large, 2048px max width or height.
• Add some logic when processing <img> tags and adding srcset, etc. that will produce usable sizes attribute. It will depend on the new data-* attribute and take into account theme's width when calculating sizes. If we are going to recalculate hard-coded width attributes (see above), this can be done here and the values used in both attributes. Alternatively we can set the width in percentage (HTML 4.0 style) in the editor, then replace them with pixels at this point.

Implementing all of the above will ensure all images are always "retina ready" both in the editor and on the site. It will also improve handling on the front-end and optimize image loading there.

This will also affect other enhancements, mostly #4914. Also, as described in #6131 the themes will be able to add more precise sizes attributes for the different images.

[azaozz]

Pinging @noisysocks and @iseulde for the editor changes. I can add the core changes.

For the editor the changes (as I see them) would be:

• Add a data-wp-attachment-id="1234" attribute to all <img> tags.
• Always insert the "large" size image.
• Add srcset attribute with the medium, large and xlarge sizes (we'll add the xlarge image size to the default sizes shortly).
• Add sizes attribute suitable for the image width in the editor.
• On saving, perhaps remove both the srcset and sizes as they would probably be different when displaying. Alternatively we can override them in the display filter.
• When setting any pixel values for image dimensions, base them off the editor width. It won't make sense to have images wider than what is visible in the editor.

TODO: consider what would be the best way to pass percentage width to the front-end.

[noisysocks]

Thanks for writing this up, @azaozz! Happy to help with the necessary editor changes. They sound good to me.

Alternatively we can set the width in percentage (HTML 4.0 style) in the editor, then replace them with pixels at this point.

My hesitation with this is that if a third party (e.g. plugin, RSS reader, API consumer) uses the post_content without processing it then they won't have valid HTML5 markup. "Honouring HTML" is one of Gutenberg's requirements.

Thinking out loud: could we use Gutenberg metadata to accomplish some of this, e.g. to store the attachment ID and percentage width?

<!-- wp:image {"attachmentId":123,"horizontalScale":0.25} -->
<img src="https://example.com/image.jpg" width="600" height="500" />
<!-- /wp:image -->

[azaozz]

Yeah, images are "dynamic blocks" (that's how they are treated in the classic editor too). They are the best kind of dynamic blocks as they have the "fallback markup" right in there, and are processed on the
front-end to enhance them.

Having that fallback markup makes images work "everywhere" even when the "display filters" haven't been run (this would actually leave the content without paragraphs, so don't think any plugin would do it).

We can certainly add the data we need to pass to the front-end to the blocks meta, however thinking we will need the same data in the <img> tags too. One reason is that parsing blocks on display is slow and we will get bigger chunks of HTML instead of the actual img tag (the figure with the caption, etc.). Another is that (perhaps) not all images will be in separate blocks, thinking about images in blockquotes, table cells, etc. Yet another reason is that if the post is edited in any other editor the blocks may be badly mangled, but the <img> tags will "survive" and stay intact (as long as we use standard HTML).

At the end this boils down to:

• Which is easier, faster, simpler, more readable/understandable: parsing the block and then processing the chunk of HTML (that may contain several elements), or parsing the <img> tags as we currently do.
• Can we guarantee that all images used in the editor will always be in wp:image blocks. That includes pasted images, images in table cells, quotes, and any other blocks and tags combinations plugins may come up with.

Imho we should use both the block meta and the img tag attributes, then (one day) we will be able to choose which to use when parsing the content on display.

[davisshaver]

@azaozz This is a well thought out proposal. I am not fully grasping why we don't use existing theme-declared image sizes though. Is there a prior discussion around this?

Currently a percentage field is exposed but I wasn't able to determine whether this does anything at present on the frontend. https://github.com/WordPress/gutenberg/blob/master/blocks/library/image/block.js#L266

The percentage approach is nice but it breaks backwards compatibility with the theme API.

[azaozz]

why we don't use existing theme-declared image sizes...

We do :) We use all available sizes (that have the same w / h ratio) when adding the srcset attribute on the front-end. Then the browser decides which image file to use.

[davisshaver]

@azaozz I believe that would function a bit differently than current behavior. Image sizes can also be used to deliver styles, not just the image crop to use. I also think that existing lazy load plugins expect the src value to be the "right" size so I'm not sure we can rely totally on srcset.

Edit: This would be the image_size_names_choose filter I think.

[mor10]

@davisshaver I've gone through several lazy load plugins in the past week. Many of them look for existing src and srcset attributes and simply replace them with other markup in PHP using a the_content filter. The challenge here is how to get core to generate the correct srcset and sizes attributes as well as physical images in the first place.

[mor10]

There are several issues related to responsive images that will have overlapping solutions. I propose we consolidate everything into this issue to avoid further fracturing of the discussion. Here's an incomplete list:

Related issues:

• Gallery images are not responsive #5674 Gallery images are not responsive
• Add 'has-wide-support' classes and mechanisms #4505 Add 'has-wide-support' classes and mechanisms
• Content displays incorrectly when switching themes, such as wide or full-width #4342 Content displays incorrectly when switching themes
• Allow theme to control sizes attribute for wide/full images #6131 Allow theme to control sizes attribute for wide/full images

Core tickets:

• Replace static with Dynamic Image Sizes for inserted images
• How to remove full size image path from srcset and more intelligent choices in srcset candidate list
• Optimize full size images

[ms-studio]

Regarding this, in @azaozz proposal:

Add new default size 2x large, 2048px max width or height

Has there been consideration of bumping up the current default pixel sizes of "thumbnail", "medium", and "large", which were originally defined in 2008? Maybe with Gutenberg it's the perfect moment of increasing them x2.

Otherwise, the terms "medium" and "large" will lose their semantic meaning.

[mor10]

Also related: #5650 content_width and new block widths

Same core issue cause: new content width paradigm makes things that used to be standard not work as expected and introduces need for rethink of how core works.