Improve handling of proxied images

[hjiangsu]

Pull Request Description

This PR fixes issues with markdown images that rely on proxied images (for certain instances that enable this feature). Additionally, I've constrained SVG images to match up with the comment images to prevent render overflow errors.

Issue Being Fixed

Issue Number: #1527

Screenshots / Recordings

proxied.markdown.mp4

Checklist

• If a new package was added, did you ensure it uses an appropriate license and is actively maintained?
• Did you use localized strings (and added appropriate descriptions) where applicable?
• Did you add semanticLabels where applicable for accessibility?

[micahmo]

Nice fix! Just out of curiosity, what is the proxy and why do we have to parse it like this?

[micahmo]

At some point we could make this into a separate SvgViewer widget.

[hjiangsu]

Just out of curiosity, what is the proxy and why do we have to parse it like this?

This was introduced as an option in Lemmy 0.19.4:
https://join-lemmy.org/news/2024-06-07_-_Lemmy_Release_v0.19.4_-_Image_Proxying_and_Federation_improvements

From what I'm understanding, this option allows images that originally came from source A, to be served from source B instead. This helps prevent deanonymization attacks and reduce load on external servers since you (the client) are not connecting to a random external server to fetch contents (and reveal possible information about your client to that given server)

For example: if someone uploads an image on lemmy.world but I'm viewing that image on lemm.ee, then lemm.ee rewrites the image url to be handled by lemm.ee (using /api/v3/image_proxy), rather than a direct connection to lemmy.world. Hope that makes sense!

To be a bit more clear, the url rewrites it to the following format: https://{instance}/api/v3/image_proxy?url={url}

[micahmo]

Thanks for the explanation! The reason I asked is because it looks like you're grabbing the original URL from the rewritten URL so we can access the original image directly, which would defeat the purpose of the proxying.

In a browser, the proxied URL works fine. So I'm wondering why we need to access the original URL directly. Any ideas?

[hjiangsu]

The reason I asked is because it looks like you're grabbing the original URL from the rewritten URL so we can access the original image directly, which would defeat the purpose of the proxying.

Hmm, you're right. This might not be the best solution now that I think about it some more. I looked a bit more into it, and it seems like there are some scenarios where the image proxying doesn't work properly. For example, I believe this is the original link that was associated with the issue: https://lemm.ee/post/39487068/14003619.

Lemmy UI doesn't display it properly either:

This is the corresponding markdown:

![](https://lemm.ee/api/v3/image_proxy?url=https%3A%2F%2Fgrabdeesnuts.com%2Fwp-content%2Fuploads%2F2023%2F06%2Flogo.svg)

This is the error when we try to access that link:

{"error":"unknown","message":"Request error: error sending request for url (http://10.0.0.55:8080/image/original?proxy=https://grabdeesnuts.com/wp-content/uploads/2023/06/logo.svg): operation timed out"}

This might just have to be an issue that we push upstream to Lemmy instead, since it affects them too.

---

I'll see if there's possibly another way to do this without removing the benefits of the image proxying.

[micahmo]

I see, maybe the issue with proxied images in comments is an isolated case. Can you verify that other proxied images which work on the web UI also work in Thunder?

Then there's the question of the dimensions fix for squished images. Are we unable to load dimensions from proxied images, even though the images themselves load fine?

[hjiangsu]

Can you verify that other proxied images which work on the web UI also work in Thunder?

It seems like this is isolated to the one example I mentioned above so far. From my brief testing, other images in comments seem to be rendering properly, even though they are being proxied.

Then there's the question of the dimensions fix for squished images. Are we unable to load dimensions from proxied images, even though the images themselves load fine?

I applied a slightly different fix for this in this PR! Instead of using the original URL, I adjusted to logic for image URL parsing to take into account the image proxy endpoint. This should fix the issues with dimensions not loading properly and respect the image proxy settings.

[micahmo]

Glad to see that the comment issue was an isolated case! And, if I'm understanding this fix correctly, we're only using the non-proxied URL to determine that it refers to an image, right? But we're never hitting it directly? Looks much better to me!

P.S. Out of curiosity, I tried to test this fix. I logged in with my lemm.ee account. But all of the images I'm seeing are non-proxied in Thunder, despite being proxied on the web UI. So I was never actually able to hit that new block of code you added. Any ideas?

[hjiangsu]

And, if I'm understanding this fix correctly, we're only using the non-proxied URL to determine that it refers to an image, right? But we're never hitting it directly?

Theoretically, this should be the case! However, I'm noticing something (which relates to the second part of your message)

P.S. Out of curiosity, I tried to test this fix. I logged in with my lemm.ee account. But all of the images I'm seeing are non-proxied in Thunder, despite being proxied on the web UI. So I was never actually able to hit that new block of code you added. Any ideas?

I looked into this, and it seems like it might be because the API returns back the original URL for images? It seems to apply the proxy to the thumbnail image, but not the post url.

[micahmo]

I looked into this, and it seems like it might be because the API returns back the original URL for images? It seems to apply the proxy to the thumbnail image, but not the post url.

Ahh interesting! On the one hand, I would say that's not our fault and we shouldn't necessarily have to handle it. On the other hand, I'm still wondering how this fix helps the squished image case, since we don't seem to be using the thumbnail URL for full-height images. At least, I could never hit this breakpoint, either by scrolling the feed or by opening images. Maybe I'm doing something wrong that's unrelated to the API response issue.

[hjiangsu]

You're right - that block of code no longer applies for the dimension fetching logic! Initially, I had this block of code still in-place for retrieveImageDimensions:

bool isImage = isImageUrl(imageUrl!);
if (!isImage) throw Exception('The URL provided was not an image');

Because of that, I applied a fix to the URL parsing to handle that case:

if (uri.path == '/api/v3/image_proxy') {
  Uri? parsedUri = Uri.tryParse(uri.queryParameters['url'] ?? '');
  if (parsedUri != null) path = parsedUri.path;
}

However, when testing this, I realized that some dimensions weren't being fetched properly because the URL didn't conform to a standard image url (ending with the image format). To solve this, I removed the isImageUrl logic since this was also in place to catch errors with retrieving images:

final imageProvider = ExtendedNetworkImageProvider(imageUrl ?? '', cache: true, cacheRawData: true);
final imageData = await imageProvider.getNetworkImageData();
if (imageData == null) throw Exception('Failed to retrieve image data from $imageUrl');

I kept in the proxy parsing logic to handle a more general case (whenever isImageUrl is called):

• checking if a custom thumbnail is an image
• opening a proxied image link in image preview

image.parsing.mp4

[micahmo]

Whew, that's a lot! 😊

If I can summarize:

• We never really had any issues loading proxied images.
• The image dimensions error was due to the URL/file type check.
  • We've generally removed that check from image loading.
  • But we've updated that check to handle proxied URLs, for the cases where it is still used.
    • And as a result, we should never hit the original image when a proxied image is provided to us.
  • However, sometimes, the Lemmy backend gives us an un-proxied URL. And we can't really do anything about that.

Fair summary? 😊

[hjiangsu]

We never really had any issues loading proxied images.

Correct! This is assuming that the proxied image resolves to a valid image and does not throw an error.

The image dimensions error was due to the URL/file type check.

Yup!

And as a result, we should never hit the original image when a proxied image is provided to us.

I'll have to do more checks here to see if this is right. From the top of my head:

• MediaView handles pretty much all of our image rendering within the feed. It prioritizes the thumbnail images first, and only falls back to the original URL if there's no thumbnail.
• Our markdown logic should handle proxied images automatically, since they are valid URLs.
• I think our image preview may use the original URL? If this is the case, then it may not use the proxied images. This also makes sense since our Share media links show the original image URLs.

However, sometimes, the Lemmy backend gives us an un-proxied URL. And we can't really do anything about that.

It seems like Lemmy doesn't rewrite the post URL - so proxy images don't apply to places where we use postView.post.url.

[micahmo]

Cool, that all sounds good. And you're right, our image viewer uses the post image with the goal of showing the highest resolution image to the user. If Lemmy wants to rewrite those to proxy URLs, they can do that in the back end. Aside from any testing you still want to do, I think this is good to go!