Skip to content

Add Apple Live Photo Support#1139

Open
bexelbie wants to merge 4 commits into
bpatrik:masterfrom
bexelbie:feature/609-live-photo-support
Open

Add Apple Live Photo Support#1139
bexelbie wants to merge 4 commits into
bpatrik:masterfrom
bexelbie:feature/609-live-photo-support

Conversation

@bexelbie
Copy link
Copy Markdown

Closes #609

What

Apple Live Photos (HEIC still + MOV video pairs) are displayed as a single media item with inline video playback, instead of appearing as two separate
entries.

The feature is gated behind Client.Media.LivePhoto.enabled (default: true). Schema version 41→42 triggers a full re-index on upgrade.

This is my first commit. I believe I have followed the guide in the code and met your standards. I'm happy to make changes.

Strategy

  • Index everything, pair at response time. Both the HEIC and MOV are indexed independently. Pairing happens in cleanUpGalleryResults when building the API response — no schema-level joins or foreign keys.
  • Metadata-based matching. Pairs are linked by a Content Identifier UUID embedded in both files (Apple MakerNote tag 0x0011 in HEIC, QuickTime content.identifier in MOV). No filename heuristics.
  • Companion videos are filtered from listings after pairing, so the grid shows one item per Live Photo.
  • contentIdentifier stays server-side only. It lives on the DB entity for indexing but is stripped before the API response — the client never sees it.

UX

  • Grid: A "LIVE" badge appears on paired photos.
  • Lightbox: Hovering (or touching) the LIVE badge plays the companion video inline, muted, matching Apple's behavior. The badge is the only interactive hotspot — it does not interfere with existing swipe/zoom/pan gestures.
  • Info panel: Shows both the still image and companion video metadata (filename, dimensions, size, duration) in separate rows.

Commits

  1. Data model, config, metadata extractioncontentIdentifier column, LiveVideoInfo interface, MakerNote parser, ffprobe extraction, config toggle, schema version bump (41→42)
  2. Response-time pairing middlewarepairLivePhotos() in GalleryMWs, contentIdentifier stripping, ContentWrapper pack/unpack for liveVideoPath/liveVideoInfo
  3. Lightbox playback — hover-to-play on badge, pointer-events: none container to avoid gesture conflicts, muted video element
  4. Grid badge + info panel — LIVE indicator in grid, companion video details in info panel with videocam icon

Tests

  • Backend: MetadataLoader extraction tests (MOV content ID, HEIC MakerNote, non-Live-Photo passthrough) and GalleryMWs pairing tests (matching, non-matching, config disabled, search results, unpaired video retention, contentIdentifier stripping)
  • Frontend: GridMedia unit tests, grid photo badge rendering, lightbox media rendering + interaction (mouse and touch events), info panel companion video display

bexelbie and others added 4 commits March 24, 2026 19:47
Add contentIdentifier optional string field to MediaMetadata and a
corresponding column in MediaEntity for storing Apple Live Photo
Content Identifier UUIDs. Add liveVideoPath (computed at response time,
not stored) and LiveVideoInfo interface for companion video metadata.
Bump DataStructureVersion 41→42.

Add ClientLivePhotoConfig with enabled boolean (default: true) under
ClientMediaConfig.LivePhoto, with uiResetNeeded: {db: true} to trigger
re-index when toggled.

MOV: Read com.apple.quicktime.content.identifier from ffprobe format
tags. HEIC: Parse Apple MakerNote IFD structure from raw exifr
makerNote bytes to extract tag 0x0011 (ContentIdentifier). Both
extraction paths are gated behind Config.Media.LivePhoto.enabled.

Tag reference: ExifTool Apple.pm by Phil Harvey.

Tests: 4 new tests covering MOV extraction, HEIC extraction, regular
video (no contentIdentifier), and disabled config (skips extraction).

Refs: bpatrik#609

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implement Live Photo pairing in GalleryMWs.cleanUpGalleryResults:
- Build contentIdentifier→video map from all videos in the response
- Match photos to companion videos by contentIdentifier equality
- Set liveVideoPath and liveVideoInfo on matched photos
- Filter matched companion videos from the response
- Unpaired videos remain visible as regular media items
- Handles both directory listings (parentDir fallback) and search
  results (per-item directory property)

Add liveVideoPath and liveVideoInfo pack/unpack support in
ContentWrapper (mapped to 'l' and 'li' respectively).

Tests: 5 new middleware tests covering pairing, non-matching IDs,
disabled config, search results, and mixed media with unpaired videos.

Refs: bpatrik#609

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add isLivePhoto() and getLiveVideoPath() to GridMedia for frontend
companion video detection and URL construction.

Add LIVE badge overlay in the lightbox that triggers video playback on
hover (mouseenter/mouseleave) and touch (touchstart/touchend). The
companion video plays muted and looped over the still image with a CSS
opacity transition. The badge sits at z-index 10 to stay above the
lightbox gesture handlers that would otherwise intercept mouse events
on the full image area.

Mobile: touch-and-hold the LIVE badge to play, release to stop.

Refs: bpatrik#609

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Show a 'LIVE' text badge in the top-right corner of grid thumbnails
for paired Live Photos, styled consistently with the existing video
duration indicator.

Add companion video details to the lightbox info panel: when viewing
a Live Photo, display a two-row file info section — the photo (image
icon, name, dimensions, megapixels, size) followed immediately by the
companion video (video camera icon, name, dimensions, size, duration)
with no visual break between rows.

Refs: bpatrik#609

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bexelbie
Copy link
Copy Markdown
Author

Untitled Untitled 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for Apple Live Photos

1 participant