Investigate strategies for including captions in video works

[nhomenda]

Descriptive summary

Captions are not currently supported in default video player.

Rationale (for feature request only)

Upcoming Title II ADA requirements for public institutions mandate captions on videos.

Related work

Updating UV to include Ramp
Add VTT captions to UV
IIIF AV Interoperability Testing
IIIF Cookbook Example

[trmccormick]

I'm not sure if this is the place for this but i was looking into this some. Maybe something like this?

Technical Plan: Video Captions in Hyrax

Goals

• Allow users to upload and associate caption files (e.g., .vtt, .srt) with video works
• Display captions in the video player for accessibility
• Support multiple languages and proper UI/UX

Proposed Steps

1. Allow Caption File Uploads

• Enable users to upload caption files as part of a work (as a FileSet)
• Optionally, add a metadata field or "type"/"use" for caption files to distinguish them

2. Associate Caption Files with Video FileSets

• When displaying a video, query for associated caption files (by mime type or "use")
• Store the relationship in metadata or by convention (e.g., filename, file type)

3. Update Video Player Partial

• Edit app/views/hyrax/file_sets/media_display/_video.html.erb to include <track> elements for each caption file:

<video ...>
  <source ... />
  <% caption_files.each do |caption| %>
    <track kind="captions" src="<%= hyrax.download_path(caption) %>" srclang="en" label="English" />
  <% end %>
</video>

• Ensure the player supports selecting captions

4. UI/UX Improvements

• Add UI hints for uploading caption files
• Optionally, allow users to specify language/label for each caption

5. Accessibility

• Ensure captions are discoverable and selectable in the video player
• Test with screen readers and keyboard navigation

6. Documentation

• Document the workflow for uploading and associating caption files

Open Questions

• How should caption files be distinguished from other files in the UI?
• Should we support multiple caption languages per video?
• What metadata should be captured for each caption file?

[kirkkwang]

@nhomenda @cjcolvar I have a local instance where i installed Ramp and here's my attempt at transcriptions. The thing is that this structure has to be a Work with a A/V file set, and also one or more vtt file sets. The downside is that you can't have multiple videos on the same work. Maybe we can implement something in the future where you can add these works with videos + vtts to a parent work which could display a viewer that has multiple works, this is one of our solutions in a Hyku instance at the moment. Looking for thoughts on this structure or any other ideas.

[nhomenda]

@kirkkwang This is fantastic. At Tufts our Hyrax has a local customization that behaves the same way with the same limitations (not using Ramp, though). I support this as a huge first step to offering captions on video works