Add VTT captions for AV content

[edsilv]

https://www.w3.org/WAI/WCAG21/quickref/?showtechniques=122%2C123%2C125%2C127%2C134%2C111#audio-only-and-video-only-prerecorded

https://www.mediaelementjs.com/

Annotate webvtt with a describing motivation.

[stephenmcconnachie]

I have lots of examples of video and associated webvtt files, if examples are required.

[edsilv]

@stephenmcconnachie Amazing! Here's one with webvtt annotated: https://iiif-commons.github.io/iiif-av-component/examples/data/iiif/lunchroom-manners.json
Please add any examples to this issue - much appreciated.

[edsilv]

https://github.com/mediaelement/mediaelement/blob/master/docs/usage.md#setting-new-captions

[markpbaggett]

https://www.w3.org/WAI/WCAG21/quickref/?showtechniques=122%2C123%2C125%2C127%2C134%2C111#audio-only-and-video-only-prerecorded

https://www.mediaelementjs.com/

Annotate webvtt with a describing motivation.

Just adding to this as this is definitely something I'd like to see in UniversalViewer. It looks like the IIIF AV community has come to the decision that the motivation should be supplementing rather than describing as in the original post. See this issue. If helpful, here is a manifest that uses the supplementing motivation with a webVTT.

[stephenwf]

The current implementation doesn't need to know the motivation, just that its a WebVTT format: stephenwf@5436bcd from the content type. Is there a case where WebVTT would appear in the annotation body without that motivation and for it to be processed differently?

[DanOlson]

What is the status of this? Are captions supported now? I see the accessibility statement says they are not, but the merged commit 5436bcd is newer and seems to add support. Please let me know, thanks!

[stephenwf]

It's implemented for AV Manifests without ranges, but the AV Component does not yet support them. So it's 50/50

[ShanaLMoore]

It's implemented for AV Manifests without ranges, but the AV Component does not yet support them. So it's 50/50

@stephenwf Has anyone identified the work needed for the AV Component to support this? If so, could you please link me? It's possible that my team may be able to contribute to these efforts.

[stephenwf]

@ShanaLMoore The code for the AV Component is here: https://github.com/IIIF-Commons/iiif-av-component, and the relevant area of the code where the <video/> tag is created can be found here: https://github.com/IIIF-Commons/iiif-av-component/blob/master/src/elements/media-element.ts

The AV component uses only a video tag, instead of https://www.mediaelementjs.com/ which has an option for supporting VTT captions. I'm not sure what would be required to either change the AV component to use media element, or how to support it without.

[LlGC-szw]

All issues will be triaged for further investigation or closure by the 28 September 2023. If your issue is still relevant and would like for it be investigated further please comment by 14 September 2023.