Physics-Aware Video Frame Sorting

Recovering Temporal Order from Corrupted Action Video Clips

Welcome to the Frame Sorting repository! This project focuses on the task of predicting the correct chronological sequence of frames from structurally corrupted video clips.

Problem Statement

Short video clips extracted from action sequences have had their temporal structure deliberately corrupted—meaning the frames are shuffled completely out of order. The primary task is to develop a model that can predict the correct chronological ordering of these shuffled frames, essentially re-teaching the visual physics of order and time.

Dataset Overview

The dataset (ml-ware-26-sherlock-files.zip) consists of thousands of .mp4 video files where frames have been shuffled.

Layout:

sherlock/
 ├── train/          ← ~5,000 .mp4 files  (video_0.mp4 … video_4999.mp4)
 ├── test/           ← ~100  .mp4 files  (video_5000.mp4 …)
 └── train_labels.json

Statistics:

• Total labeled videos: ~5,611
• Frame counts: Vary across videos (min: 15, max: 300, mean: ~107.4 frames).

Label Semantics

The ground truth mappings in train_labels.json map the corrupted frame sequence back to the correct chronological order. Example:

{ "video_0": [5, 6, 7, 8, 9, 4, 3, 2, 1, 0] }

Submission Format

The final predictions must be submitted as a structured CSV file (e.g., sample_submission.csv) containing the video identifier and the predicted index array.

id,label
video_5000,"[0, 1, 2, 3, 4, ...]"

Approach & Codebase

The core workflow—spanning dataset exploration, frame extraction, visualization of distribution (e.g., frame-count distributions), and machine learning frame-sorting routines—can be found in the primary Jupyter Notebook (such as thisIsEnd (14).ipynb and variations).

Key steps implemented in the notebook:

1. Dataset Exploration & Visualization: Checking label formats, analyzing frame count distribution, and rendering CDFs.
2. Visual Sanity Checks: Reordering frames using the provided labels to ensure biological and physical chronological correctness.
3. Model Architecture: Training sequence models to recognize visual continuity and physics actions.

Getting Started

1. Ensure you have the ml-ware-26-sherlock-files.zip dataset unzipped into the respective directory.
2. Open the main Jupyter Notebook to explore the temporal ordering pipelines.
3. Generate the expected orderings to map closely identical to the structure shown in sample_submission.csv.

---

Created for the ML/Computer Vision sequence ordering task. Commits tracked under RagS8i/Frame_Sorting.