Subtask 2: Model Implementation

[rahul-tuli]

Problem Statement

Implement the FastMTP model architecture - a single MTP head with position-shared weights that performs recursive K-step prediction.

We need to understand their exact architecture and implement it within the speculators framework.

References

• Paper: FastMTP arXiv:2509.18362 - Section 3.1, Figure 1
• Reference Implementation: Tencent-BAC/FastMTP
• Existing Code: src/speculators/models/eagle3/ for patterns

What We Need

1. FastMTPConfig - Configuration class registered with the speculators framework
2. FastMTPSpeculator - Model class implementing the MTP head and recursive prediction
3. Tests - Verify model instantiation, forward pass, verifier attachment
4. Documentation - Architecture explanation and usage

Success Criteria

• Model can be loaded via SpeculatorModel.from_pretrained()
• Forward pass produces correct output shapes for K-step prediction
• Verifier attachment works (train_only mode for training)
• Integration tests pass

Notes

The architecture must match the reference implementation. Key questions: How is the MTP head structured? How are hidden states and embeddings combined? How do position-shared weights work?

[elizabetht]

Hi there! Can I work on this? @rahul-tuli @dsikka

[rahul-tuli]

Hi @elizabetht, Thank you for your interest in contributing to speculators, We have internally started working on this, since this is a blocking task for other things, once this is up, we'll appreciate your help with the other subtasks