[Data][LLM] Add support for classification and scoring models in Ray Data LLM

[kouroshHakha]

Summary

This PR adds support for sequence classification and scoring models in Ray Data LLM batch inference. Previously, only generate and embed task types were supported. This change enables users to run batch inference with models like nvidia/nemocurator-fineweb-nemotron-4-edu-classifier for educational content classification.

Motivation

vLLM supports multiple task types for different model architectures:

• generate - Text generation (LLMs)
• embed - Embedding models
• classify - Sequence classification models
• score - Cross-encoder scoring models

Ray Data LLM only supported generate and embed. This PR adds classify and score to enable batch inference with classification models.

Usage

from ray.data.llm import vLLMEngineProcessorConfig, build_processor

config = vLLMEngineProcessorConfig(
    model_source="nvidia/nemocurator-fineweb-nemotron-4-edu-classifier",
    task_type="classify",  # NEW: use 'classify' for classification models
    engine_kwargs={"max_model_len": 512},
    apply_chat_template=False,
    detokenize=False,
)

processor = build_processor(
    config,
    preprocess=lambda row: dict(prompt=row["text"]),
    postprocess=lambda row: {"score": float(row["embeddings"][0])},
)

ds = ray.data.from_items([{"text": "Sample text to classify"}])
result = processor(ds)

Testing

• New classification_example.py is automatically picked up by CI via the existing glob pattern in BUILD.bazel
• Tested locally with nvidia/nemocurator-fineweb-nemotron-4-edu-classifier

[gemini-code-assist]

Code Review

This pull request adds support for classification and scoring models to Ray Data's LLM batch inference capabilities, which previously only supported generation and embedding tasks. The changes are well-implemented, introducing classify and score task types and updating the vLLM engine stage to handle them. The documentation and examples have also been updated accordingly. I've provided a few suggestions to improve the robustness of the example code and enhance the maintainability of the core logic.

[jeffreywang-anyscale]

Looks like you addressed Gemini comments in the latest revision. LGTM.