refactor(models/chat): improve async_openai code structure and readability

[kcz358]

Summary

• Extract prepare_messages() method to separate message formatting logic, allowing subclasses to override
• Refactor complex generate_until() concurrency control from 130 lines into focused helper methods
• Add comprehensive docstrings to all new methods and classes
• Create new async_openai_qwen3_vl.py model that inherits from AsyncOpenAIChat and overrides only prepare_messages()
• Register async_openai_qwen3_vl in model registry

Changes

Code Structure Improvements

• _AdaptiveConcurrencyTracker: Dataclass for tracking adaptive concurrency statistics
• _get_initial_concurrency(): Calculate initial concurrency level
• _compute_dispatch_order(): Compute request dispatch order with prefix-aware sorting
• _process_with_retry(): Handle single request execution with retry logic
• _should_update_concurrency(): Determine when to update concurrency
• _update_concurrency(): Update concurrency based on tracked statistics
• _run_scheduling_loop(): Main async scheduling loop

Simplified generate_until()

The run() inner function is now only 8 lines, making the overall flow much clearer:

async def run():
    pbar = tqdm(...)
    current_concurrency = self._get_initial_concurrency()
    dispatch_order = self._compute_dispatch_order(requests)
    res = await self._run_scheduling_loop(requests, dispatch_order, pbar, current_concurrency)
    pbar.close()
    return res

New Model: async_openai_qwen3_vl

• Inherits from AsyncOpenAIChat with minimal code duplication
• Overrides only prepare_messages() to use to_qwen3_vl_openai_messages()
• Maintains all async concurrency control and retry logic from parent class

Files Changed

• lmms_eval/models/chat/async_openai.py: Refactored with new helper methods (+244, -127)
• lmms_eval/models/chat/async_openai_qwen3_vl.py: New file (+35 lines)
• lmms_eval/models/__init__.py: Registered new model (+1 line)
• .gitignore: Ignore scripts directory

Testing

Verified with --limit 5 test runs on both async_openai and async_openai_qwen3_vl models.

[Luodian]

Review & Changes Pushed

The refactoring of async_openai.py (extracting helper methods, _AdaptiveConcurrencyTracker, etc.) is solid — nice work cleaning up generate_until().

I've pushed a commit on top of your branch that replaces the separate async_openai_qwen3_vl model class with a message_format parameter on AsyncOpenAIChat. Here's the reasoning:

Why message_format parameter instead of a separate class

The AsyncOpenAIQwen3VLChat class only overrides prepare_messages() to swap to_openai_messages() for to_qwen3_vl_openai_messages(). That's a one-line behavioral difference. Creating a new file + class + registry entry for this has a scalability problem: when Qwen4-VL or another model needs its own message format, we'd need async_openai_qwen4_vl.py, async_openai_gemini.py, etc. — class explosion for what is fundamentally a configuration choice.

The new approach:

# Before (separate model):
python -m lmms_eval --model async_openai_qwen3_vl --model_args pretrained=...

# After (same model, parameterized):
python -m lmms_eval --model async_openai --model_args pretrained=...,message_format=qwen3_vl

Adding a new format in the future is just adding an elif in prepare_messages() — no new files, no registry changes.

Other fixes in this commit

1. f-string bug: all_response += "</{call.function.name}>" was missing the f prefix — it concatenated the literal string </{call.function.name}> instead of the variable value. Fixed to f"</{call.function.name}>".

2. DRY violation: Both the parent and child class duplicated video_kwargs construction. Extracted _build_video_kwargs() so prepare_messages() stays focused on format dispatch.

3. .gitignore: Removed duplicate scripts entry (scripts/ already covers it).