[CI][VLM]: Add UT for multimodal cache eviction

[ZhengWG]

Motivation

Restores the test_vlm_mmmu_benchmark_with_small_cache test to verify VLM MMMU cache eviction logic under memory constraints, which was affected by PR f5b3ccd.

related with #12984

Modifications

1. Readded test scenario with 5MB embedding cache and log-based eviction detection

2. Added debug logging for "Cache eviction" events in MultiModalStaticCache.set

Accuracy Tests

Test Result:

pytest test/srt/models/test_vlm_models.py -k small_cache -s

Parameter explanation:

-k small_cache - Run only tests matching the pattern "small_cache"
-s - Show print statements and disable output capturing (useful for debugging)

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @ZhengWG, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the robustness of the VLM multimodal cache by restoring and improving its unit testing. It ensures that the cache's eviction mechanism functions correctly under memory pressure, which is critical for performance and stability. The changes include re-enabling a specific test scenario and adding detailed logging to verify the eviction process.

Highlights

• Test Restoration: Reintroduced the test_vlm_mmmu_benchmark_with_small_cache to verify VLM MMMU cache eviction logic under memory constraints, which was previously affected by another PR.
• Cache Eviction Logging: Added debug logging for 'Cache eviction' events within the MultiModalStaticCache.set method, providing visibility into when and how much data is evicted.
• Eviction Detection Utility: Implemented a new helper method _detect_eviction_in_logs to programmatically parse log output and confirm that cache eviction events are occurring as expected during tests.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[ZhengWG]

@mickqian Plz take a loot at it.

[gemini-code-assist]

Code Review

This pull request effectively restores and enhances testing for the multimodal cache eviction logic. By adding a specific unit test with a small cache size and introducing debug logs for eviction events, it significantly improves the verifiability and observability of this critical feature. The implementation is solid. I've provided a couple of minor suggestions to enhance code style and readability.

[ZhengWG]

@JustinTong0323 @yuan-luo Can you help review it~

[BBuf]

/tag-and-rerun-ci

[yuan-luo]

/rerun-failed-ci

[ZhengWG]

@yuan-luo
https://github.com/sgl-project/sglang/actions/runs/20423993529/job/58702021377?pr=13370
The current CI timeout seems to be triggered by DeepGEMM JIT compilation and is likely unrelated to the changes in this PR:

[yuan-luo]

/rerun-failed-ci

[yuan-luo]

/rerun-failed-ci