feat: basic support for server-level multimodal cache

[mickqian]

Motivation

Previously the MultimodalCache is only for chunked_prefill, and the embedding's lifetime ends with prefill.

Modifications

This PR provides the basic support for a server-level multimodal cache, with LRU as evict policy

TODOs

1. make cache size and evict policy configurable

Accuracy Tests

testing with qwen/qwen2.5-VL-7B-Instruct

Before

Benchmark time: 119.34484012098983
Overall accuracy: 0.514

After

Benchmark time: 119.47181225300301
Overall accuracy: 0.514

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[merrymercy]

@mickqian please resolve the conflicts.

[zhangml]

Hello, could you please provide a command for using server-level multimodal cache?