---

PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualization

Jiajun Zhang^{1, 4, †}, Jianke Zhang^{2, †}, Zeyu Cui³, Jiaxi Yang⁵, Lei Zhang⁵
Binyuan Hui³, Qiang Liu⁴, Zilei Wang¹, Liang Wang⁴, Junyang Lin³

¹ USTC    ² THU    ³ Qwen Team, Alibaba Inc.    ⁴ CASIA    ⁵ SIAT
^† Equal Contribution

📧 Main Contact: zhangjiajun519@gmail.com

PlotCraft is a rigorous benchmark designed to evaluate the advanced data visualization capabilities of LLMs. It presents ~1k challenging tasks to assess how well models can generate and refine complex plots from natural language instructions.

Key Features:

• Comprehensive Scope: Includes 982 tasks covering 48 chart types across 8 major domains (e.g., Finance, Health, Research).
• Dual Evaluation Modes: The first benchmark to systematically test both: Single-Turn Generation: From scratch, based on an initial request.
• Multi-Turn Refinement: Iteratively debugging and enhancing existing code.
• Focus on Complexity: Tasks are designed with compositional complexity, requiring multi-panel layouts and combined chart types to test a model's spatial and logical reasoning.
• Realistic Workflow: Built from scratch using real-world datasets and zero-reference instructions (text-only), simulating a practical data analyst workflow.

Data and Scripts

https://github.com/QwenLM/Qwen3-Coder/tree/main/qwencoder-eval/instruct/PlotCraft

Installation

git clone https://github.com/QwenLM/Qwen3-Coder.git
cd Qwen2.5-Coder/qwencoder-eval/instruct/PlotCraft
conda create -n plotcraftbench python=3.13
conda activate plotcraftbench
pip install -r requirements.txt

Prepare Data

zip -F data.zip --out complete_data.zip
unzip complete_data.zip -d data

Method 1: Automated Download via Kaggle API

1. Obtain Kaggle API credentials: Navigate to your Kaggle account settings (Account tab) and select 'Create New Token'. This downloads kaggle.json containing your API credentials.

2. Configure credentials: Move kaggle.json to the appropriate location:

   • Linux/macOS: ~/.kaggle/kaggle.json
   • Windows: C:\Users\<Windows-username>\.kaggle\kaggle.json

3. Install Kaggle CLI:

pip install kaggle

4. Download datasets:

python download_datasets.py data/

Method 2: Manual Download

Alternatively, download datasets manually using the URLs specified in download_url.json files located in each subdirectory (e.g., data/<dataset-name>/download_url.json). After downloading, place all CSV and XLSX files in the root of their respective subdirectories.

Quick Start

Configure API Credentials

Edit run_single_turn.sh or run_multi_turn.sh to set your OpenAI-compatible API endpoint and keys:

API_KEY=""          # Evaluation API key
API_KEY_GEN=""      # Generation API key

Run Evaluation

Single-Turn Mode (generate plots from scratch):

bash run_single_turn.sh <model_name> <api_base_url>

Multi-Turn Mode (iterative refinement):

bash run_multi_turn.sh <model_name> <api_base_url>

Results will be saved to results_single_turn/ (single-turn) or results_multi_turn/ (multi-turn).

Citation

If you use PlotCraft in your research, please cite our paper:

@article{zhang2025plotcraft,
  title={PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualization},
  author={Zhang, Jiajun and Zhang, Jianke and Cui, Zeyu and Yang, Jiaxi and Zhang, Lei and Hui, Binyuan and Liu, Qiang and Wang, Zilei and Wang, Liang and Lin, Junyang},
  journal={arXiv preprint arXiv:2511.00010},
  year={2025}
}