[Benchmark] Add GEMM & MoE kernel bench

[Sunny-bot1]

Motivation

Profile kernel performance for various quantized kernels and add layer-level unit tests to more accurately reflect the kernel’s performance within the model.

Modifications

quantized gemm kernel

• WINT4
• WINT8
• block_wise_fp8
• ...

quantized moe

• WINT4
• WINT8
• block_wise_fp8
• w4a8
• w4afp8
• ...

parallel method

• TP
• EP

Usage or Command

OpPerformanceTester Usage

        tester = OpPerformanceTester(
            op_name="w4a8-moe",
            op_fn=fused_moe,
            num_layers=self.model_config.num_hidden_layers,
            weight_size=weight_size,
            gate=gating,
        )

        tester.benchmark(
            input_size=self.model_config.hidden_size,
            batch_sizes=[10, 20, 40, 60, 80, 100, 128],
        )

profile results

python tests/layers/test_quantized_linear.py

Input Size: 8192, Output Size: 10240
======== qkv_proj-block_wise_fp8 Performance ========
Batch Size      Last 5 Times (us)                        Last Time (us)  TFlops          TB/s           
1               [34.26 34.26 34.26 34.26 34.26]          34.26           4.9             2.4            
8               [34.07 34.07 34.07 34.07 34.07]          34.07           39.4            2.5            
16              [34.07 34.07 34.07 34.07 34.07]          34.07           78.8            2.5            
32              [34.07 34.07 34.07 33.89 34.07]          34.07           157.6           2.5            
128             [38.52 38.52 38.52 38.52 38.52]          38.52           557.5           2.2            
Input Size: 8192, Output Size: 8192
======== o_proj-block_wise_fp8 Performance ========
Batch Size      Last 5 Times (us)                        Last Time (us)  TFlops          TB/s           
1               [30.93 30.93 30.93 30.93 30.93]          30.93           4.3             2.2            
8               [30.19 30.19 30.19 30.19 30.19]          30.19           35.6            2.2            
16              [30. 30. 30. 30. 30.]                    30.0            71.6            2.2            
32              [28.33 28.33 28.33 28.33 28.33]          28.33           151.6           2.4            
128             [32.41 32.41 32.41 32.41 32.41]          32.41           530.1           2.1

EP=8

python -m paddle.distributed.launch --log_dir log  --gpus "0,1,2,3,4,5,6,7" tests/layers/test_w4a8_moe.py

Accuracy Tests

pass

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!