Vector Self-Checking by georgiatai · Pull Request #1038 · riscv/riscv-arch-test

georgiatai · 2026-03-04T22:57:36Z

Vector self-checking implementation
- need more testing on all instructions (passing basic arithmatics (e.g. vadd, vsub, etc.) and mask producing (e.g. vmand, etc.)
- added _INST_PTR for new failure code
moved std_vec into custom coverpoints

… into VectorTestFix

… lmul

… into VectorTestFix

… into vectorcoverage

… into zerofail

… into selfcheck

georgiatai · 2026-03-04T23:47:46Z

Received feedback in meeting, will implement and fix:

Add comments (especially assumptions for base suite and each instruction)
Check if instructions before failedtest are needed considering new failure code
Store vtype in the beginning
Change VTMP2 to MTMP2, make all VTMP/MTMP -> VT1/MT1
See if _MASK_REG can be eliminated (try li inside the macro)
vmseq.vi not applicable for mask producing instructions, change to something similar to _CMP
Change non-selfcheck to nop
Mention assumption that sail has to be preserving the old values

davidharrishmc

Verbal review with Georgia yesterday.
Generally looks good.
Replace unused code with nops in signature generation mode
Comment principles of operation much more closely.

georgiatai · 2026-03-06T01:15:08Z

Implemented feedback in meeting. Still testing different instructions but could start reviewing:

Add comments (especially assumptions for base suite and each instruction)
- done
Check if instructions before failedtest are needed considering new failure code
- still there, did not change
Store vtype in the beginning
- done
Change VTMP2 to MTMP2, make all VTMP/MTMP -> VT1/MT1
- changed VTMP2 to MTMP2 but decided to keep the TMP naming
See if _MASK_REG can be eliminated (try li inside the macro)
- changed to li inside the macro
vmseq.vi not applicable for mask producing instructions, change to something similar to _CMP
- implement branching and took out _CMP (pass MASKPROD_FLAG instead)
Change non-selfcheck to nop
- done
Mention assumption that sail has to be preserving the old values
- done in top level comment

will add target later after more testing!

georgiatai · 2026-03-06T02:43:34Z

Refer to PR #924 for the changes on RISCV_coverage_common.svh.
Somehow this was reverted in #1012 by @jacassidy. I'm not sure if there's something in this reverting that will break Vls/Vf (don't think this affects Vf but not that sure about Vls).
But for the demo on Vx, if these changes are not in the state which they are in in #924, then some coverage, e.g. widening instructions, will be missing.
In conclusion, they should be in the state of #924 for now.

jacassidy · 2026-03-06T03:35:55Z

Those changes were made because @jordancarlin said that non-vector supporting suites were failing to run coverage with hardcoded SEW support. Please fix the implementation to run with widening instructions instead of hardcoding a temporary solution and pushing it to main.

Copilot

Pull request overview

This PR introduces vector “self-checking” support by adding signature-compare macros for vector results and updating the vector test generator to emit per-test labels/coverpoint identifiers, alongside refactoring coverage templates to include std_vec locally.

Changes:

Add new RVTEST_SIGUPD_V / RVTEST_SIGUPD_V_LEN macros intended to support SELFCHECK vector comparisons and richer failure reporting.
Update vector test generation to pass _INST_PTR-style labels and coverpoint IDs (cp) through to signature update sites; add some length-suite handling.
Move std_vec into multiple custom vector coverage templates and adjust coverage common macros for supported SEWs.

Reviewed changes

Copilot reviewed 3 out of 43 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
tests/env/signature.h	Adds new vector SIGUPD macros for SELFCHECK/non-SELFCHECK behavior and length-suite comparisons.
generators/testgen/scripts/vector_testgen_common.py	Updates vector signature update emission and test labeling (`inst_ptr`), plus length-suite preload logic.
generators/testgen/scripts/vector-testgen-unpriv.py	Threads new `cp` argument through test generation; changes XLEN coverage and edge sets.
generators/coverage/templates/vector/*.sv	Adds `std_vec` coverpoint to many custom coverpoint templates (and updates headers).
framework/src/act/fcov/coverage/RISCV_coverage_common.svh	Changes how supported SEW/ELEN/LMUL capability macros are defined for coverage compilation.

Comments suppressed due to low confidence (1)

generators/testgen/scripts/vector_testgen_common.py:1673

writeVecTest emits vsetivli ... m{sig_lmul} in the priv path, but sig_lmul is now computed as lmul * size_multiplier and can be fractional (e.g., 0.5/0.25/0.125). This will generate invalid assembly like m0.5. Use getLmulFlag(sig_lmul) (or keep sig_lmul as an encoded string like f2/f4/f8) when formatting LMUL in vsetivli, and ensure callers only pass a legal LMUL encoding.

tests/env/signature.h

Copilot · 2026-03-06T07:14:58Z

tests/env/signature.h

+        LREG _TEMP_REG, 0(_SIG_PTR)                                 ;\
+        beq  _TEMP_REG, _TEMP_REG, 1f                               ;\
+    1:                                                              ;\
+        jal _LINK_REG, failedtest_##_LINK_REG##_##_TEMP_REG         ;\


The beq immediately before the jal is beq _TEMP_REG, _TEMP_REG, 1f, which forces the integer failure decoder to treat the same register as both expected and actual. failure_code.h extracts failing_reg from rs2 of this beq and then reads failing_value from the saved state of that register, so the report will misleadingly show failing_value == expected_value. Consider making this beq compare an actual scalar derived from the vector result (e.g., element 0 via vmv.x.s / vfirst result) against the expected scalar loaded from the signature, so failure reports are actionable.

more so related to error report so skipping for this PR. will fix and is on to-do

tests/env/signature.h

Copilot · 2026-03-06T07:14:59Z

generators/testgen/scripts/vector-testgen-unpriv.py

+      redgesv = [0, 1, 2, 2**xlen-1, 2**xlen-2, 2**(xlen-1), 2**(xlen-1)+1, 2**(xlen-1)-1, 2**(xlen-1)-2]
      if (xlen == 32):


redgesv in unpriv generator was trimmed to remove the alternating-bit patterns and other stress values that were previously included (e.g., walk-odd/even masks). This reduces scalar edge coverage and can hide issues in shift/logic instructions. If the intent is to limit runtime, consider keeping the patterns behind a flag (fast vs thorough) rather than removing them unconditionally.

the reduced ones are listed later, repetitive values cause multiple definitions and test to fail

Copilot · 2026-03-06T07:15:00Z

generators/testgen/scripts/vector_testgen_common.py

                if (arch == "Vls"):
-                    for effew in ["8", "16", "32", "64"]:
-                        testplans["Vls" + effew] = tp
+                    # for effew in ["8", "16", "32", "64"]:
+                    #     testplans["Vls" + effew] = tp
                    del testplans["Vls"]
                if (arch == "Vf"):


readTestplans() now deletes the Vls testplan entry without creating per-EEW variants (Vls8/16/32/64). This effectively drops all VLS-directed test generation unless those covergroups are handled elsewhere, and is a significant behavioral change that isn’t mentioned in the PR description. If VLS should still be generated, restore the per-EEW expansion or otherwise keep Vls in the returned testplans map.

deliberate choice for now

Copilot · 2026-03-06T07:15:00Z

tests/env/signature.h

+        /* Save architecture state of instruction under test (vl and vtype) */                                      \
+        csrr        _TEMP_REG, vl            ;                                                                      \
+        csrr        _TEMP_REG2, vtype        ;                                                                      \
+        /* Set vl = VLMAX for full-register comparison*/                                                            \
+        vsetvli     _LINK_REG, x0, e ## _SEW ##, m ## _LMUL ##, ta, ma ;                                            \
+        /* Load reference from signature and compute mismatch mask */                                               \


RVTEST_SIGUPD_V_LEN saves the original vl and vtype (csrr into _TEMP_REG/_TEMP_REG2) and then overwrites vtype/vl with vsetvli ... vlmax, but never restores the saved state before exiting the macro. That will leak a modified vl/vtype into subsequent test code and can cause widespread false failures. Add a restore step near the PASS/FAIL join (e.g., vsetvl using the saved vl and vtype) before incrementing _SIG_PTR / returning.

don't think it needs to be restored since each test sets its own

tests/env/signature.h

generators/testgen/scripts/vector-testgen-unpriv.py

… into selfcheck

…test into selfcheck

georgiatai · 2026-03-06T13:17:22Z

Running tests in /home/gtai/cvw/addins/riscv-arch-test-cvw/work/spike-rv32-max/elfs with command: spike --isa=rv32imafdcbv_zicbom_zicboz_zicbop_zicfilp_zicond_zicsr_zicntr_zicclsm_zifencei_zihintntl_zihintpause_zihpm_zimop_zabha_zacas_zawrs_zfa_zfbfmin_zfh_zcb_zcmop_zbc_zkn_zks_zkr_zvfbfmin_zvfbfwma_zvfh_zvbb_zvbc_zvkg_zvkned_zvknha_zvknhb_zvksed_zvksh_zvkt_sscofpmf_smcntrpmf_sstc_svinval:
        All 211 tests passed.

Implemented all copilot comments other than the ones I don't agree with (explanation provided), and all tests should pass now for Vx32... I didn't get to test the other extensions.

The only change I made in makefile is to make EXTENSIONS = Vx32. Run the following commands:

make clean
make vector-tests
make spike-rv32 --jobs

make vector-tests is set to generate only RV32 Vx tests. The whole run shouldn't take more than a few minutes.

One thing that I would be slightly concerned is regarding the changes on RISCV_common_coverage.svh. According to comments above something was breaking, but I am not sure what broke so I couldn't do more testing. If anything though, I would just revert that file when not running vector. Sorry if that's a hassle.

georgiatai added 30 commits January 27, 2026 21:53

addign vector configs

925a0ee

fix config to updated

f7e69f5

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

63fb080

… into VectorTestFix

fix vls coming up in vx report

5524fb7

add rs edges to tests and change common for all support in fractional…

7b2bb40

… lmul

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

c1da25c

… into VectorTestFix

adding vector macros to configs with V enabled

acda6eb

change default sigReg and linkReg

7ccda04

change default path of ref config in makefile

18f1d8b

riscv_coverage_common.svh comments on vector macros

301a55d

coverage retrieve for vext and vrgather.vv and vmv.s.x

5581dce

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

b3ba8b4

… into VectorTestFix

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

3bd4555

… into vectorcoverage

link reg fix in vector testgen

4c91356

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

b527d75

… into zerofail

fixes on coverpoints/tests for widening instructions

a45a18c

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

9ba2b27

… into zerofail

coverage retrieving for masking edges and macro usage fix

c85161b

bringing back Vf coverage

9e42013

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

a1cfe93

… into zerofail

delete unnecessary setup code in vector testgen

eb6d36c

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

9cb4079

… into zerofail

initial version of vector self-check

8baf8d1

merge

a2c93c3

cp_custom std_vec fix

ab60602

SIGUPD_V macro and testgen update

2bbac19

sigupd update

b73025c

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

1599af0

… into selfcheck

adding inst_ptr

e739c43

inst_ptr fix

3518500

davidharrishmc reviewed Mar 5, 2026

View reviewed changes

georgiatai added 3 commits March 5, 2026 17:01

self-checking update

453e720

sigupd_v base comment update

cfc6ed1

sigupd_v base comment update

2bd1ea8

riscv_coverage_common fix for vector

5e6ea61

update

9f49d36

jordancarlin requested a review from Copilot March 6, 2026 07:05

Merge branch 'act4' into selfcheck

c4fed81

Copilot started reviewing on behalf of jordancarlin March 6, 2026 07:06 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

georgiatai added 8 commits March 6, 2026 00:41

self-check fix

856557b

Merge branch 'act4' of https://github.com/riscv-non-isa/riscv-arch-test…

1e18330

… into selfcheck

Merge branch 'selfcheck' of https://github.com/georgiatai/riscv-arch-…

614bb66

…test into selfcheck

selfcheck working for Vx

3c14945

leaving only Vx test generation for now

a11d6c5

implement some copilot suggestions

717d50e

macro fix from copilot comment

4aa4cb2

macro alignment

db8a27d

georgiatai added 4 commits March 6, 2026 10:37

temporary fixes

12ec664

coverage and spike working

4edcf53

more instructions tested working

e757766

Vx32 all passing coverage and spike

35e8a9b

		redgesv = [0, 1, 2, 2xlen-1, 2xlen-2, 2(xlen-1), 2(xlen-1)+1, 2(xlen-1)-1, 2(xlen-1)-2]
		if (xlen == 32):

Conversation

georgiatai commented Mar 4, 2026

Uh oh!

georgiatai commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidharrishmc left a comment

Choose a reason for hiding this comment

Uh oh!

georgiatai commented Mar 6, 2026

Uh oh!

georgiatai commented Mar 6, 2026

Uh oh!

jacassidy commented Mar 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

georgiatai Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

georgiatai Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

georgiatai Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

georgiatai Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

georgiatai commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

georgiatai commented Mar 4, 2026 •

edited

Loading

georgiatai commented Mar 6, 2026 •

edited

Loading