Testplan

Testpoints

Stage V1 Testpoints

smoke

Test: rv_dm_smoke

A basic smoke test.

. Read the DTM register idcode and verify that it reflects the correct value.

  • Enable debug by setting lc_hw_debug_en to true.
  • Write to the DMI register field dmcontrol[dmactive] via JTAG and verify that the dmactive output pin reflects the same value.
  • Write to the DMI register field dmcontrol[haltreq] via JTAG and verify that the debug_req output pin reflects the same value.
  • Write to the DMI register field dmcontrol[ndmreset] via JTAG and verify that the ndmreset output pin reflects the same value.
  • Wiggle the unavailable input and read the DTM register field dmstatus[allunavail] to verify that it reflects the same value as the input signal.

jtag_dtm_csr_hw_reset

Test: rv_dm_jtag_dtm_csr_hw_reset

Verify the reset values of JTAG DTM CSRs as indicated in the RAL specification.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DTM regisers.

In these set of tests, the lc_hw_debug_en is set to true, scanmode & scan_rst_n inputs to false.

jtag_dtm_csr_rw

Test: rv_dm_jtag_dtm_csr_rw

Verify accessibility of JTAG DTM CSRs as indicated in the RAL specification.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DTM regisers.

jtag_dtm_csr_bit_bash

Test: rv_dm_jtag_dtm_csr_bit_bash

Verify no aliasing within individual bits of JTAG DTM CSR.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DTM regisers.

jtag_dtm_csr_aliasing

Test: rv_dm_jtag_dtm_csr_aliasing

Verify no aliasing within the JTAG DTM CSR address space.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DTM regisers.

jtag_dmi_csr_hw_reset

Test: rv_dm_jtag_dmi_csr_hw_reset

Verify the reset values of JTAG DMI CSRs as indicated in the RAL specification.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DMI regisers.

In these set of tests, the lc_hw_debug_en is set to true, scanmode & scan_rst_n inputs to false.

Also, the dmcontrol[dmactive] field is set to 1 at the start, to ensure CSR accesses to all other registers go through.

jtag_dmi_csr_rw

Test: rv_dm_jtag_dmi_csr_rw

Verify accessibility of JTAG DMI CSRs as indicated in the RAL specification.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DMI regisers.

jtag_dmi_csr_bit_bash

Test: rv_dm_jtag_dmi_csr_bit_bash

Verify no aliasing within individual bits of JTAG DMI CSR.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DMI regisers.

jtag_dmi_csr_aliasing

Test: rv_dm_jtag_dmi_csr_aliasing

Verify no aliasing within the JTAG DMI CSR address space.

See hw/dv/tools/dvsim/testplans/csr_testplan.hjson for more description, This follows the same approach, but applies to the JTAG DMI regisers.

jtag_dmi_cmderr_busy

Test: rv_dm_cmderr_busy

Verify that attempt to generate command error result in cmderr field of abstractcs register are set.

  • write to the command, abstractcs or abstractauto registers when an abstract command is executing.OR
  • Read or write to one of the data or progbuf registers, when an abstract command is executing.
  • Verify that the cmderr field of abstractcs register is set to 1.

jtag_dmi_cmderr_not_supported

Test: rv_dm_cmderr_not_supported

Verify that attempt to generate command error result in cmderr field of abstractcs register are set.

  • Try to execute abstract command that is not supported like Quick Access & Access Memory command.
  • Verify that the cmderr field of abstractcs register is set to 2.

cmderr_exception

Test: rv_dm_cmderr_exception

Verify that tha cmderr should set to 3, if an excepton occurred while executing the command.

-Write into memory mapped register called EXCEPTION(0x118) through tl-device interface. -Verify that the cmderr field of abstractcs register is set to 3.

mem_tl_access_resuming

Test: rv_dm_mem_tl_access_resuming

Verify whether the selected hart has been resumed or not.

-Write into memory mapped register called RESUMING(0x110) through tl-device interface. -Verify that the anyresumeack and allresumeack bits of the dmstatus register are set. -After that, set the resumereq bit of dmcontrol register. -Verify that the anyresumeack and allresumeack bits of the dmstatus register are cleared.

mem_tl_access_halted

Test: rv_dm_mem_tl_access_halted

Verify whether the selected hart has been halted or not.

  • Set the haltreq bit of dmcontrol register.
  • Write into memory mapped register called HALTED(0x100) through tl-device(mem_tl_if) interface.
  • Verify that the anyhalted and allhalted bits of the dmstatus register are set.

cmderr_halt_resume

Test: rv_dm_cmderr_halt_resume

Verify that attempt to generate command error result in cmderr field of abstractcs register are set.

  • Try to execute abstract command when hart is not in halt/resume state.
  • Assert unavailable signal through core interface.
  • The abstract command could not be execute.
  • Verify that the cmderr field of abstractcs register is set to 4.

dataaddr_rw_access

Test: rv_dm_dataaddr_rw_access

Verify the read write access of the data registers through the memory mapped DataAddr register.

  • Write random data to the shadowed data registers via the corresponding DataAddr registers.
  • Read back the written data and verify that it matches the expected values.

csr_hw_reset

Test: rv_dm_csr_hw_reset

Verify the reset values as indicated in the RAL specification.

  • Write all CSRs with a random value.
  • Apply reset to the DUT as well as the RAL model.
  • Read each CSR and compare it against the reset value. it is mandatory to replicate this test for each reset that affects all or a subset of the CSRs.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_rw

Test: rv_dm_csr_rw

Verify accessibility of CSRs as indicated in the RAL specification.

  • Loop through each CSR to write it with a random value.
  • Read the CSR back and check for correctness while adhering to its access policies.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_bit_bash

Test: rv_dm_csr_bit_bash

Verify no aliasing within individual bits of a CSR.

  • Walk a 1 through each CSR by flipping 1 bit at a time.
  • Read the CSR back and check for correctness while adhering to its access policies.
  • This verify that writing a specific bit within the CSR did not affect any of the other bits.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_aliasing

Test: rv_dm_csr_aliasing

Verify no aliasing within the CSR address space.

  • Loop through each CSR to write it with a random value
  • Shuffle and read ALL CSRs back.
  • All CSRs except for the one that was written in this iteration should read back the previous value.
  • The CSR that was written in this iteration is checked for correctness while adhering to its access policies.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_mem_rw_with_rand_reset

Test: rv_dm_csr_mem_rw_with_rand_reset

Verify random reset during CSR/memory access.

  • Run csr_rw sequence to randomly access CSRs
  • If memory exists, run mem_partial_access in parallel with csr_rw
  • Randomly issue reset and then use hw_reset sequence to check all CSRs are reset to default value
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.

regwen_csr_and_corresponding_lockable_csr

Tests:

  • rv_dm_csr_rw
  • rv_dm_csr_aliasing

Verify regwen CSR and its corresponding lockable CSRs.

  • Randomly access all CSRs
  • Test when regwen CSR is set, its corresponding lockable CSRs become read-only registers

Note:

  • If regwen CSR is HW read-only, this feature can be fully tested by common CSR tests - csr_rw and csr_aliasing.
  • If regwen CSR is HW updated, a separate test should be created to test it.

This is only applicable if the block contains regwen and locakable CSRs.

mem_walk

Test: rv_dm_mem_walk

Verify accessibility of all memories in the design.

  • Run the standard UVM mem walk sequence on all memories in the RAL model.
  • It is mandatory to run this test from all available interfaces the memories are accessible from.

mem_partial_access

Test: rv_dm_mem_partial_access

Verify partial-accessibility of all memories in the design.

  • Do partial reads and writes into the memories and verify the outcome for correctness.
  • Also test outstanding access on memories

Stage V2 Testpoints

idcode

Test: rv_dm_smoke

Verify that the JTAG IDCODE reads the correct value.

  • Set a different JTAG ID code an rv_dm design parameter.
  • Read the IDCODE register in JTAG DTM register space, via JTAG.
  • Verify it reads back what was set.

jtag_dtm_hard_reset

No Tests Implemented

Verify that the debugger can camcel an on-going DMI transaction with a hard reset.

  • Perform a random legal DMI write to a chosen DMI register and immediately write to dtmcs[dmihardreset] bit.
  • Read the target DMI register to verify the write did not succeed.

jtag_dtm_idle_hint

No Tests Implemented

Verify that the JTAG debugger does not need to stay any longer than the advertized idle time indicated in dtmcs register.

  • Read the dtmcs[idle] value and configure the JTAG DMI register adapter to insert that many delay between transactions.
  • Issue random legal DMI accesses back to back and ensure that all of them complete successfully, without dmistat reflecting an error.

jtag_dmi_failed_op

No Tests Implemented

Verify that a failed DMI op is reflected properly in dtmcs register.

  • Issue a random legal DMI write to a chosen DMI register.
  • Before that access completes, issue another random legal DMI access.
  • Read the dtmcs[dmistat] register to verify busy error.
  • Clear it by writing to dtmcs[dmireset] register.
  • Poll the DMI access for completion and verify that the first write completed successfully by doing a read-check.
  • TBO - unclear how to generate other types of failed DMI transactions.

jtag_dmi_dm_inactive

No Tests Implemented

Verify that writes to DMI registers other than dmcontrol[dmactive] and ignored and they all retain the reset values on reads.

  • Perform random writes to DMI registers while dmcontrol[dmactive] is 0 (exclude it).
  • Read all DMI registers back and verify they reflect POR values.

sba

Tests:

  • rv_dm_sba_tl_access
  • rv_dm_delayed_resp_sba_tl_access

Verify all types of accesses from JTAG to SBA.

  • Enable debug by setting lc_hw_debug_en to true and activate the dm
  • Write to SBA registers to inject randomized 8/16/32bit writes and reads.
  • Randomize readonaddr and readondata settings and ensure those have the intended effect.
  • Generate sbbusy = 1 scenario by picking a very large response delay, since JTAG accesses are much slower (SBA TL accesses tend to complete faster than JTAG can read the SBCS register).

bad_sba

Test: rv_dm_bad_sba_tl_access

Verify attempts to make bad SBA accesses results in the error sticky bits getting set.

  • Enable debug by setting lc_hw_debug_en to true and activate the dm.
  • Build on the previous sba test and randomly inject errors in the following ways:
    • Inject a response error on the TL response channel (d_error = 1) and verify that the sberror bit is set to 2.
    • Inject a randomized SBA access that is not aligned to the size of the transfer and verify that the sberror bit is set to 3.
    • Inject a randomized SBA access that is greater that the supported transfer size and verify that the sberror bit is set to 4.
    • Inject an integrity error on the TL response channel and verify that the sberror bit is set to 7. Verify that an alert is seen. Reset the DUT.
    • Inject a new SBA traffic before the previous one completes - verify that the sbbusyerror asserts. This is achieved by setting a large TL response delay (TODO).
    • Have more than one of these types of errors trigger at the same time.

Note: The following error scenarios are not supported by the design:

  • pulp-platform/riscv-dbg#128: response timeout (sberror = 1) is not implemented.

sba_autoincrement

Test: rv_dm_autoincr_sba_tl_access

Verify the address autoincrement functionality of SBA.

  • Enable debug by setting lc_hw_debug_en to true and activate the dm.
  • Write to the SBCS register to inject randomized 8/16/32bit writes. Set the autoincrement bit.
  • Pick a random address and write to sbaddress0 register.
  • Write a random value to sbdata0, repeating a randomized number of times.
  • Each write should trigger a new SBA TL write access with the address incremented with the transfer size. Verify that the SBA TL access shows up.
  • Repeat the same procedure for reads. For reads, set the readondata register and read the sbdata0 a randomized number of times - each read should trigger a new SBA TL read access.
  • Intermittently also generate busy and error conditions.

jtag_dmi_debug_disabled

No Tests Implemented

Verify that the JTAG DMI register space cannot be accessed if pinmux_hw_debug_en is a non strict true value.

  • Set pinmux_hw_debug_en to true and activate the dm.
  • Write a known value to a randomly picked DMI CSR.
  • Set pinmux_hw_debug_en to non-true value.
  • Write a different value to the same DMI CSR.
  • Attempt to read that DMI CSR - it should read all 0s.
  • Note that the above steps are rendered moot because the JTAG DTM itself will not receive any incoming JTAG transactions.
  • Set pinmux_hw_debug_en to true and again read the DMI CSR - it should reflect the originally written value.

sba_debug_disabled

No Tests Implemented

Verify that access to SBA interface is disabled if lc_hw_debug_en is set to not strict true value.

  • Set lc_hw_debug_en to true and activate the dm.
  • Attempt to inject a legal randomized SBA access - verify that it comes through.
  • Set lc_hw_debug_en to non-true value and activate the dm.
  • Attempt to inject a legal randomized SBA access - verify that no SBA TL accesses appear on the TL interface.
  • Reapeat, a random number of times.
  • Verify via assertion checks, no transactions were seen on the SBA TL interface.

ndmreset_req

No Tests Implemented

Verify that the debugger can issue a non-debug reset to the rest of the system.

  • Set lc_hw_debug_en / pinmux_hw_debug_en to true value and activate the dm.
  • Pick a random set of write/readable CSRs in all 3 RALs (JTAG DMI, regs RAL and debug mem RAL). Write known values to them.
  • Write the dmcontrol register to assert ndmreset - verify that the ndmreset output stays asserted.
  • Mimic the CPU going into reset by asserting unavailable input, asserting rst_ni and setting lc_hw_debug_en to false. Keep pinmux_hw_debug_en set to true to keep the JTAG side open while the ndmreset is in progress.
  • Read the dmstatus register to verify allunavail is asserted.
  • De-assert the ndmreset by writing 0 to the dmcontrol register field, verify the ndmreset output deasserted as well.
  • Mimic the CPU going out of reset by de-asserting unavailable input and rst_ni after some delay. Set lc_hw_debug_en to true again as well.
  • Read the previously written registers back and verify that they reflect the previously written value, proving that ndmreset assertion had no effect on them.

hart_unavail

Test: rv_dm_hart_unavail

Verify that the debugger can read the status of unavailable hart

  • Set lc_hw_debug_en to true value and activate the dm.
  • Randomly assert and de-assert the unavailable input (indicates the CPU is under reset). This ideally happens when ndmreset is issued by the debugger.
  • Periodically issue reads to dmstatus register to verify allunavail matches the unavailable input value

tap_ctrl_transitions

Tests:

  • rv_dm_tap_fsm
  • rv_dm_tap_fsm_rand_reset

Verify all JTAG TAP controller FSM transitions

This test should verify that the standardized JTAG TAP FSM is working correctly.

stress_all

Test: rv_dm_stress_all

A ‘bug hunt’ test that stresses the DUT to its limits.

  • Combine above sequences in one test and run as many of them as possible in parallel. Run the rest sequentially.
  • Randomly inject a full device reset intermittently.

alert_test

Test: rv_dm_alert_test

Verify common alert_test CSR that allows SW to mock-inject alert requests.

  • Enable a random set of alert requests by writing random value to alert_test CSR.
  • Check each alert_tx.alert_p pin to verify that only the requested alerts are triggered.
  • During alert_handshakes, write alert_test CSR again to verify that: If alert_test writes to current ongoing alert handshake, the alert_test request will be ignored. If alert_test writes to current idle alert handshake, a new alert_handshake should be triggered.
  • Wait for the alert handshakes to finish and verify alert_tx.alert_p pins all sets back to 0.
  • Repeat the above steps a bunch of times.

tl_d_oob_addr_access

Test: rv_dm_tl_errors

Access out of bounds address and verify correctness of response / behavior

tl_d_illegal_access

Test: rv_dm_tl_errors

Drive unsupported requests via TL interface and verify correctness of response / behavior. Below error cases are tested bases on the TLUL spec

  • TL-UL protocol error cases
    • invalid opcode
    • some mask bits not set when opcode is PutFullData
    • mask does not match the transfer size, e.g. a_address = 0x00, a_size = 0, a_mask = 'b0010
    • mask and address misaligned, e.g. a_address = 0x01, a_mask = 'b0001
    • address and size aren’t aligned, e.g. a_address = 0x01, a_size != 0
    • size is greater than 2
  • OpenTitan defined error cases
    • access unmapped address, expect d_error = 1 when devmode_i == 1
    • write a CSR with unaligned address, e.g. a_address[1:0] != 0
    • write a CSR less than its width, e.g. when CSR is 2 bytes wide, only write 1 byte
    • write a memory with a_mask != '1 when it doesn’t support partial accesses
    • read a WO (write-only) memory
    • write a RO (read-only) memory
    • write with instr_type = True

tl_d_outstanding_access

Tests:

  • rv_dm_csr_hw_reset
  • rv_dm_csr_rw
  • rv_dm_csr_aliasing
  • rv_dm_same_csr_outstanding

Drive back-to-back requests without waiting for response to ensure there is one transaction outstanding within the TL device. Also, verify one outstanding when back- to-back accesses are made to the same address.

tl_d_partial_access

Tests:

  • rv_dm_csr_hw_reset
  • rv_dm_csr_rw
  • rv_dm_csr_aliasing
  • rv_dm_same_csr_outstanding

Access CSR with one or more bytes of data. For read, expect to return all word value of the CSR. For write, enabling bytes should cover all CSR valid fields.

Stage V2S Testpoints

tl_intg_err

Tests:

  • rv_dm_tl_intg_err
  • rv_dm_sec_cm

Verify that the data integrity check violation generates an alert.

  • Randomly inject errors on the control, data, or the ECC bits during CSR accesses. Verify that triggers the correct fatal alert.
  • Inject a fault at the onehot check in u_reg.u_prim_reg_we_check and verify the corresponding fatal alert occurs

sec_cm_bus_integrity

No Tests Implemented

Verify the countermeasure(s) BUS.INTEGRITY.

sec_cm_lc_hw_debug_en_intersig_mubi

No Tests Implemented

Verify the countermeasure(s) LC_HW_DEBUG_EN.INTERSIG.MUBI.

sec_cm_dm_en_ctrl_lc_gated

No Tests Implemented

Verify the countermeasure(s) DM_EN.CTRL.LC_GATED.

sec_cm_sba_tl_lc_gate_fsm_sparse

No Tests Implemented

Verify the countermeasure(s) SBA_TL_LC_GATE.FSM.SPARSE.

sec_cm_mem_tl_lc_gate_fsm_sparse

No Tests Implemented

Verify the countermeasure(s) MEM_TL_LC_GATE.FSM.SPARSE.

sec_cm_exec_ctrl_mubi

No Tests Implemented

Verify the countermeasure(s) EXEC.CTRL.MUBI.

Stage V3 Testpoints

stress_all_with_rand_reset

Test: rv_dm_stress_all_with_rand_reset

This test runs 3 parallel threads - stress_all, tl_errors and random reset. After reset is asserted, the test will read and check all valid CSR registers.

Covergroups

regwen_val_when_new_value_written_cg

Cover each lockable reg field with these 2 cases:

  • When regwen = 1, a different value is written to the lockable CSR field, and a read occurs after that.
  • When regwen = 0, a different value is written to the lockable CSR field, and a read occurs after that.

This is only applicable if the block contains regwen and locakable CSRs.

sba_access_cg

Cover all possible types of accesses over SBA.

  • cp: reads and write accesses
  • cp: supported transfer sizes
  • cp: auto-increment address with back-to-back acesses
  • cp: for reads, cover readonaddr and readondata settings
  • cp: sberror cases with unaligned address and unsupported size
  • cp: sbbusyerror case - attempt new access before the previous one completed
  • cr: cross all of the above

tl_errors_cg

Cover the following error cases on TL-UL bus:

  • TL-UL protocol error cases.
  • OpenTitan defined error cases, refer to testpoint tl_d_illegal_access.

tl_intg_err_cg

Cover all kinds of integrity errors (command, data or both) and cover number of error bits on each integrity check.

Cover the kinds of integrity errors with byte enabled write on memory if applicable: Some memories store the integrity values. When there is a subword write, design re-calculate the integrity with full word data and update integrity in the memory. This coverage ensures that memory byte write has been issued and the related design logic has been verfied.