Testplan

Testpoints

Stage V1 Testpoints

smoke

Test: kmac_smoke

KMAC smoke test will contain a number of rounds, and acts as a base for many other tests. In each round, we run a full KMAC hashing operation:

  • Randomly enable interrupts, operation mode (SHA3/SHAKE/CSHAKE/KMAC), key length (if applicable), constraining all settings to be legal.
  • Set function name to “KMAC” and set customization string as empty if applicable.
  • Randomly set endianness of input msg and internal keccak state.
  • Randomly provide a sideloaded key, do not set cfg.sideload.
  • Set output length between 1-keccak_rate bytes if applicable.
  • Randomly select either SW or EDN as the source of entropy
  • Trigger KMAC to start absorbing input message.
    • During absorption stage randomly read from STATE window, expect 0.
  • Write message to MSG_FIFO window, maximum length of 32 bytes.
  • Check SHA3 engine status.
  • Trigger KMAC to finish absorbing stage, check kmac_done is set and status.squeeze is set.
  • Read output digest, and compare against output from reference C++ model.
    • In masked configuration: both shares are XORed to get unmasked result digest.
    • In unmasked configuration: Share1 is the result digest, Share2 should be 0.
  • Signal cmd.done to tell KMAC to clear internal state.
  • Try reading output digest again, confirm that it is 0.

This test, and all other tests in the testplan, will be checked for correctness in two different ways:

  • first, a DPI-C model is used to check that the correct digest value is produced
  • a cycle-accurate model is implemented in the scoreboard to provide constant checks during each hash operation and ensure that internal state is being updated correctly

csr_hw_reset

Test: kmac_csr_hw_reset

Verify the reset values as indicated in the RAL specification.

  • Write all CSRs with a random value.
  • Apply reset to the DUT as well as the RAL model.
  • Read each CSR and compare it against the reset value. it is mandatory to replicate this test for each reset that affects all or a subset of the CSRs.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_rw

Test: kmac_csr_rw

Verify accessibility of CSRs as indicated in the RAL specification.

  • Loop through each CSR to write it with a random value.
  • Read the CSR back and check for correctness while adhering to its access policies.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_bit_bash

Test: kmac_csr_bit_bash

Verify no aliasing within individual bits of a CSR.

  • Walk a 1 through each CSR by flipping 1 bit at a time.
  • Read the CSR back and check for correctness while adhering to its access policies.
  • This verify that writing a specific bit within the CSR did not affect any of the other bits.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_aliasing

Test: kmac_csr_aliasing

Verify no aliasing within the CSR address space.

  • Loop through each CSR to write it with a random value
  • Shuffle and read ALL CSRs back.
  • All CSRs except for the one that was written in this iteration should read back the previous value.
  • The CSR that was written in this iteration is checked for correctness while adhering to its access policies.
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.
  • Shuffle the list of CSRs first to remove the effect of ordering.

csr_mem_rw_with_rand_reset

Test: kmac_csr_mem_rw_with_rand_reset

Verify random reset during CSR/memory access.

  • Run csr_rw sequence to randomly access CSRs
  • If memory exists, run mem_partial_access in parallel with csr_rw
  • Randomly issue reset and then use hw_reset sequence to check all CSRs are reset to default value
  • It is mandatory to run this test for all available interfaces the CSRs are accessible from.

regwen_csr_and_corresponding_lockable_csr

Tests:

  • kmac_csr_rw
  • kmac_csr_aliasing

Verify regwen CSR and its corresponding lockable CSRs.

  • Randomly access all CSRs
  • Test when regwen CSR is set, its corresponding lockable CSRs become read-only registers

Note:

  • If regwen CSR is HW read-only, this feature can be fully tested by common CSR tests - csr_rw and csr_aliasing.
  • If regwen CSR is HW updated, a separate test should be created to test it.

This is only applicable if the block contains regwen and locakable CSRs.

mem_walk

Test: kmac_mem_walk

Verify accessibility of all memories in the design.

  • Run the standard UVM mem walk sequence on all memories in the RAL model.
  • It is mandatory to run this test from all available interfaces the memories are accessible from.

mem_partial_access

Test: kmac_mem_partial_access

Verify partial-accessibility of all memories in the design.

  • Do partial reads and writes into the memories and verify the outcome for correctness.
  • Also test outstanding access on memories

Stage V2 Testpoints

long_msg_and_output

Test: kmac_long_msg_and_output

Same as the smoke test, except with a message of up to 100KB. Max firmware input size for KMAX is around 392KB, but this is too large for a DV simulation. Average input size from firmware would be around 60-100KB, so we use 100KB as a max input size for DV, but will enable easy extensibility for emulation testing where we can enable much larger messages. Allow output length to vary up to 1KB (for XOF functions). If output length is greater than keccak_rate bytes, keccak rounds will be run manually to squeeze extra output data. Set function name as “KMAC” and enable full randomization of customization string (if applicable).

burst_write

Test: kmac_burst_write

This is the same as the long_message test, except we burst-write chunks of the message into the msg_fifo, and disable intermediate status/CSR checks.

test_vectors

Tests:

  • kmac_test_vectors_sha3_224
  • kmac_test_vectors_sha3_256
  • kmac_test_vectors_sha3_384
  • kmac_test_vectors_sha3_512
  • kmac_test_vectors_shake_128
  • kmac_test_vectors_shake_256
  • kmac_test_vectors_kmac
  • kmac_test_vectors_kmac_xof

These tests drive NIST test vectors for SHA3/SHAKE/KMAC into the design and check the output against the expected digest values.

sideload

Test: kmac_sideload

Same as the smoke test, except we set cfg.sideload and provide a valid sideloaded key as well as a valid SW-provided key. KMAC should operate on the sideloaded key regardless of the cfg_shadowed.sideload field value.

app

Test: kmac_app

Test that the Keymgr/ROM/LC can all initiate a KMAC operation through the application interface - Keymgr uses KMAC hash, while ROM/LC use CShake. Use an array of kmac_app_agents to send message data to the KMAC and to control the hashing logic, and set cfg.sideload if the Keymgr is enabled. The result digest sent to the kmac_app_agent will be compared against the result from the DPI reference model. In addition, read from the STATE window afterwards and confirm that this access is blocked and will return 0.

app_with_partial_data

Test: kmac_app_with_partial_data

Basd on the kmac_app test, this test will send partial data from application interface by sending strb with values other than 8'hFF. Because the scoreboard is cycle accurate and does not support this feature, this test will not check status and intr_state registers, but will check other registers and all interface data including digest.

entropy_refresh

Test: kmac_entropy_refresh

Test entropy interface for KMAC.

This test randomly chooses to execute either a SW-controlled hash or a hash from the App interface. All configuration fields are left as is, but now we will request EDN entropy by setting the following registers with random value:

  • “cmd.entropy_req”: Request an EDN entropy
  • “cmd.hash_cnt_clear”: Clear the entropy_refresh_hash_cnt
  • “entropy_refresh_threshold_shadowed”: The threshold when KMAC should request an EDN entropy

In masked mode, scoreboard will check:

  • Register entropy_refresh_hash_cnt value
  • KMAC requests EDN at the correct time

In unmasked mode, scoreboard will check:

  • Register entropy_refresh_hash_cnt always returns 0

error

Test: kmac_error

Try several error sequences:

  • Update key/prefix/config during absorption/process/squeeze stage.
  • Write msg to msg_fifo during process/squeeze stage
  • When in KMAC mode, set the function name to not “KMAC”
  • Incorrect SHA3 control flow:
    • Issue Process/Run/Done cmds before issuing Start
    • Issue Run/Done before issuing Process
    • Issue Start after issuing Process
    • If squeezing data, issue Start/Process after issuing Run
  • Incorrect KMAC configurations (e.g. set KMAC strength as 512).
  • Provide software inputs during operation of the application interface

key_error

Test: kmac_key_error

Test kmac responses correctly when keymgr app sends request but key is not valid. Stimulus:

  • Configure kmac and send keymgr request, but did not set valid bit for keymgr key input.
  • Wait randomly clock cycles.
  • Issue err_processed, then wait for the keymgr interface handshake to finish.
  • Repeat the above sequence a few times.
  • Issue correct kmac request from app or sw interface.

Check:

  • Check error related registers including err_code, status, and interrupt.
  • Check keymgr interface responses with all zero digests and error bit is set.
  • Check kmac can resume normal functionalities after processing this error case.

edn_timeout_error

Test: kmac_edn_timeout_error

Test kmac responses correctly when EDN response timeout. Based on kmac app sequence, this sequence configures kmac to ensure an EDN timeout error by writing entropy_period.wait_timer and entropy_period.prescaler registers. Check:

  • Check error related registers including err_code, status, and interrupt.
  • Check keymgr interface responses with error bit sets to 1.
  • Check kmac can resume normal functionalities after processing this error case.
  • Check timeout error will not trigger if the entropy_mode is set to SW, or masking is disabled.

entropy_mode_error

Test: kmac_entropy_mode_error

Test kmac responses correctly when entropy mode is configured incorrectly. Based on kmac edn_timeout sequence, this sequence write incorrect ral.cfg_shadowed.entropy_mode before entropy is fetched. Check:

  • Check error related registers including err_code, status, and interrupt.
  • Check keymgr interface responses with error bit sets to 1.
  • Check kmac can resume normal functionalities after processing this error case.
  • Check timeout error will not trigger if masking is disabled.

entropy_ready_error

Test: kmac_entropy_ready_error

Test kmac responses correctly when entropy_ready field is not set. Based on kmac app sequence, this sequence does not write 1 to ral.cfg_shadowed.entropy_ready field before a kmac operation. Check:

  • If masking is enabled and the kmac operation is EDN mode:
    • If it is a SW operation, check error related registers including err_code, status, and interrupt.
    • If it is an APP operation, check keymgr interface responses with error bit sets to 1.
  • If masking is not enabled or the operation does not require entropy:
    • Check no error triggered.

lc_escalation

Test: kmac_lc_escalation

Randomly set lc_escalate_en to value that is not Off during Kmac operations.

Checks:

  • Fatal alert fires continuously until reset is issued.
  • Status fields: alert_fatal_fault is set to 1 and sha3_idle is reset to 0.
  • Kmac does not accept any SW or APP requests.
  • Digest window always output all 0s.

stress_all

Test: kmac_stress_all

  • Combine above sequences in one test to run sequentially, except csr sequence and some error tests that disabled scoreboard.
  • Randomly add reset between each sequence

intr_test

Test: kmac_intr_test

Verify common intr_test CSRs that allows SW to mock-inject interrupts.

  • Enable a random set of interrupts by writing random value(s) to intr_enable CSR(s).
  • Randomly “turn on” interrupts by writing random value(s) to intr_test CSR(s).
  • Read all intr_state CSR(s) back to verify that it reflects the same value as what was written to the corresponding intr_test CSR.
  • Check the cfg.intr_vif pins to verify that only the interrupts that were enabled and turned on are set.
  • Clear a random set of interrupts by writing a randomly value to intr_state CSR(s).
  • Repeat the above steps a bunch of times.

alert_test

Test: kmac_alert_test

Verify common alert_test CSR that allows SW to mock-inject alert requests.

  • Enable a random set of alert requests by writing random value to alert_test CSR.
  • Check each alert_tx.alert_p pin to verify that only the requested alerts are triggered.
  • During alert_handshakes, write alert_test CSR again to verify that: If alert_test writes to current ongoing alert handshake, the alert_test request will be ignored. If alert_test writes to current idle alert handshake, a new alert_handshake should be triggered.
  • Wait for the alert handshakes to finish and verify alert_tx.alert_p pins all sets back to 0.
  • Repeat the above steps a bunch of times.

tl_d_oob_addr_access

Test: kmac_tl_errors

Access out of bounds address and verify correctness of response / behavior

tl_d_illegal_access

Test: kmac_tl_errors

Drive unsupported requests via TL interface and verify correctness of response / behavior. Below error cases are tested bases on the TLUL spec

  • TL-UL protocol error cases
    • invalid opcode
    • some mask bits not set when opcode is PutFullData
    • mask does not match the transfer size, e.g. a_address = 0x00, a_size = 0, a_mask = 'b0010
    • mask and address misaligned, e.g. a_address = 0x01, a_mask = 'b0001
    • address and size aren’t aligned, e.g. a_address = 0x01, a_size != 0
    • size is greater than 2
  • OpenTitan defined error cases
    • access unmapped address, expect d_error = 1 when devmode_i == 1
    • write a CSR with unaligned address, e.g. a_address[1:0] != 0
    • write a CSR less than its width, e.g. when CSR is 2 bytes wide, only write 1 byte
    • write a memory with a_mask != '1 when it doesn’t support partial accesses
    • read a WO (write-only) memory
    • write a RO (read-only) memory
    • write with instr_type = True

tl_d_outstanding_access

Tests:

  • kmac_csr_hw_reset
  • kmac_csr_rw
  • kmac_csr_aliasing
  • kmac_same_csr_outstanding

Drive back-to-back requests without waiting for response to ensure there is one transaction outstanding within the TL device. Also, verify one outstanding when back- to-back accesses are made to the same address.

tl_d_partial_access

Tests:

  • kmac_csr_hw_reset
  • kmac_csr_rw
  • kmac_csr_aliasing
  • kmac_same_csr_outstanding

Access CSR with one or more bytes of data. For read, expect to return all word value of the CSR. For write, enabling bytes should cover all CSR valid fields.

Stage V2S Testpoints

shadow_reg_update_error

Test: kmac_shadow_reg_errors

Verify shadowed registers’ update error.

  • Randomly pick a shadowed register in the DUT.
  • Write it twice with different values.
  • Verify that the update error alert is triggered and the register value remains unchanged.
  • Verify the update_error status register field is set to 1.
  • Repeat the above steps a bunch of times.

shadow_reg_read_clear_staged_value

Test: kmac_shadow_reg_errors

Verify reading a shadowed register will clear its staged value.

  • Randomly pick a shadowed register in the DUT.
  • Write it once and read it back to clear the staged value.
  • Then write it twice with the same new value (but different from the previous step).
  • Read it back to verify the new value and ensure that the update error alert did not trigger.
  • Verify the update_error status register field remains the same value.
  • Repeat the above steps a bunch of times.

shadow_reg_storage_error

Test: kmac_shadow_reg_errors

Verify shadowed registers’ storage error.

  • Randomly pick a shadowed register in the DUT.
  • Backdoor write to shadowed or committed flops to create a storage fatal alert.
  • Check if fatal alert continuously fires until reset.
  • Verify that all other frontdoor write attempts are blocked during the storage error.
  • Verify that storage_error status register field is set to 1.
  • Reset the DUT.
  • Read all CSRs to ensure the DUT is properly reset.
  • Repeat the above steps a bunch of times.

shadowed_reset_glitch

Test: kmac_shadow_reg_errors

Verify toggle shadowed_rst_n pin can trigger storage error.

  • Randomly drive shadowed_rst_n pin to low or rst_n pin to low.
  • check if any registers have been written before the reset. If so check if storage error fatal alert is triggered.
  • Check status register.
  • Drive shadowed_rst_n pin or rst_n pin back to high.
  • If fatal alert is triggered, reset the DUT.
  • Read all CSRs to ensure the DUT is properly reset.
  • Repeat the above steps a bunch of times.

shadow_reg_update_error_with_csr_rw

Test: kmac_shadow_reg_errors_with_csr_rw

Run shadow_reg_update_error sequence in parallel with csr_rw sequence.

  • Randomly select one of the above sequences.
  • Apply csr_rw sequence in parallel but disable the csr_access_abort to ensure all shadowed registers’ write/read to be executed without aborting.
  • Repeat the above steps a bunch of times.

tl_intg_err

Tests:

  • kmac_tl_intg_err
  • kmac_sec_cm

Verify that the data integrity check violation generates an alert.

  • Randomly inject errors on the control, data, or the ECC bits during CSR accesses. Verify that triggers the correct fatal alert.
  • Inject a fault at the onehot check in u_reg.u_prim_reg_we_check and verify the corresponding fatal alert occurs

sec_cm_bus_integrity

Test: kmac_tl_intg_err

Verify the countermeasure(s) BUS.INTEGRITY.

sec_cm_lc_escalate_en_intersig_mubi

Test: kmac_lc_escalation

Verify global LC_ESCALATE_EN mubi

sec_cm_sw_key_key_masking

Test: kmac_smoke

Verify the countermeasure(s) SW_KEY.KEY.MASKING.

sec_cm_key_sideload

Test: kmac_sideload

Verify the key from KeyMgr is sideloaded.

sec_cm_cfg_shadowed_config_shadow

Test: kmac_shadow_reg_errors

Verify the countermeasure(s) CFG_SHADOWED.CONFIG.SHADOW.

sec_cm_fsm_sparse

Test: kmac_sec_cm

Verify the countermeasure(s) FSM.SPARSE.

sec_cm_ctr_redun

Test: kmac_sec_cm

Verify the countermeasure(s) CTR.REDUN.

sec_cm_packer_ctr_redun

Test: kmac_sec_cm

Verify the countermeasure(s) PACKER.CTR.REDUN.

sec_cm_cfg_shadowed_config_regwen

Test: kmac_smoke

Verify the countermeasure(s) CFG_SHADOWED.CONFIG.REGWEN.

sec_cm_fsm_global_esc

Test: kmac_lc_escalation

Verify the countermeasure(s) FSM.GLOBAL_ESC.

sec_cm_fsm_local_esc

Test: kmac_sec_cm

Verify the countermeasure(s) FSM.LOCAL_ESC.

sec_cm_logic_integrity

Test: N/A

Verify the countermeasure(s) LOGIC.INTEGRITY.

sec_cm_absorbed_ctrl_mubi

Test: kmac_mubi

Verify the countermeasure(s) ABSORBED.CTRL.INTEGRITY

sec_cm_sw_cmd_ctrl_sparse

Test: kmac_smoke

Verify the countermeasure(s) SW_CMD.CTRL.INTEGRITY

Stage V3 Testpoints

throughput

Test: kmac_throughput

Measure the throughput of the various hashing calculations and make sure they correspond to the expected throughput range for the design.

stress_all_with_rand_reset

Test: kmac_stress_all_with_rand_reset

This test runs 3 parallel threads - stress_all, tl_errors and random reset. After reset is asserted, the test will read and check all valid CSR registers.

Covergroups

app_cg

Covers several scenarios related to the app interface:

  • A single data beat is sent
  • All partial data lengths have been seen (only applies to the last data beat)
  • Errors are reported through this interface
  • Done signal is sent while keccak rounds are currently active/inactive

Note that this covergroup will be duplicated once per app interface.

app_sw_cfg_cg

Covers several scenarios related to the app interface and cfg_shadowed register value:

  • Hash_mode field value.
  • Kmac_en field value.
  • Kstrength field value.

These sw register settings should not affect kmac app interface calculation. Note that this covergroup will be duplicated once per app interface.

cmd_process_cg

Covers that the KMAC can handle seeing a CmdProcess command during the following sets of scenarios:

  • various msgfifo status (full/empty/in between)
  • while keccak rounds are currently active/inactive

config_cg

Covers that all valid configuration settings for the KMAC have been tested. Individual config settings that will be covered include:

  • hashing mode (sha3/shake/kmac)
  • security strength (128/224/256/384/512)
  • key length (128/192/256/384/512)
  • message endianness enable/disable
  • digest endianness enable/disable
  • XOF mode when using KMAC hashing All valid combinations of the above will also be crossed.

entropy_timer_cg

Cover the values for the entropy_period register’s prescaler and wait_timer fields. Cross these field ranges with whether the entropy EDN mode is on or off.

error_cg

Covers all error scenarios:

  • ErrKeyNotValid: covers that secret key is invalid when KeyMgr initiates App operation
  • ErrSwPushedMsgFifo: covers that SW writes the msgfifo while App interface is active
  • ErrSwIssuedCmdInAppActive: covers that SW writes all possible commands to the KMAC while App interface is active
  • ErrWaitTimerExpired: covers that the KMAC timed out while waiting for EDN entropy
  • ErrIncorrectEntropyMode: covers that incorrect entropy modes are detected by the KMAC
  • ErrUnexpectedModeStrength: covers that 128-bit strength is seen for SHA3, and all but 128/256 bit strengths are seen for XOF functions
  • ErrIncorrectFunctionName: covers that the function name is configured incorrectly when KMAC mode is enabled
  • ErrSwCmdSequence: covers that SW issues commands to the KMAC out of order

msg_len_cg

Covers various input message length ranges, to ensure that KMAC can operate successfully on different sized messages. The minimum tested msg length is 0 bytes, and the maximum length is 10_000 bytes, we will cover that an acceptable distribution of lengths has been seen, and specifically cover some corner cases (like length 0).

msgfifo_level_cg

Covers that all possible fifo statuses have been seen when running different hash operations (like sha3/shake/cshake/kmac), such as various fifo depths, fifo full, and fifo empty.

msgfifo_write_mask_cg

Covers that the msgfifo has been written using all possible TLUL masks.

output_digest_len_cg

Similar to the msg_len_cg, we also want to cover various output digest lengths to ensure that KMAC can successfully produce outputs of varying sizes. Note that this only applies to XOF functions, as SHA3 functions have a fixed output length.

prefix_range_cg

The prefix used for CSHAKE and KMAC (function_name + customization_string) are only allowed to be valid alphabet letters, or a space character. This covergroup covers that all of these characters have appeared in a prefix value.

regwen_val_when_new_value_written_cg

Cover each lockable reg field with these 2 cases:

  • When regwen = 1, a different value is written to the lockable CSR field, and a read occurs after that.
  • When regwen = 0, a different value is written to the lockable CSR field, and a read occurs after that.

This is only applicable if the block contains regwen and locakable CSRs.

sha3_status_cg

Covers that all sha3-related status fields have eventually been seen.

shadow_field_errs_cg

Cover all shadow register errors for each register field.

For all register fields within the shadowed register, this coverpoint covers the following errors:

  • Update error
  • Storage error

sideload_cg

Covers that the KMAC sees scenarios where the sideloaded key is provided and should be used (en_sideload==1, app_mode==AppKeymgr), and scenarios where the sideloaded key is provided but should not be used.

state_read_mask_cg

Covers that the state windows have been read using all possible TLUL masks.

tl_errors_cg

Cover the following error cases on TL-UL bus:

  • TL-UL protocol error cases.
  • OpenTitan defined error cases, refer to testpoint tl_d_illegal_access.

tl_intg_err_cg

Cover all kinds of integrity errors (command, data or both) and cover number of error bits on each integrity check.

Cover the kinds of integrity errors with byte enabled write on memory if applicable: Some memories store the integrity values. When there is a subword write, design re-calculate the integrity with full word data and update integrity in the memory. This coverage ensures that memory byte write has been issued and the related design logic has been verfied.