Skip to content
This repository has been archived by the owner on Sep 6, 2021. It is now read-only.

Latest commit

 

History

History
210 lines (177 loc) · 26.6 KB

scmi_testlist.md

File metadata and controls

210 lines (177 loc) · 26.6 KB

SCMI Tests - Scenario document

Table of Contents:

Introduction

This document outlines the test scenarios and how individual test cases are implemented in the current version of the SCMI test suite.

General Rules

In writing the tests, these rules have been followed:


All test case identifiers must be unique.


Every test case compares return values against expected results that are provided by the user through the platform properties to determine a PASS or FAIL verdict. In case some protocols are not supported by particular platform, those test cases need to be skipped.


All message fields that are defined by specification as 'Reserved, Must be Zero' must be checked and reported as compliance failure if not adhered to.


The SCMI test library comprises of a collection of test suites that are grouped per protocol.


Some tests might require a pre-condition or a post-condition check. These checks are captured as part of test cases that must come immediately preceding or just after the relevant test case payload function.

A typical example for a test case that needs a precondition is the get command which has a set command as a precondition.

A typical example for a test case that needs a postcondition is a subscribe for notification command which has unsubscribe for notification command as the postcondition.

Base Protocol Tests

This section outlines the test specification for SCMI Base Protocol.

Test ID Test Intent Verification Steps Comments/Commands Used
test_b001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_b002 Query the protocol attributes. Check num_agents and num_protocols against expected value. PROTOCOL_ATTRIBUTES
test_b003 Query for mandatory command support. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_b004 Send invalid command id for base protocol.
Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_b005 Query for vendor information. Check vendor name against expected value. BASE_DISCOVER_VENDOR
test_b006 Pre-Condition: BASE_DISCOVER_SUB_VENDOR support.
Query for sub vendor information.
Check sub vendor name against expected value. PROTOCOL_MESSAGE_ATTRIBUTES
BASE_DISCOVER_SUB_VENDOR
test_b007 Query for vendor specific implementation version. Check implementation version against expected value. BASE_DISCOVER_IMPLEMENTATION_VERSION
test_b008 Query protocol list with valid and invalid skip value. 1. Check protocol list against expected value for valid skip value.
2. Check INAVLID_PARAMETERS return status for invalid skip value.
BASE_DISCOVER_LIST_PROTOCOLS
test_b009 Pre-Condition: BASE_DISCOVER_AGENT support.
Query for agent name with valid and invalid agent id.
1. Check agent name starts with "platform" when agent id = 0.
2. Check agent name with expected value for valid agent id.
3. Check agent name matches expected value when agent_id = 0xFFFFFFFF.
4. Check NOT_FOUND return status for invalid agent id.
PROTOCOL_MESSAGE_ATTRIBUTES
BASE_DISCOVER_AGENT
test_b010 Pre-Condition: BASE_NOTIFY_ERRORS support.
1. Register and Unregister for error notification from platform.
2. Send command with invalid parameters.
1. Check SUCCESS status returned for register and unregister.
2. Check INVALID_PARAMETERS return status for invalid notify_enable.
PROTOCOL_MESSAGE_ATTRIBUTES
BASE_NOTIFY_ERRORS
test_b011 Pre-Condition: BASE_SET_DEVICE_PERMISSIONS support.
1. Try setting device permissions for invalid agent.
2. Send command with invalid parameters.
3. Remove acces of a invalid device.
1. Check NOT_FOUND status is returned.
2. Check INVALID_PARAMETERS status is returned.
3. Check NOT_FOUND status is returned.
PROTOCOL_MESSAGE_ATTRIBUTES
BASE_SET_DEVICE_PERMISSIONS
test_b012 Pre-Condition: BASE_SET_DEVICE_PERMISSIONS support.
1. Deny agent access to a valid device.
2. Try accessing denied device.
3. Restore the protocol access with BASE_SET_DEVICE_PERMISSIONS.
1. Check NOT_FOUND status is returned when access denied agent.
2. Agent should be able to access device after permissions restored.
BASE_SET_DEVICE_PERMISSIONS
test_b013 Pre-Condition: BASE_SET_PROTOCOL_PERMISSIONS support.
1. Try setting protocol permission for invalid agent.
2. Try to remove access for base_protocol.
3. Send command with invalid parameters.
4. Remove access of invalid device.
1. Check NOT_FOUND status is returned.
2. Check NOT_FOUND status is returned.
3. Check INVALID_PARAMETERS is returned.
4. Check NOT_FOUND status is returned.
PROTOCOL_MESSAGE_ATTRIBUTES
BASE_SET_PROTOCOL_PERMISSIONS
test_b014 Pre-Condition: BASE_SET_PROTOCOL_PERMISSIONS support.
1. Deny agent access to a valid protocol.
2. Try accessing command of denied protocol.
3. Restore the protocol access with BASE_SET_PROTOCOL_PERMISSIONS.
1. Check NOT_FOUND status is returned when access denied protocol.
2. Agent should be able to access protocol after permissions restored.
BASE_SET_PROTOCOL_PERMISSIONS
test_b015 Pre-Condition: BASE_RESET_AGENT_CONFIGURATION support.
1. Try resetting device and protocol permissions for invalid agent.
2. Send command with invalid flags value.
1. Check NOT_FOUND status is returned.
2. Check INVALID_PARAMETERS status is returned.
PROTOCOL_MESSAGE_ATTRIBUTES
BASE_RESET_AGENT_CONFIGURATION
test_b016 Pre-Condition: BASE_SET_DEVICE_PERMISSIONS and BASE_RESET_AGENT_CONFIGURATION support.
1. Deny agent access to a valid device.
2. Try accessing denied device.
3. Restore the device access with BASE_RESET_AGENT_CONFIGURATION.
1. Check NOT_FOUND status is returned when access denied device.
2. Agent should be able to access device after permissions restored.
BASE_SET_DEVICE_PERMISSIONS
BASE_RESET_AGENT_CONFIGURATION
test_b017 Pre-Condition: BASE_SET_PROTOCOL_PERMISSIONS and BASE_RESET_AGENT_CONFIGURATION support.
1. Deny agent access to a valid protocol.
2. Try accessing command of denied protocol.
3. Restore the protocol access with BASE_RESET_AGENT_CONFIGURATION.
1. Check NOT_FOUND status is returned when access denied protocol.
2. Agent should be able to access protocol after permissions restored.
BASE_SET_PROTOCOL_PERMISSIONS
BASE_RESET_AGENT_CONFIGURATION

Power Domain Management Protocol Tests

This section outlines the test specification for SCMI Power Domain Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_p001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_p002 Query the protocol attributes. Check power domain protocol attributes against expected values. PROTOCOL_ATTRIBUTES
test_p003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_p004 1. Send invalid command id for power domain protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_p005 Query power domain attributes with valid power domain id. Check power domain attributes with expected value for valid power domain id. POWER_DOMAIN_ATTRIBUTES
test_p006 Query power domain attributes with non-existent power domain id. Check NOT_FOUND return status for non-existent power domain id. POWER_DOMAIN_ATTRIBUTES
test_p007 Pre-Condition: Power domain should support power state set.
Send command with invalid flags and power state value.
Check INVALID_PARAMETERS status is returned. POWER_STATE_SET
POWER_DOMAIN_ATTRIBUTES
test_p008 Send power state set command for not supported domain. Check NOT_SUPPORTED status is returned. POWER_STATE_SET
POWER_DOMAIN_ATTRIBUTES
test_p009 Try setting power state of non-existent power domain id Check NOT_FOUND status is returned. POWER_STATE_SET
test_p010 Query power state get for valid domain id. Check SUCCESS return status. POWER_STATE_GET
test_p011 Query power state get for non-existent power domain id. Check NOT_FOUND return status for non-existent power domain id. POWER_STATE_GET
test_p012 Pre-Condition: POWER_STATE_NOTIFY support.
Send command with invalid notify_enable.
Check INVALID_PARAMETERS status is returned. POWER_STATE_NOTIFY
POWER_DOMAIN_ATTRIBUTES
test_p013 Pre-Conditions: POWER_STATE_NOTIFY support.
Send command for not supported domain.
Check NOT_SUPPORTED status is returned. POWER_STATE_NOTIFY
POWER_DOMAIN_ATTRIBUTES
test_p014 Pre-Condition: POWER_STATE_NOTIFY support.
Enable power state notification for non-existent domain.
Check NOT_FOUND status is returned POWER_STATE_NOTIFY
test_p015 Pre-Condition: POWER_STATE_CHANGE_REQUESTED_NOTIFY support
Invoke command with invalid notify_enable.
Check INVALID_PARAMETERS status is returned. POWER_STATE_CHANGE_REQUESTED_NOTIFY
test_p016 Pre-Condition: POWER_STATE_CHANGE_REQUESTED_NOTIFY support
Set power state change notification for non-existent power domain.
Check NOT_FOUND status is returned. POWER_STATE_CHANGE_REQUESTED_NOTIFY

System Power Management Protocol Tests

This section outlines the test specification for SCMI System Power Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_s001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_s002 Query the protocol attributes. Check attributes byte is zero. PROTOCOL_ATTRIBUTES
test_s003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_s004 1. Send invalid command id for system power protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_s005 Pre-Condition: SYSTEM_POWER_STATE_SET support.
Query system power state set with invalid flag.
Check INVALID_PARAMETERS status is returned. SYSTEM_POWER_STATE_SET
test_s006 Pre-Condition: SYSTEM_POWER_STATE_GET support.
Query system power state and check state with expected value.
Check SUCCESS status is returned. SYSTEM_POWER_STATE_GET
test_s007 Pre-Condition: SYSTEM_POWER_STATE_NOTIFY support.
Query system power state notify with invalid notify_enable.
Check INVALID_PARAMETERS status is returned. SYSTEM_POWER_STATE_NOTIFY

Performance Domain Management Protocol Tests

This section outlines the test specification for SCMI Performance Domain Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_d001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_d002 Query the protocol attributes. Check performance protocol attributes against expected values. PROTOCOL_ATTRIBUTES
test_d003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_d004 1. Send invalid command id for performance protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_d005 Query performance domain attributes with valid performance domain id. Check performance domain attributes with expected values. PERFORMANCE_DOMAIN_ATTRIBUTES
test_d006 Query performance domain attributes with non-existent performance domain id. Check NOT_FOUND return status for non-existent performance domain id. PERFORMANCE_DOMAIN_ATTRIBUTES
test_d007 1. Query performance describe level for all domains.
2. Query performance describe level with invalid level index
1. Check SUCCESS status is returned.
2. Check INVALID_PARAMETERS status is returned.
PERFORMANCE_DESCRIBE_LEVELS
test_d008 Query performance describe level for non-existing performance domain. Check NOT_FOUND status is returned. PERFORMANCE_DESCRIBE_LEVELS
test_d009 1. Get current performance limits.
2. Set new performance limits.
3.Read limits and verify with new performance limits.
4. Restore the default limits.
Check SUCCESS status is returned. PERFORMANCE_LIMITS_SET
PERFORMANCE_LIMITS_GET
test_d010 1. Query with invalid maximum limit.
2. Query with invalid minimum limit.
3. Query with invalid minimum greater than maximum limit.
1 & 2. Check OUT_OF_RANGE status is returned.
3. Check INVALID_PARAMETERS status is returned.
PERFORMANCE_LIMITS_SET
test_d011 1. Query set limit with not-supported performance domain.
2. Query performance domain set limit with non-existent performance domain id.
1. Check DENIED status is returned.
2. Check NOT_FOUND status is returned.
PERFORMANCE_LIMITS_SET
test_d012 Query performance domain get limit with non-existent performance domain id. Check NOT_FOUND status is returned. PERFORMANCE_LIMITS_GET
test_d013 1. Get current performance level .
2. Set new performance level.
3.Read level and verify with new performance level.
4. Restore the default level.
Check SUCCESS status is returned. PERFORMANCE_LEVEL_SET
PERFORMANCE_LEVEL_GET
test_d014 1. Query with performance level greater than maximum limit.
2. Query with performance level less than minimum limit.
Check OUT_OF_RANGE status is returned. PERFORMANCE_LEVEL_SET
test_d015 1. Query set level with not-supported performance domain.
2. Query performance domain set level with non-existent performance domain id.
1. Check DENIED status is returned.
2. Check NOT_FOUND status is returned.
PERFORMANCE_LEVEL_SET
test_d016 Query performance domain get level with non-existent performance domain id. Check NOT_FOUND status is returned. PERFORMANCE_LEVEL_GET
test_d017 Pre-Condition: PERFORMANCE_NOTIFY_LIMITS command support.
Query with invalid notify_enable.
Check INVALID_PARAMETERS status is returned. PERFORMANCE_NOTIFY_LIMITS
test_d018 Pre-Condition: PERFORMANCE_NOTIFY_LIMITS command support.
1. Query notify limit with not-supported performance domain.
2. Query performance domain notify limit with non-existent performance domain id.
1. Check NOT_SUPPORTED status is returned.
2. Check NOT_FOUND status is returned.
PERFORMANCE_NOTIFY_LIMITS
test_d019 Pre-Condition: PERFORMANCE_NOTIFY_LEVEL command support.
Query with invalid notify_enable.
Check INVALID_PARAMETERS status is returned. PERFORMANCE_NOTIFY_LEVEL
test_d020 Pre-Condition: PERFORMANCE_NOTIFY_LEVEL command support.
1. Query notify level with not-supported performance domain.
2. Query performance domain notify level with non-existent performance domain id.
1. Check NOT_SUPPORTED status is returned.
2. Check NOT_FOUND status is returned.
PERFORMANCE_NOTIFY_LEVEL
test_d021 Pre-Condition: PERFORMANCE_DESCRIBE_FASTCHANNEL command support.
Query describe fast channel for all valid domains.
Check SUCCESS status is returned. PERFORMANCE_DESCRIBE_FASTCHANNEL
test_d022 Pre-Condition: PERFORMANCE_DESCRIBE_FASTCHANNEL command support.
Query describe fast channel with non-existent performance domain id.
Check NOT_FOUND status is returned. PERFORMANCE_DESCRIBE_FASTCHANNEL
test_d023 Pre-Condition: PERFORMANCE_DESCRIBE_FASTCHANNEL command support.
Query describe fast channel with non-existent performance message id.
Check NOT_FOUND status is returned. PERFORMANCE_DESCRIBE_FASTCHANNEL
test_d024 Pre-Condition: PERFORMANCE_DESCRIBE_FASTCHANNEL command support.
Query describe fast channel with not-supported performance domain.
Check NOT_SUPPORTED status is returned. PERFORMANCE_DESCRIBE_FASTCHANNEL
test_d025 Pre-Condition: PERFORMANCE_DESCRIBE_FASTCHANNEL command support.
Query describe fast channel with not-supported message id.
Check NOT_SUPPORTED status is returned. PERFORMANCE_DESCRIBE_FASTCHANNEL
test_d026 1. Get the domain which has limit change notify and set limit support.
2. Enable limit notification.
3. Get current performance limits for the domain.
4. Set new performance limits.
5. Check if notification of new limit is received & verify limits.
6. Disable limit change notification.
7. Restore the default limits
Check SUCCESS status is returned. PERFORMANCE_LIMITS_SET
PERFORMANCE_LIMITS_GET
PERFORMANCE_NOTIFY_LIMITS
test_d027 1. Get the domain which has level change notify and set level support.
2. Enable level notification.
3. Get current performance level for the domain.
4. Set new performance level.
5. Check if notification of new level is received & verify level.
6. Disable level change notification.
7. Restore the default level
Check SUCCESS status is returned. PERFORMANCE_LEVEL_SET
PERFORMANCE_LEVEL_GET
PERFORMANCE_NOTIFY_LEVEL

Clock Management Protocol Tests

This section outlines the test specification for SCMI Clock Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_c001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_c002 Query the protocol attributes. Check clock protocol attributes against expected values. PROTOCOL_ATTRIBUTES
test_c003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_c004 1. Send invalid command id for clock protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_c005 Query clock attributes for all system clocks. Check clock attributes are returned. CLOCK_ATTRIBUTES
test_c006 Query clock attributes with invalid clock id. Check NOT_FOUND status is returned. CLOCK_ATTRIBUTES
test_c007 1. Query clock describe rate for all system clocks.
2. Query clock describe rate with invalid rate index
1. Check SUCCESS status is returned.
2. Check OUT_OF_RANGE status is returned.
CLOCK_DESCRIBE_RATES
test_c008 Query clock describe rate for non-existing clock id. Check NOT_FOUND status is returned. CLOCK_DESCRIBE_RATES
test_c009 1. Get current clock rates.
2. Set new clock rates in synchronous mode.
3. Get clock rates and verify with new rates.
4. Restore the default clock rates.
Check SUCCESS status is returned. CLOCK_RATE_SET
CLOCK_RATE_GET
test_c010 1. Get current clock rates.
2. Set new clock rates in asynchronous mode.
3. Wait for delayed response and verify with new rates.
4. Restore the default clock rates.
Check SUCCESS status is returned. CLOCK_RATE_SET
CLOCK_RATE_GET
test_c011 1. Query set clock rate with invalid rates.
2. Query set clock rate with invalid flags.
Check INVALID_PARAMETERS status is returned. CLOCK_RATE_SET
test_c012 Set clock rate for invalid clock id. Check NOT_FOUND status is returned. CLOCK_RATE_SET
test_c013 Query clock rates for all system clocks. Check clock rates are returned. CLOCK_RATE_GET
test_c014 Query clock rate for invalid clock id. Check NOT_FOUND status is returned. CLOCK_RATE_GET
test_c015 Configure clock with invalid attributes. Check INVALID_PARAMETERS status is returned. CLOCK_CONFIG_SET
test_c016 Configure clock for invalid clock id. Check NOT_FOUND status is returned. CLOCK_CONFIG_SET
test_c017 1.Configure clock device with new state.
2. Get clock attributes and verify clock device state.
3. Restore the default clock device state
Check SUCCESS status is returned. CLOCK_CONFIG_SET
CLOCK_ATTRIBUTES

Sensor Management Protocol Tests

This section outlines the test specification for SCMI Sensor Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_m001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_m002 Query the protocol attributes. Check num_sensors against expected value. PROTOCOL_ATTRIBUTES
test_m003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_m004 1. Send invalid command id for sensor protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_m005 1. Query sensor descriptor.
2. Query sensor descriptor with invalid descriptor index.
1. Check descriptor values against expected values.
2.Check INVALID_PARAMETERS status is returned.
SENSOR_DESCRIPTION_GET
test_m006 Pre-Condition: SENSOR_TRIP_POINT_NOTIFY support.
Query sensor trip point notify with invalid event control.
Check INVALID_PARAMETERS status is returned. SENSOR_TRIP_POINT_NOTIFY
test_m007 Pre-Condition: SENSOR_TRIP_POINT_NOTIFY support.
Query sensor trip point notify with invalid sensor id.
Check NOT_FOUND status is returned. SENSOR_TRIP_POINT_NOTIFY
test_m008 Pre-Condition: SENSOR_TRIP_POINT_CONFIG support.
1. Query trip point config with incorrect reserve bits.
2. Query trip point config with invalid trip point id
Check INVALID_PARAMETERS status is returned. SENSOR_TRIP_POINT_CONFIG
test_m009 Pre-Condition: SENSOR_TRIP_POINT_CONFIG support.
Query trip point config with invalid sensor id
Check NOT_FOUND status is returned. SENSOR_TRIP_POINT_CONFIG
test_m010 Pre-Condition: SENSOR_TRIP_POINT_CONFIG support.
Query trip point config with valid parameters
Check SUCCESS status is returned. SENSOR_TRIP_POINT_CONFIG
test_m011 Pass invalid flags value for sensor read. Check INVALID_PARAMETERS status is returned. SENSOR_READING_GET
test_m012 Read sensor data for invalid sensor id. Check NOT_FOUND status is returned. SENSOR_READING_GET
test_m013 Read sensor data in sync mode. Check SUCCESS status is returned. SENSOR_READING_GET
test_m014 Pre-Condition: Sensor should support async read.
Read sensor data in async mode.
1. Check SUCCESS status is returned.
2. Wait for delayed response
SENSOR_READING_GET
test_m015 Read sensor data in async mode for unsupported sensors. Check NOT_SUPPORTED status is returned. SENSOR_READING_GET

Reset Management Protocol Tests

This section outlines the test specification for SCMI Reset Management Protocol.

Test ID Test Intent Verification Step Comments/Commands Used
test_r001 Query the protocol version information. Check Version against expected value. PROTOCOL_VERSION
test_r002 Query the protocol attributes. Check num of reset domains against expected value. PROTOCOL_ATTRIBUTES
test_r003 Query for mandatory command availability. Check command implementation status. PROTOCOL_MESSAGE_ATTRIBUTES
test_r004 1. Send invalid command id for reset protocol.
2. Query the protocol message attributes with invalid msg_id.
Check NOT_FOUND status is returned. PROTOCOL_MESSAGE_ATTRIBUTES
test_r005 Query reset domain attributes. Check attributes values against expected values. RESET_DOMAIN_ATTRIBUTES
test_r006 Query reset domain attributes with invalid domain id. Check NOT_FOUND status is returned. RESET_DOMAIN_ATTRIBUTES
test_r007 Query reset protocol reset command with invalid domain id. Check NOT_FOUND status is returned. RESET
test_r008 1. Query reset protocol reset command with invalid reserved bits.
2. Query reset protocol reset command with invalid flags.
1. Check INVALID_PARAMETERS status is returned.
2. Check NOT_SUPPORTED status is returned.
RESET
test_r009 Query reset protocol reset command with invalid reset_state. Check INVALID_PARAMETERS status is returned. RESET
test_r010 Pre-Condition: RESET_PROTOCOL_NOTIFY support.
Query reset protocol notify with invalid domain id.
Check NOT_FOUND status is returned. RESET_PROTOCOL_NOTIFY
test_r011 Pre-Condition: RESET_PROTOCOL_NOTIFY support.
Query reset protocol notify with invalid notify enable.
Check INVALID_PARAMETERS status is returned. RESET_PROTOCOL_NOTIFY

Copyright (c) 2019-2020, Arm Limited and Contributors. All rights reserved.