Chapter 17 Memory System Resource Partitioning and Monitoring
This section describes the SMMU extensions to support Memory System Resource Partitioning and Monitoring, MPAM [3].
This support is described in three areas:
-
How MPAM identifiers are assigned to client device transactions.
-
How MPAM identifiers are generated for SMMU-originated transactions.
-
SMMU support for partitioning of internal resources.
The MPAM architecture defines new per-transaction attributes that affect system behavior or the behavior of components that the transactions pass through or Completers that satisfy a transaction. The additional attributes are two identifiers:
-
Partition ID, or PARTID.
-
Performance Monitoring Group, or PMG.
The PARTID and PMG namespaces are specific to a Security state. A Non-secure stream is assigned a Non-secure PARTID and PMG. In the general case, a Secure stream is assigned a Secure PARTID and PMG.
In the default case, SMMU-originated accesses and client-originated accesses for Realm state use the Realm PARTID space.
See section 17.7 Determination of PARTID space values for full details.
17.1 Discovery and global configuration
Support for MPAM is optional in SMMUv3.2 or later. Support can be discovered using SMMU_IDR3.MPAM and the SMMU_(*_)MPAMIDR register in the programming interface of each Security state. If the SMMU supports MPAM, it might not be supported in all Security states.
When SMMU_IDR3.MPAM == 1, the SMMU_()MPAMIDR registers are present. The PMG and PARTID capabilities of a given Security state can be discovered using SMMU()MPAMIDR. MPAM is supported for a Security state when SMMU_IDR3.MPAM == 1 and either of the corresponding SMMU()MPAMIDR.PARTID_MAX or SMMU(_)MPAMIDR.PMG_MAX fields are not zero.
The SMMU_(*_)GMPAM registers provide Global MPAM configuration used for some types of SMMU-originated transactions. See section 17.4 Assignment of PARTID and PMG for SMMU-originated transactions .
The SMMU_(S_)GBPMPAM registers provide MPAM configuration for client transactions that globally bypass the SMMU. When translation is enabled for a Security state, via SMMU_(*_)CR0.SMMUEN == 1, the MPAM configuration for a client transaction is determined from the corresponding STE.
17.2 Assignment of PARTID and PMG for client transactions
When the system uses MPAM, the SMMU assigns the PARTID and PMG for all client transactions. There is no support for pass-through or modification of a PARTID or PMG that are provided with a request by a client device.
The determination of the PARTID and PMG for all client transactions follows this procedure:
-
If the corresponding programming interface is disabled, SMMU_(*)CR0.SMMUEN == 0, the transaction bypasses and the MPAM attributes are given by the corresponding SMMU(S_)GBPMPAM.GBP_{PMG,PARTID} fields.
-
Otherwise, when translation is enabled by SMMU_(*_)CR0.SMMUEN == 1, the STE configures the MPAM attributes of the given stream.
When SMMU_(*_)CR0.SMMUEN == 1, the STE configures the MPAM behavior. The transaction PARTID and PMG are assigned as follows:
| STE.Confg[2:0] | STE.S1MPAM Final transaction access |
|---|---|
| 0b100(Bypass) | X PARTID =STE.PARTID, PMG =STE.PMG. |
| 0b110(Stage 2)(1) | X PARTID =STE.PARTID, PMG =STE.PMG. |
| 0b101(stage 1) | 0 PARTID =STE.PARTID, PMG =STE.PMG. |
| 1 PARTID =CD.PARTID, PMG =CD.PMG. | |
| 0b111(Nested stage 1 + stage 2) (2) | 0 PARTID =STE.PARTID, PMG =STE.PMG. |
| 1 PARTID = STE.VMS.PARTID_MAP[CD.PARTID], PMG =CD.PMG. |
(1) In a nested configuration, where STE.Config ==0b111, STE.S1DSS == 0b01 will cause a transaction without a SubstreamID to be treated as though Config == 0b110, including for MPAM ID assignment.
(2) If the VMS is not supported or not enabled, the STE.VMSPtr field is not a valid pointer and is not accessed. See 5.6.1 VMS presence and fetching for how to determine whether VMS is supported.
In nested configurations that enable stage 1 control of MPAM with STE.S1MPAM == 1, the CD.PARTID[4:0] value is a virtual PARTID and is translated to a physical PARTID using the VMS.PARTID_MAP structure located through the stream’s STE.VMSPtr field. See section 5.6 Virtual Machine Structure for information on the VMS. The VMS.PARTID_MAP is not used when translation configuration is stage 1-only or stage 2-only, or when STE.S1MPAM == 0, because the PARTID in use is a physical ID in these configurations.
Note: STE.S1MPAM provides backward compatibility for software that is unaware of MPAM, allowing it to operate without requiring a VMS to be allocated.
In an SMMU with RME, the MPAM PARTID and PMG values used for NoStreamID accesses is an IMPLEMENTATION DEFINED choice between:
-
Values provided by the device.
-
Zero.
17.3 PCIe ATS transactions
An ATS Translation Request causes configuration and translation table access using PARTID/PMG values the same as for an equivalent Untranslated transaction on that StreamID and SubstreamID.
The PARTID and PMG assigned to an ATS Translated transaction are as follows:
| SMMU_(R_)CR0.ATSCHK | Confguration | PARTID and PMG |
|---|---|---|
| 0 (Non-secure only) | Any | SMMU_GBPMPAM.GBP_{PARTID, PMG} |
| 1 | Stage 1 only | If UseS1MPAM == 0: PARTID =STE.PARTIDPMG =STE.PMG |
| If UseS1MPAM == 11: PARTID =CD.PARTIDPMG =CD.PMG | ||
| 1 | Stage 1 + stage 2 | If UseS1MPAM == 0: PARTID =STE.PARTIDPMG =STE.PMG |
| If UseS1MPAM == 11: PARTID = | ||
| STE.VMS.PARTID_MAP[CD.PARTID] PMG =CD.PMG | ||
| 1 | Stage 2 only | PARTID =STE.PARTIDPMG =STE.PMG |
| 1 | Split Stage ATS | If UseS1MPAM == 0: PARTID =STE.PARTIDPMG =STE.PMG |
| If UseS1MPAM == 11: PARTID = | ||
| STE.VMS.PARTID_MAP[CD.PARTID] PMG =CD.PMG |
1 UseS1MPAM is 1 if a PASID is presented to the SMMU on an ATS Translated transaction, STE.S1MPAM == 1 and either of the following are true:
-
SMMU_IDR3.PASIDTT == 1.
-
SMMU_IDR3.PASIDTT == 0 and the implementation uses PASID to determine MPAM attributes when a PASID is presented. The set of endpoints for which this behavior occurs is IMPLEMENTATION DEFINED.
Otherwise it is 0.
Note: When SMMU_(R_)CR0.ATSCHK == 1, the STE is fetched to verify whether Translated transactions are permitted. The STE.{PARTID, PMG} fields are therefore available for these transactions. The CD information is only available for an ATS Translated transaction when SMMU_IDR3.PASIDTT is 1 and a PASID is provided.
Note: When translation is enabled for the Non-secure SMMU programming interface, that is SMMU_CR0.SMMUEN == 1, the SMMU_GBPMPAM.GBP_{PARTID, PMG} configuration is not used for ordinary transactions because they are assigned PARTID and PMG using the STE fields. This configuration is used for ATS Translated transactions when SMMU_CR0.ATSCHK == 0.
When SMMU_(R_)CR0.ATSCHK == 1, if a PASID is provided with a Translated transaction from an endpoint and SMMU_IDR3.PASIDTT is 0, an implementation is permitted but not required to use the PASID to determine a PARTID or PMG from a CD as described for configurations with UseS1MPAM == 1, in the table above.
Note: Before PCIe 5.0, endpoints using the PCIe interconnect do not provide a PASID with Translated transactions. However, embedded endpoints that support ATS and PASIDs might be integrated in a way that provides a PASID with Translated transactions.
17.4 Assignment of PARTID and PMG for SMMU-originated transactions
In addition to forwarding client transactions, the SMMU makes additional requests to the memory system for access to configuration, translation structures, queues and for MSIs originating from the SMMU. When MPAM is supported and enabled for a Security state, the PARTIDs and PMGs assigned to these transactions are as follows:
| Structure or request type | PARTID and PMG |
|---|---|
| L1STD and STE | |
| Access to all SMMU queues | Determined by the corresponding SMMU_(*)GMPAM.SO{PARTID,PMG} felds. |
| SMMU MSI writes | |
| VMS | |
| CIT fetches and VSTT fetches | |
| L1CD and CD | PARTID and PMG are determined bySTE.PARTIDandSTE.PMG. |
| Stage 2 Translation Table | See section17.2_Assignment of PARTID and PMG for client transactions_for more |
| Descriptors | information. |
| DCMDQ fetches | |
| Stage 1 Translation Table | PARTID and PMG are determined in the same way as for a non-ATS client transaction: |
| Descriptors | • If both stage 1 and stage 2 are enabled andSTE.S1MPAM== 1, these are |
| determined bySTE.VMSPtr-> PARTID_MAP[CD.PARTID] andCD.PMG. If | |
| STE.S1MPAM== 0, these are determined bySTE.PARTIDandSTE.PMG. | |
| • In a stage 1-only confguration, ifSTE.S1MPAM== 1 these are determined by | |
| CD.PARTIDandCD.PMG. IfSTE.S1MPAM== 0, these are determined by | |
| STE.PARTIDandSTE.PMG. | |
| See section17.2_Assignment of PARTID and PMG for client transactions_for more | |
| information. | |
| HDBSS access | Determined by the corresponding SMMU_(*_)HDBSS_MPAM.{PARTID, PMG} felds. |
| HACDBS access | Determined by the corresponding SMMU_(*_)HACDBS_MPAM.{PARTID, PMG} |
| felds. |
Note: In a nested configuration, in order to fetch stage 1 translation table descriptors using the PARTID and PMG that will be used for the final transaction, the CD.PARTID first needs to be translated to a physical PARTID using the VMS. Although this requires an additional access before the translation table walk can begin, when an implementation provides TLB partitioning, the PARTID is required in order to insert TLB entries.
In systems that support RME, any GPT access is made with the MPAM attributes of the client-originated or SMMU-originated access that caused the GPT access, with the following exception:
-
The MPAM PARTID and PMG values used for GPT walks caused by NoStreamID accesses is an IMPLEMENTATION DEFINED choice between:
-
Values provided by the device.
-
Zero.
-
When a GPT access is made to check the output address of a translation request, the GPT access uses the MPAM attributes that would be used for an equivalent Untranslated transaction for the same StreamID and SubstreamID.
17.5 Assignment of PARTID and PMG for PMCG-originated MSIs
A PMCG might independently support MPAM if the PMCG supports MSIs, so that MPAM information can be assigned to the MSI write.
Note: As a PMCG might be implemented separately to an SMMU, or in a distributed manner, the PMCG might generate MSI transactions in a different way to that of the SMMU queue/table accesses, so is configured separately.
Arm recommends that all PMCGs associated with an SMMU support the same MPAM facilities as the SMMU. This means that support for MPAM is recommended to be consistent among all PMCGs and SMMU, and that the components support PARTID and PMG sizes large enough to carry any valid system PARTID and PMG values.
Note: Because a PMCG might be implemented in a different component to the SMMU, or to other PMCGs, it is not an architectural requirement that a PMCG supports MPAM.
The PMCG has the following registers for MPAM discovery and configuration:
-
SMMU_PMCG_MPAMIDR gives the maximum Non-secure PARTID and PMG values that can be configured for an MSI.
-
SMMU_PMCG_S_MPAMIDR gives the maximum Secure PARTID and PMG values that can be configured for an MSI.
-
SMMU_PMCG_GMPAM configures the PARTID and PMG values for a PMCG MSI.
-
When the PMCG is configured to use a Non-secure attribute on an MSI, this register configures Non-secure PARTID and PMG values.
-
When the PMCG is configured to use a Secure attribute on an MSI, and SMMU_PMCG_SCR.MSI_MPAM_NS is 0 or RES0, this register configures Secure PARTID and PMG values.
-
When the PMCG is configured to use a Secure attribute on an MSI, and SMMU_PMCG_SCR.MSI_MPAM_NS is 1, this register configures Non-secure PARTID and PMG values.
-
Note: The target PA space of an MSI is determined by both SMMU_PMCG_SCR.NSMSI and SMMU_PMCG_SCR.NSRA.
-
Note: Arm expects that SMMU_PMCG_SCR.NSMSI is only set during the transition of a PMCG from Non-secure to Secure state, and therefore using the output PA space to select the MPAM Security state is consistent with the requirements of the MPAM specification.
-
See section 10.5.2 Register details for more information on these registers.
When a PMCG that does not support MPAM is integrated in an MPAM-aware system, Arm recommends that the following mapping is used for PMCG MSIs:
-
PARTID = 0
-
PMG = 0
-
If both the system and the PMCG support Secure state, then:
-
If the MSI targets Secure PA space, then Secure PARTID space is used.
-
If the MSI targets Non-secure PA space, then Non-secure PARTID space is used.
-
-
Otherwise, Non-secure PARTID space is used.
17.6 SMMU support for partitioning and monitoring of internal resources
SMMU-internal resources are permitted to be affected by the PARTID of a client device transaction. The behavior of and control interface for this facility is IMPLEMENTATION DEFINED.
Note: For example, the SMMU might use this facility to implement a TLB-partitioning scheme.
The SMMU might use the PMG of the PARTID to provide monitoring facilities. The behavior of and control interface for this monitor facility is IMPLEMENTATION DEFINED.
Arm recommends that an implementation uses the MPAM MMR interface as defined in [3] for partitioning and monitoring of internal resources. The base address of these registers is IMPLEMENTATION DEFINED.
17.7 Determination of PARTID space values
MPAM PARTID and PMG values are qualified with a PARTID space.
In systems without support for RME, PARTID space is determined by the memory system attribute MPAM_NS.
In systems with support for RME, PARTID space is determined by the memory system attribute MPAM_SP.
For the purposes of assigning MPAM attributes, an SMMU with the Realm programming interface and support for MPAM is a four-space MPAM component.
The encodings of MPAM_NS and MPAM_SP in the memory system are as follows:
| RME supported | Identifer | Encoding |
|---|---|---|
| No | MPAM_NS | 0: Secure PARTID space |
| 1: Non-secure PARTID space | ||
| Yes | MPAM_SP | 0b00: Secure PARTID space |
| 0b01: Non-secure PARTID space | ||
| 0b10: Root PARTID space | ||
| 0b11: Realm PARTID space |
Note: Some SMMU register and configuration structure fields have the name MPAM_NS. These fields determine the target PARTID space of the accesses they govern, regardless of whether the memory system uses MPAM_NS or MPAM_SP.
All SMMU-originated and client-originated transactions for Non-secure state are issued with Non-secure PARTID space.
Accesses from a Non-secure StreamID to the NSP PA space use the Non-secure PARTID space.
The MPAM_NS or MPAM_SP values used for NoStreamID accesses to SA PA space and NSP PA space, and for GPT walks caused by these accesses, are independently an IMPLEMENTATION DEFINED choice between:
-
The value provided by the device.
-
The MPAM_NS or MPAM_SP value corresponding to the Non-secure PARTID space.
-
For SA only, the MPAM_NS or MPAM_SP value corresponding to the Root PARTID space.
Note: If the MPAM PARTID is the value provided by the device, then MPAM_NS or MPAM_SP must be values provided by the device as well.
See section 3.25.10 Granular Data Isolation .
If SMMU_S_MPAMIDR.HAS_MPAM_NS == 0, all SMMU-originated and client-originated transactions for Secure state are issued with Secure PARTID space.
If SMMU_S_MPAMIDR.HAS_MPAM_NS == 1, some transactions for Secure state might be issued with Non-secure PARTID space, according to the rules in this section.
If SMMU_R_MPAMIDR.HAS_MPAM_NS == 1, some accesses for Realm state use Non-secure PARTID space, according to the configuration of SMMU_R_GMPAM.MPAM_NS or STE.MPAM_NS as appropriate.
For MPAM PARTID and PMG register and structure fields that control MPAM for the Secure programming interface, the associated MPAM_NS field determines whether accesses are issued with Secure PARTID space or Non-secure PARTID space.
PARTID and PMG values for accesses for Secure state are determined from the register and structure fields specified in section 17.2 Assignment of PARTID and PMG for client transactions and section 17.4 Assignment of
PARTID and PMG for SMMU-originated transactions . Only the selection of PARTID space and the determination of maximum PARTID and PMG values are affected by this feature.
The full list of MPAM_NS bits for Secure state, which each describe the Secure accesses they apply to are:
-
SMMU_S_GBPMPAM.MPAM_NS
-
SMMU_S_GMPAM.MPAM_NS
-
STE.MPAM_NS
-
SMMU_PMCG_SCR.MSI_MPAM_NS
Note: There is no MPAM_NS bit in CD or VMS. They inherit the choice of PARTID space from the STE that led to them.
In systems that support RME, the MPAM_NS or MPAM_SP values used for NoStreamID accesses, and GPT walks caused by NoStreamID accesses, are independently an IMPLEMENTATION DEFINED choice between:
-
The value provided by the device. This option must be used if the PARTID and PMG values used are the values provided by the device.
-
The MPAM_NS or MPAM_SP value corresponding to the target PA space of the NoStreamID access.
For an SMMU that does not support MPAM_SP, then it is permitted to map MPAM_SP to MPAM_NS as follows:
-
NoStreamID accesses to Root PA space use the Secure MPAM_NS value of 0.
-
NoStreamID accesses to Realm PA space use the Non-secure MPAM_NS value of 1.
Any IMPLEMENTATION DEFINED determination of MPAM attributes must comply with the requirements of the MPAM architecture [3].