Enabling the System Memory Management Unit for Xen on Versal Gen 2

Overview

On AMD Versal Gen 2 devices, the EDF platform BSP configures the SMMU so Xen can translate and restrict DMA traffic issued by the peripherals assigned to a domain.

Two pieces of configuration in the platform BSP enable this mechanism:

  • The device tree carries an iommus property for each DMA-capable peripheral that Xen supports. The property maps the peripheral to its SMMU stream ID.

  • The Xen boot entry passes iommu=force on the Xen command line so that the hypervisor takes control of the IOMMU implementation.

EDF uses one device tree for U-Boot, Linux, and Xen. At runtime, select the boot entry from the systemd-boot menu. The SMMU node keeps status = "disabled"; in the shipped device tree, so a native Linux boot stays unaffected. iommu=force makes Xen enable the SMMU on the Xen boot path.

The Versal Gen 2 multidomain machines include the device tree mappings:

  • versal-2ve-2vm-vek385-multidomain

  • versal-2ve-2vm-vek385-revb-multidomain

  • versal-2ve-2vm-vek386-multidomain

  • versal-2ve-2vm-vmk365-multidomain

Device Tree Configuration

The Xen device tree include is versal-2ve-2vm-xen.dtsi from the meta-xilinx tree, and the shared platform include is platform-misc.dtsi from the meta-amd-adaptive-socs BSP tree. The four Versal Gen 2 machine YAMLs list both files in CONFIG_YOCTO_BBMC_LINUX_DTSI. The Xen include enables the SMMU node. Applying the shared platform include afterward disables the node again so the same device tree supports native Linux boots.

The Xen include attaches each supported peripheral to the SMMU with an iommus property. For example:

&smmu {
    status = "okay";
};

&gem0 {
    iommus = <&smmu 0x234>;
};

&gem1 {
    iommus = <&smmu 0x235>;
};

A platform include applied after the Xen include returns the SMMU node to the disabled state, so the same device tree can still be used for a native Linux boot:

&smmu {
    status = "disabled";
};

The iommus properties remain in the device tree in both cases. They take effect only when the SMMU is enabled.

Booting and Verifying

The Xen boot entry is present in images built from the edf-platform-disk-image recipe, which is the platform variant that adds the Xen and OpenAMP packages. The Linux-only edf-linux-disk-image recipe does not include the xen.efi and edf-xen.conf files, so it has no EDF Xen entry to select. Because of this, the Xen flow uses the platform disk image. The generated boot artifacts change from the Linux image names to the equivalent edf-platform-disk-image-... names.

At boot, systemd-boot shows an EDF Linux entry and an EDF Xen entry. EDF Linux is the default, and the menu times out after five seconds. Select EDF Xen to boot the hypervisor.

After Dom0 is up, confirm that the hypervisor picked up the option by checking the xen_commandline field. It must contain iommu=force.

# xl info

Confirm that peripherals whose DMA the SMMU translates, such as USB or Ethernet controllers used for storage and networking, still work. If a peripheral lacks its iommus property, its DMA traffic can fail when the SMMU is enabled.

Running Under QEMU

The Building Images for QEMU procedure documents VEK385 and VEK386. The example here uses VEK385 Rev B.

First set up the Yocto build environment as described in that procedure. Build the board boot image, QEMU helper, and Xen-capable disk image:

$ MACHINE=versal-2ve-2vm-vek385-revb-multidomain bitbake edf-ospi
$ MACHINE=amd-cortexa78-mali-common bitbake qemu-helper-native
$ MACHINE=amd-cortexa78-mali-common bitbake edf-platform-disk-image

The launch uses slirp networking, so skip the tap-interface setup.

Create the working directory and copy both machines’ artifacts into it before merging their configurations:

$ mkdir -p tmp/deploy/images/vek385-qemu
$ cp -r tmp/deploy/images/versal-2ve-2vm-vek385-revb-multidomain/* tmp/deploy/images/vek385-qemu
$ cp -r tmp/deploy/images/amd-cortexa78-mali-common/* tmp/deploy/images/vek385-qemu

Merge the board boot configuration with the platform disk image configuration, then launch QEMU with the prepared configuration:

$ ./<path-to>/yocto/edf/sources/meta-xilinx/meta-xilinx-core/scripts/qemuboot-tool \
  load tmp/deploy/images/vek385-qemu/BOOT-versal-2ve-2vm-vek385-revb-multidomain.qemuboot.conf \
  remove image_link_name \
  remove image_name \
  merge tmp/deploy/images/vek385-qemu/edf-platform-disk-image-amd-cortexa78-mali-common.rootfs.qemuboot.conf \
  > tmp/deploy/images/vek385-qemu/vek385-combined.qemuboot.conf
$ runqemu tmp/deploy/images/vek385-qemu/vek385-combined.qemuboot.conf nographic slirp

This command sequence first prepares the combined qemuboot.conf from the board boot artifacts and the edf-platform-disk-image rootfs, then hands the prepared configuration to runqemu. This Xen example uses the platform image recipe in place of the linked procedure’s Linux-only image recipe, so the generated image and artifact paths differ.

At the boot menu, select the EDF Xen entry, then verify iommu=force as described in Booting and Verifying.