Source Downloads and Shared State Cache

Every Yocto Project build relies on two caches. The downloads cache (DL_DIR) holds the source archives and repository checkouts that BitBake fetches for each recipe. The shared state cache (SSTATE_DIR) holds the packaged results of tasks that have already run, so a later build restores a task result instead of rebuilding it.

EDF publishes both caches on the network and configures the build to use them automatically. A build machine with a working connection to the Internet needs no setup: this page describes the mechanism for readers who want to understand it, and documents the offline case where the caches have to be provided locally.

Note

The downloads and sstate-cache bundles on the downloads page are optional. They exist for offline builds and for pre-populating a cache on a slow link. A normal online build does not need them.

Online Builds

On a build machine that can reach the Internet there is nothing to do. The local.conf template that edf-init-build-env installs from the meta-amd-edf layer already points the build at the AMD source mirror and shared state mirror, and BitBake consults both transparently.

The relevant part of the template appears below for reference. These settings need no action: edf-init-build-env puts them in place, and neither copying nor editing them is required.

Mirror settings in the default local.conf template
AMD-EDF_VERSION = "26.06"

INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "https://edf.amd.com/sswreleases/amd-edf/${AMD-EDF_VERSION}/downloads"

SSTATE_MIRRORS = " \
file://.* https://edf.amd.com/sswreleases/amd-edf/${AMD-EDF_VERSION}/aarch64/sstate-cache/PATH;downloadfilename=PATH \n \
"

The effect of those three settings is:

  • SOURCE_MIRROR_URL names the AMD-hosted copy of the sources for the release.

  • INHERIT += "own-mirrors" puts that URL in front of every upstream source location, for each of the fetcher types the class lists, among them git, https, ftp, npm, and crate.

  • SSTATE_MIRRORS names the AMD-hosted shared state cache for the aarch64 target architecture.

A first build on a fresh host therefore pulls sources and prebuilt task output from AMD rather than rebuilding everything locally, and populates the local DL_DIR and SSTATE_DIR as it goes.

Where Artifacts Come From

Because EDF ships own-mirrors by default, it helps to know the order in which BitBake looks for a source archive:

  1. Local downloads cache. If DL_DIR already holds the archive, BitBake uses it and fetches nothing.

  2. Pre-mirrors (PREMIRRORS). The own-mirrors class prepends SOURCE_MIRROR_URL here, so the AMD mirror is tried before any upstream location. This is what makes a release reproducible even when an upstream project moves or deletes a tarball.

  3. The upstream location in the recipe (SRC_URI). Reached only when the pre-mirror does not have the archive.

  4. Mirrors (MIRRORS). The fallback list, tried last.

The shared state cache follows the same shape with one fewer step: BitBake uses a matching object from the local SSTATE_DIR if there is one, and otherwise tries each entry in SSTATE_MIRRORS. A shared state miss is not an error. The affected task simply runs locally, and its result is written to SSTATE_DIR for the next build.

Offline and Air-Gapped Builds

A build host with no route to edf.amd.com cannot use the hosted mirrors. There are two ways to handle that, and the choice is a trade-off between download size and build time.

Providing the Caches in Advance

Downloading the downloads and sstate-cache bundles on a machine that can reach the Internet, moving them to the build host, and pointing the mirror variables at the unpacked copies gives an offline build the same acceleration an online one gets. This is optional, not mandatory, and the bundles are large: check the sizes in the downloads table before committing to the transfer.

Unpack both bundles into the build directory. Both file names carry the release and a build-date stamp; the pattern below matches whichever stamp the download carries.

$ tar -xf downloads_edf_26.06.1_*.tar.gz -C <path-to>/yocto/edf/build
$ tar -xf sstate-cache_edf_26.06.1_*.tar.gz -C <path-to>/yocto/edf/build

Then repoint the mirror variables in build/conf/local.conf at the unpacked directories and take the build off the network:

SOURCE_MIRROR_URL = "file:///<path-to>/yocto/edf/build/downloads"
SSTATE_MIRRORS = "\
file://.* file:///<path-to>/yocto/edf/build/sstate-cache/PATH \n \
"
BB_NO_NETWORK = "1"

INHERIT += "own-mirrors" is already present in the template, so the file:// URL takes effect as a pre-mirror without further changes. BB_NO_NETWORK makes any attempt to reach the network a hard error, so a misconfigured mirror path fails immediately instead of stalling the build on fetch timeouts.

The same procedure appears as a step in the Operating System Integration and Development walkthrough.

Building Without the Bundles

The bundles are a speed-up, not a prerequisite. A host that can still reach the upstream project servers, but not the AMD mirrors, builds everything from source: BitBake fetches each recipe’s SRC_URI directly, every task runs locally, and the local SSTATE_DIR fills up as the build proceeds. The result is identical; only the first build is slow.

That local shared state cache is what makes the second build fast. It is per-host and per-workspace, so it accelerates later iterations on the same machine, and is not a substitute for the hosted mirror on a freshly installed host.

Note

An offline build with a downloads mirror and no shared state mirror is the case most likely to expose a host toolchain older than the Yocto Project’s minimum tool versions, because every component is compiled locally. Check the host tools before starting one; see Prepare the Build Host Tools.

Sharing the Caches Between Builds

Both caches default to a location inside the build directory, which means a second build directory starts from nothing. Moving them somewhere central lets every build directory, and every user on the host, share one copy. See Optimizing Build Time Locally - Central Download and Shared State Locations for the DL_DIR and SSTATE_DIR settings that do this.