Chapter 8: DPDK EAL And SPDK Env | SPDK From First Principles

Reader Promise

By the end of this chapter, a beginner should be able to explain why an SPDK process starts by initializing an "environment" before it initializes storage subsystems, why that environment is usually DPDK EAL, and why errors about hugepages, VFIO, IOVA, NUMA, core masks, and permissions are not random setup chores. They are the foundation that lets later SPDK code poll devices from userspace and hand DMA-safe buffers to hardware.

This chapter is intentionally practical. If a production diskengine node fails before bdevs appear, the failure usually lives here: CPU selection, hugepage memory, PCI ownership, IOMMU/VFIO, shared memory IDs, virtual-to-physical translation, or permissions.

The official documentation lines up with that view. SPDK's system configuration guide treats IOMMU, VFIO, hugetlbfs access, device permissions, and memlock limits as required host configuration. DPDK's Environment Abstraction Layer documentation describes EAL as the layer that gains access to low-level resources such as hardware, memory, core assignment, timers, interrupts, and memory zones. The Linux VFIO documentation describes VFIO as the kernel interface for direct userspace device access under IOMMU protection, with IOMMU groups as the ownership boundary.

Mental Model

The SPDK env layer is the process contract with the host machine.

Before SPDK can submit fast I/O, it needs answers to questions ordinary C programs usually ignore:

Which CPU cores will run the event loops?
Where can the process allocate memory that will not move while hardware uses it?
Can hardware DMA to that memory under the current IOMMU and IOVA mode?
How does a userspace pointer become an address a device can use?
Which PCI devices are visible to the process?
Is this process the only SPDK process using the hugepage namespace, or is it sharing state with another process?
Which NUMA node owns the memory touched by each polling core?

DPDK EAL answers much of that. SPDK wraps it in spdk_env_* APIs so most SPDK libraries do not call DPDK directly. That wrapper matters. It lets bdev, nvme, thread, reactor, iobuf, and transport code ask for "DMA memory", "a mempool", "a memzone", "the current core", or "the physical address for this buffer" without knowing whether the platform-specific implementation is DPDK or something else.

The beginner trap is to think of EAL as "the networking library." In SPDK, EAL is the platform bring-up layer: core masks, hugepage-backed allocation, PCI enumeration, memzones, mempools, launch-time thread affinity, shared memory, and memory map callbacks.

Where The Env Fits In Startup

The startup path is easiest to understand as a narrowing funnel. The application starts with broad app options, turns the env-related subset into env options, then asks the DPDK-backed env implementation to initialize the host-facing runtime.

main()
  prepares spdk_app_opts
  calls spdk_app_start()
    app_copy_opts()
    app_setup_env()
      fills spdk_env_opts from app opts
      calls spdk_env_init()
        builds DPDK EAL command line
        calls rte_eal_init()
        initializes PCI env, memory map, vtophys
    initializes reactors and threads
    initializes subsystems

This chapter focuses on the app_setup_env() -> spdk_env_init() segment. The next startup chapter picks up at reactors and subsystems.

Source Anchors

include/spdk/env.h: struct spdk_env_opts, spdk_env_opts_init(), spdk_env_init(), spdk_env_fini(), spdk_malloc(), spdk_zmalloc(), spdk_dma_malloc(), spdk_dma_zmalloc(), spdk_mempool_create(), spdk_memzone_reserve(), spdk_vtophys()
include/spdk/event.h: struct spdk_app_opts, spdk_app_opts_init(), spdk_app_start()
lib/event/app.c: app_setup_env(), app_copy_opts(), spdk_app_start()
lib/env_dpdk/init.c: build_eal_cmdline(), spdk_env_init(), spdk_env_dpdk_post_init(), spdk_env_fini()
lib/env_dpdk/env.c: spdk_malloc(), spdk_zmalloc(), spdk_dma_malloc_socket(), spdk_dma_zmalloc_socket(), spdk_mempool_create_ctor(), spdk_memzone_reserve_aligned()
lib/env_dpdk/memory.c: vtophys_init(), spdk_vtophys(), mem_disable_vtophys(), vtophys_notify(), vtophys_iommu_init()
lib/env_dpdk/pci.c: PCI device enumeration, BAR mapping, hotplug, and DMA BAR mapping paths
scripts/setup.sh: host preparation for hugepages, VFIO/UIO binding, IOMMU group viability, and device setup

The Two Option Structures

SPDK has both application options and environment options.

struct spdk_app_opts is the public event-framework option structure. It includes the app name, JSON config, RPC address, reactor mask, memory size, PCI allow/block lists, hugepage options, interrupt mode, trace options, delay_subsystem_init, and other event-framework concerns.

struct spdk_env_opts is lower-level. It is what the env implementation needs: process name, core mask or lcore map, shared memory ID, memory channel count, main core, memory size, hugepage flags, PCI settings, IOVA mode, base virtual address, VF token, and NUMA behavior.

The bridge is lib/event/app.c:app_setup_env(). It creates a local struct spdk_env_opts, initializes defaults, copies the env-relevant fields from struct spdk_app_opts, then calls spdk_env_init().

Excerpt from lib/event/app.c:

env_opts.opts_size = sizeof(env_opts);
spdk_env_opts_init(&env_opts);

env_opts.name = opts->name;
env_opts.core_mask = opts->reactor_mask;
env_opts.lcore_map = opts->lcore_map;
env_opts.shm_id = opts->shm_id;
env_opts.mem_channel = opts->mem_channel;
env_opts.main_core = opts->main_core;
env_opts.mem_size = opts->mem_size;
env_opts.hugepage_single_segments = opts->hugepage_single_segments;
env_opts.unlink_hugepage = opts->unlink_hugepage;
env_opts.hugedir = opts->hugedir;
env_opts.no_pci = opts->no_pci;
env_opts.num_pci_addr = opts->num_pci_addr;
env_opts.pci_blocked = opts->pci_blocked;
env_opts.pci_allowed = opts->pci_allowed;
env_opts.base_virtaddr = opts->base_virtaddr;
env_opts.env_context = opts->env_context;
env_opts.iova_mode = opts->iova_mode;
env_opts.vf_token = opts->vf_token;
env_opts.no_huge = opts->no_huge;
env_opts.enforce_numa = opts->enforce_numa;

rc = spdk_env_init(&env_opts);

The important beginner detail is that a command-line option often lands in spdk_app_opts, but the failure message may come later from DPDK EAL or the env layer after the value has been translated. For example, an application-facing "reactor mask" becomes an EAL CPU selection argument. A PCI allowlist becomes -a or --allow arguments. A no-PCI option becomes both --no-pci and a change to SPDK's vtophys behavior.

opts_size is part of the ABI story. Both app and env option structures carry size fields so an older caller can pass a smaller structure and the library can keep defaults for fields the caller does not know about. That is why the code initializes defaults first and then copies fields conditionally in the broader app/env copy helpers.

How EAL Arguments Are Built

lib/env_dpdk/init.c:build_eal_cmdline() converts spdk_env_opts into DPDK arguments. It is worth reading slowly because many startup failures are explained there.

The first decision is whether the process participates in DPDK shared configuration. If shm_id < 0, SPDK adds --no-shconf. That is the single-process style: DPDK shared configuration files are disabled, which simplifies cleanup but also means there is no DPDK secondary process support through that shared configuration.

Excerpt from lib/env_dpdk/init.c:

/* disable shared configuration files when in single process mode. This allows for cleaner shutdown */
if (opts->shm_id < 0) {
	args = push_arg(args, &argcount, _sprintf_alloc("%s", "--no-shconf"));
	if (args == NULL) {
		return -1;
	}
}

/* Either lcore_map or core_mask must be set. If both, or none specified, fail */
if ((opts->core_mask == NULL) == (opts->lcore_map == NULL)) {
	if (opts->core_mask && opts->lcore_map) {
		fprintf(stderr,
			"Both, lcore map and core mask are provided, while only one can be set\n");
	} else {
		fprintf(stderr, "Core mask or lcore map must be specified\n");
	}
	free_args(args, argcount);
	return -1;
}

The next decision is CPU syntax. DPDK documents that only one core selection form should be used at a time: --lcores, -l, or -c. SPDK mirrors that by accepting either lcore_map or core_mask, not both.

Excerpt from lib/env_dpdk/init.c:

if (opts->lcore_map) {
	/* If lcore list is set, generate --lcores parameter */
	args = push_arg(args, &argcount, _sprintf_alloc("--lcores=%s", opts->lcore_map));
} else if (opts->core_mask[0] == '-') {
	/*
	 * Set the coremask:
	 *
	 * - if it starts with '-', we presume it's literal EAL arguments such
	 *   as --lcores.
	 *
	 * - if it starts with '[', we presume it's a core list to use with the
	 *   -l option.
	 *
	 * - otherwise, it's a CPU mask of the form "0xff.." as expected by the
	 *   -c option.
	 */
	args = push_arg(args, &argcount, _sprintf_alloc("%s", opts->core_mask));
} else if (opts->core_mask[0] == '[') {
	char *l_arg = _sprintf_alloc("-l %s", opts->core_mask + 1);

	if (l_arg != NULL) {
		int len = strlen(l_arg);

		if (l_arg[len - 1] == ']') {
			l_arg[len - 1] = '\0';
		}
	}
	args = push_arg(args, &argcount, l_arg);
} else {
	args = push_arg(args, &argcount, _sprintf_alloc("-c %s", opts->core_mask));
}

The bracket branch strips a trailing ] and passes the list through as -l. Otherwise SPDK passes the value as -c <mask>.

That small branch explains a large class of startup bugs. If an operator supplies an invalid mask, maps an lcore above the DPDK build's RTE_MAX_LCORE, or accidentally combines two CPU selection styles, the process may fail before any bdev or NVMe probe happens. The storage stack has not started yet. EAL rejected the CPU contract.

No PCI, No Huge, Hugepage Options, And IOVA

--no-pci is not just a bus scan option. In SPDK's DPDK env, it also disables vtophys map creation because a no-PCI process should not need PCI DMA address translation. That is useful for some tests and non-PCI tools, but it is wrong for direct NVMe PCI access.

--no-huge is even more constrained. Normal SPDK deployment wants DPDK hugepage memory. SPDK permits a no-huge mode, but the code rejects combinations that do not make sense: no hugepages cannot be combined with hugepage-specific options, it requires an explicit memory size, and physical-address IOVA mode is rejected.

Excerpt from lib/env_dpdk/init.c:

/* set no pci  if enabled */
if (opts->no_pci) {
	args = push_arg(args, &argcount, _sprintf_alloc("--no-pci"));
	if (args == NULL) {
		return -1;
	}
	mem_disable_vtophys();
}

The no-huge path is separate. It validates that the operator has not asked for hugepage-only behavior while also disabling hugepages.

Excerpt from lib/env_dpdk/init.c:

if (no_huge) {
	if (opts->hugepage_single_segments || opts->unlink_hugepage || opts->hugedir) {
		fprintf(stderr, "--no-huge invalid with other hugepage options\n");
		free_args(args, argcount);
		return -1;
	}

	if (opts->mem_size < 0) {
		fprintf(stderr,
			"Disabling hugepages requires specifying how much memory "
			"will be allocated using -s parameter\n");
		free_args(args, argcount);
		return -1;
	}

	/* iova-mode=pa is incompatible with no_huge */
	if (opts->iova_mode &&
	    (strcmp(opts->iova_mode, "pa") == 0)) {
		fprintf(stderr, "iova-mode=pa is incompatible with specified "
			"no-huge parameter\n");
		free_args(args, argcount);
		return -1;
	}

	args = push_arg(args, &argcount, _sprintf_alloc("--no-huge"));
	args = push_arg(args, &argcount, _sprintf_alloc("--legacy-mem"));
	args = push_arg(args, &argcount, _sprintf_alloc("--iova-mode=va"));
}

The no-huge path returns an error if physical-address IOVA mode is requested, then adds --no-huge, --legacy-mem, and --iova-mode=va.

IOVA mode is the device-visible address mode. DPDK's EAL parameters document --iova-mode <pa|va>. SPDK either passes the operator's explicit setting through, or applies platform-specific defaults when it knows DPDK's guess may be wrong.

Excerpt from lib/env_dpdk/init.c:

if (opts->iova_mode) {
	/* iova-mode=pa is incompatible with no_huge */
	args = push_arg(args, &argcount, _sprintf_alloc("--iova-mode=%s", opts->iova_mode));
	if (args == NULL) {
		return -1;
	}
}

If the operator did not specify an IOVA mode, SPDK still has platform checks in the else branch. For example, unsafe VFIO no-IOMMU mode needs physical-address IOVA:

/* When using vfio with enable_unsafe_noiommu_mode=Y, we need iova-mode=pa,
 * but DPDK guesses it should be iova-mode=va. Add a check and force
 * iova-mode=pa here. */
if (!no_huge && rte_vfio_noiommu_is_enabled()) {
	args = push_arg(args, &argcount, _sprintf_alloc("--iova-mode=pa"));
	if (args == NULL) {
		return -1;
	}
}

On x86, SPDK also checks whether the CPU and environment can support the IOMMU behavior DPDK would otherwise infer:

/* DPDK by default guesses that it should be using iova-mode=va so that it can
 * support running as an unprivileged user. However, some systems (especially
 * virtual machines) don't have an IOMMU capable of handling the full virtual
 * address space and DPDK doesn't currently catch that. Add a check in SPDK
 * and force iova-mode=pa here. */
if (!no_huge && !x86_cpu_support_iommu()) {
	args = push_arg(args, &argcount, _sprintf_alloc("--iova-mode=pa"));
	if (args == NULL) {
		return -1;
	}
}

The inference to carry forward is simple: IOVA mode is not a cosmetic option. It determines whether the address programmed into device descriptors is treated as a physical address or as an I/O virtual address handled through IOMMU mappings. If this is wrong, a device may fail to attach, fail DMA mapping, or later DMA to an address the kernel refuses to map.

What `spdk_env_init()` Actually Does

lib/env_dpdk/init.c:spdk_env_init() is the DPDK-backed implementation. It is the boundary between SPDK's portable env API and DPDK's actual EAL initialization.

Its sequence is:

Validate whether this is first initialization or reinitialization.
Validate opts_user and opts_size.
Copy options using env_copy_opts().
Initialize OpenSSL settings.
Call build_eal_cmdline().
Print the DPDK EAL parameter list.
Copy the argument array because DPDK may rearrange it.
Call rte_eal_init().
Determine whether legacy memory mode is needed.
Call spdk_env_dpdk_post_init().

Excerpt from lib/env_dpdk/init.c:

rc = build_eal_cmdline(opts);
if (rc < 0) {
	SPDK_ERRLOG("Invalid arguments to initialize DPDK\n");
	return -EINVAL;
}

SPDK_PRINTF("Starting %s / %s initialization...\n", SPDK_VERSION_STRING, rte_version());

args_print = _sprintf_alloc("[ DPDK EAL parameters: ");
if (args_print == NULL) {
	return -ENOMEM;
}
for (i = 0; i < g_eal_cmdline_argcount; i++) {
	args_tmp = args_print;
	args_print = _sprintf_alloc("%s%s ", args_tmp, g_eal_cmdline[i]);
	if (args_print == NULL) {
		free(args_tmp);
		return -ENOMEM;
	}
	free(args_tmp);
}
SPDK_PRINTF("%s]\n", args_print);
free(args_print);

That printed EAL parameter list is important operational evidence. When startup fails, compare the intended app options to the actual DPDK arguments printed here. Many "SPDK" failures are an unexpected EAL argv problem.

The call into DPDK is direct:

fflush(stdout);
orig_optind = optind;
optind = 1;
rc = rte_eal_init(g_eal_cmdline_argcount, dpdk_args);
optind = orig_optind;

free(dpdk_args);

if (rc < 0) {
	if (rte_errno == EALREADY) {
		SPDK_ERRLOG("DPDK already initialized\n");
	} else {
		SPDK_ERRLOG("Failed to initialize DPDK\n");
	}
	return -rte_errno;
}

When rte_eal_init() returns an error, SPDK has not reached reactors, subsystems, NVMe probing, bdev creation, or RPC runtime. You are still in platform bring-up.

After DPDK succeeds, SPDK performs post-init work that is specific to SPDK's env abstraction.

Excerpt from lib/env_dpdk/init.c:

spdk_env_dpdk_post_init(bool legacy_mem)
{
	int rc;

	rc = pci_env_init();
	if (rc < 0) {
		SPDK_ERRLOG("pci_env_init() failed\n");
		return rc;
	}

	rc = mem_map_init(legacy_mem);
	if (rc < 0) {
		SPDK_ERRLOG("Failed to allocate mem_map\n");
		return rc;
	}

	rc = vtophys_init();
	if (rc < 0) {
		SPDK_ERRLOG("Failed to initialize vtophys\n");
		return rc;
	}

	return 0;
}

So when spdk_env_init() succeeds, the process has more than "DPDK started." It has an SPDK-compatible env implementation ready for PCI device handling, memory map tracking, and address translation.

Hugepages And Why Normal `malloc()` Is Not Enough

SPDK storage paths often pass buffers to hardware or to other DMA-capable components. Normal heap memory is convenient for CPU-only data structures, but it does not automatically satisfy the constraints needed by direct device I/O:

The memory must remain resident while hardware uses it.
The memory must be representable in the device's DMA address space.
SPDK must be able to translate or register it under the current IOVA/IOMMU mode.
Some structures must be cache-line aligned or IOVA-contiguous.
For multi-process cases, the memory may need a DPDK shared-memory identity.

DPDK's EAL documentation says that, on Linux, EAL uses mmap() in hugetlbfs for physical memory allocation and exposes that memory to DPDK services such as mempools. DPDK's Linux system requirements document explains hugepage setup because the memory subsystem depends on hugepage support for high-performance packet and DMA-style memory pools. SPDK reuses that foundation for storage.

The public SPDK DMA allocation APIs are declared in include/spdk/env.h:

spdk_dma_malloc()
spdk_dma_malloc_socket()
spdk_dma_zmalloc()
spdk_dma_zmalloc_socket()
spdk_dma_realloc()
spdk_dma_free()

The DPDK env implementation makes the dependency explicit. spdk_dma_malloc_socket() is not a separate allocator. It calls spdk_malloc() with flags that say the memory must be DMA-capable and shareable.

Excerpt from lib/env_dpdk/env.c:

void *
spdk_malloc(size_t size, size_t align, uint64_t *unused, int numa_id, uint32_t flags)
{
	void *buf;

	if (flags == 0 || unused != NULL) {
		return NULL;
	}

	align = spdk_max(align, RTE_CACHE_LINE_SIZE);
	buf = rte_malloc_socket(NULL, size, align, numa_id);
	if (buf == NULL && !g_enforce_numa && numa_id != SOCKET_ID_ANY) {
		buf = rte_malloc_socket(NULL, size, align, SOCKET_ID_ANY);
	}
	return buf;
}

void *
spdk_dma_malloc_socket(size_t size, size_t align, uint64_t *unused, int numa_id)
{
	return spdk_malloc(size, align, unused, numa_id, (SPDK_MALLOC_DMA | SPDK_MALLOC_SHARE));
}

There are three practical lessons in this small excerpt.

First, SPDK enforces at least cache-line alignment, even if the caller asks for less. Second, allocations are NUMA-aware, but unless enforce_numa is set, they can fall back to any socket. Third, the DMA allocator uses DPDK's allocator, not libc malloc().

Beginner rule: if an SPDK API says a buffer must be allocated with spdk_dma_malloc() or one of its variants, do not substitute malloc(). The code may compile, but a controller, DMA engine, RDMA NIC, vfio-user path, or zero-copy path may fail later when it tries to translate, register, or share the buffer.

Vtophys And IOVA

spdk_vtophys() is the reader-friendly name for a hard problem: translate a virtual address in the process into an address usable for DMA. In the DPDK env, the public function is in lib/env_dpdk/memory.c.

The map used by spdk_vtophys() is created during env post-init:

if (g_vtophys) {
	g_vtophys_map = spdk_mem_map_alloc(SPDK_VTOPHYS_ERROR, &vtophys_map_ops, NULL);
	if (g_vtophys_map == NULL) {
		DEBUG_PRINT("vtophys map allocation failed\n");
		spdk_mem_map_free(&g_numa_map);
		spdk_mem_map_free(&g_phys_ref_map);
		return -ENOMEM;
	}
}

return 0;

The map's operations include a notify callback:

const struct spdk_mem_map_ops vtophys_map_ops = {
	.notify_cb = vtophys_notify,
	.are_contiguous = vtophys_check_contiguous_entries,
};

That callback path is how SPDK learns about memory being registered, mapped, or unmapped. The exact address values depend on IOVA mode and IOMMU state, but the ownership idea is stable: SPDK keeps a memory map so later I/O code can ask for the device-usable address of a buffer.

Excerpt from lib/env_dpdk/memory.c:

uint64_t
spdk_vtophys(const void *buf, uint64_t *size)
{
	uint64_t vaddr, paddr, mask;

	/* vtophys map do not get created in no-pci env */
	if (g_vtophys_map == NULL) {
		return SPDK_VTOPHYS_ERROR;
	}

	vaddr = (uint64_t)buf;
	paddr = spdk_mem_map_translate(g_vtophys_map, vaddr, size);
	if (paddr == SPDK_VTOPHYS_ERROR) {
		return SPDK_VTOPHYS_ERROR;
	}

	mask = (paddr & VTOPHYS_4KB) ? MASK_4KB : MASK_2MB;
	return VTOPHYS_ADDR(paddr) + (vaddr & mask);
}

This function does not magically make an arbitrary pointer DMA-safe. It looks up a pointer in the env memory map. If the pointer was not allocated or registered through the right path, translation can fail. If --no-pci disabled vtophys setup, translation returns SPDK_VTOPHYS_ERROR.

Important modes:

With IOVA as physical address, device-visible addresses correspond to physical addresses.
With IOVA as virtual address, device-visible addresses are I/O virtual addresses mapped through IOMMU state.
With --no-pci, SPDK disables vtophys setup because direct PCI DMA translation is not expected.
With --no-huge, SPDK forces --iova-mode=va in this code path and rejects iova-mode=pa.

Misconception to kill: "Hugepages automatically mean every pointer can be used for DMA." No. The buffer still needs to come from the right allocator or memory registration path, and the device must be able to address it under the current IOVA/IOMMU mode.

PCI Ownership And VFIO

SPDK is a userspace storage stack. For direct NVMe PCI access, the kernel NVMe driver must not own the controller. The device is usually bound to vfio-pci, and the process uses VFIO and DPDK PCI enumeration to map device resources.

The Linux VFIO documentation describes VFIO as an IOMMU and device agnostic framework for exposing direct device access to userspace in an IOMMU-protected environment. It also explains why groups matter: a VFIO group appears as /dev/vfio/$GROUP, and if a group contains multiple devices, those devices must all be bound to VFIO or unbound before group operations are allowed. SPDK's setup script encodes that same rule.

Excerpt from scripts/setup.sh:

local iommu_group=${pci_iommu_groups["$bdf"]}
if [ -e "/dev/vfio/$iommu_group" ]; then
	if [ -n "$TARGET_USER" ]; then
		chown "$TARGET_USER" "/dev/vfio/$iommu_group"
	fi
fi

local iommug=("${!iommu_groups[iommu_group]}")
local _bdf _driver
if ((${#iommug[@]} > 1)) && [[ $driver_name == vfio* ]]; then
	pci_dev_echo "$bdf" "WARNING: detected multiple devices (${#iommug[@]}) under the same IOMMU group!"
	for _bdf in "${iommug[@]}"; do
		[[ $_bdf == "$bdf" ]] && continue
		_driver=$(readlink -f "/sys/bus/pci/devices/$_bdf/driver") && _driver=${_driver##*/}
		if [[ $_driver == "$driver_name" ]]; then
			continue
		fi
		# See what DPDK considers to be a "viable" iommu group: dpdk/lib/eal/linux/eal_vfio.c -> rte_vfio_setup_device()
		pci_dev_echo "$bdf" "WARNING: ${_bdf##*/} not bound to $driver_name (${_driver:-no driver})"

The object ownership chain is:

Linux owns the PCI device through some kernel driver.
scripts/setup.sh unbinds or rebinds devices for SPDK use.
With IOMMU enabled, SPDK prefers vfio-pci.
VFIO exposes a group device node such as /dev/vfio/5.
DPDK opens the VFIO group and maps DMA and device resources.
SPDK's PCI env exposes devices to higher-level libraries such as NVMe.

The driver-selection logic in scripts/setup.sh shows the preference order.

Excerpt from scripts/setup.sh:

if [[ "${DRIVER_OVERRIDE}" == "none" ]]; then
	driver_name=none
elif [[ -n "${DRIVER_OVERRIDE}" ]]; then
	driver_path="$DRIVER_OVERRIDE"
	driver_name="${DRIVER_OVERRIDE##*/}"
	# modprobe and the sysfs don't use the .ko suffix.
	driver_name=${driver_name%.ko}
	# path = name -> there is no path
	if [[ "$driver_path" = "$driver_name" ]]; then
		driver_path=""
	fi
elif is_iommu_enabled; then
	driver_name=vfio-pci
	# Just in case, attempt to load VFIO_IOMMU_TYPE1 module into the kernel - this
	# should be done automatically by modprobe since this particular module should
	# be a part of vfio-pci dependencies, however, on some distros, it seems that
	# it's not the case. See #1689.
	if modinfo vfio_iommu_type1 > /dev/null; then
		modprobe vfio_iommu_type1
	fi
elif ! check_for_driver uio_pci_generic || modinfo uio_pci_generic > /dev/null 2>&1; then
	driver_name=uio_pci_generic

The SPDK system configuration guide says the same thing at a higher level: when an IOMMU is present and enabled, vfio-pci is recommended, and scripts/setup.sh automatically selects it. Some devices may require uio_pci_generic instead, but then IOMMU settings need extra care.

Failure patterns:

Kernel still owns the NVMe device: SPDK cannot directly drive it.
The device is in an IOMMU group with other devices still bound to host drivers: VFIO group is not viable.
/dev/vfio/$GROUP permissions exclude the target user: DPDK cannot open the group.
VFIO maps SPDK memory and hits the user's locked-memory limit: attach may fail and syslog may show memlock-related VFIO messages.
PCI allowlist excludes the target device: env initializes, but the expected controller does not appear.

`scripts/setup.sh` Is Runtime Preparation, Not Ceremony

The script is not a ceremonial install step. It prepares the Linux host so EAL and VFIO can do the things SPDK assumes at runtime.

Hugepage allocation is direct sysfs/procfs manipulation. HUGEMEM, NRHUGE, HUGEPGSZ, HUGENODE, SKIP_HUGE, CLEAR_HUGE, and PERSIST_HUGE all feed into this setup.

Excerpt from scripts/setup.sh:

check_hugepages_alloc() {
	local hp_int=$1
	local allocated_hugepages

	allocated_hugepages=$(< "$hp_int")

	if ((NRHUGE <= allocated_hugepages)) && [[ $SHRINK_HUGE != yes ]]; then
		echo "INFO: Requested $NRHUGE hugepages but $allocated_hugepages already allocated ${2:+on node$2}"
		return 0
	fi

	echo $((NRHUGE < 0 ? 0 : NRHUGE)) > "$hp_int"

	allocated_hugepages=$(< "$hp_int")
	if ((allocated_hugepages < NRHUGE)); then
		cat <<- ERROR

			## ERROR: requested $NRHUGE hugepages but $allocated_hugepages could be allocated ${2:+on node$2}.

The error text mentions memory fragmentation because Linux may be unable to reserve the requested number of physically contiguous hugepages after the system has been running. A reboot or early boot reservation can change the outcome.

The Linux configuration path also mounts hugetlbfs if needed, configures hugepages, adjusts mount ownership for the target user, and warns about memlock when VFIO is selected.

Excerpt from scripts/setup.sh:

function configure_linux() {
	configure_linux_pci
	hugetlbfs_mounts=$(linux_hugetlbfs_mounts)

	if [ -z "$hugetlbfs_mounts" ]; then
		hugetlbfs_mounts=/mnt/huge
		echo "Mounting hugetlbfs at $hugetlbfs_mounts"
		mkdir -p "$hugetlbfs_mounts"
		mount -t hugetlbfs nodev "$hugetlbfs_mounts"
	fi

	configure_linux_hugepages

	if [ "$driver_name" = "vfio-pci" ]; then
		if [ -n "$TARGET_USER" ]; then
			for mount in $hugetlbfs_mounts; do
				chown "$TARGET_USER" "$mount"
				chmod g+w "$mount"
			done

			MEMLOCK_AMNT=$(su "$TARGET_USER" -c "ulimit -l")
			if [[ $MEMLOCK_AMNT != "unlimited" ]]; then
				MEMLOCK_MB=$((MEMLOCK_AMNT / 1024))

This is why "run setup.sh" is not a vague support answer. It changes concrete kernel-facing state that EAL depends on: hugepage availability, hugetlbfs mount points, PCI driver binding, VFIO group access, and memlock visibility.

Mempools And Memzones

SPDK uses fixed-size object pools heavily because runtime allocation is expensive and failure-prone in hot I/O paths. A mempool is a named pool of fixed-size elements, often with per-core caches. It is a good fit for requests, completions, messages, and other objects that churn quickly.

The public mempool APIs live in include/spdk/env.h:

spdk_mempool_create()
spdk_mempool_create_ctor()
spdk_mempool_get()
spdk_mempool_get_bulk()
spdk_mempool_put()
spdk_mempool_put_bulk()
spdk_mempool_count()
spdk_mempool_lookup()

The DPDK-backed implementation clamps the per-lcore cache size so no more than half the objects can be trapped in caches, then calls rte_mempool_create().

Excerpt from lib/env_dpdk/env.c:

/* No more than half of all elements can be in cache */
tmp = (count / 2) / rte_lcore_count();
if (cache_size > tmp) {
	cache_size = tmp;
}

if (cache_size > RTE_MEMPOOL_CACHE_MAX_SIZE) {
	cache_size = RTE_MEMPOOL_CACHE_MAX_SIZE;
}

mp = rte_mempool_create(name, count, ele_size, cache_size,
			0, NULL, NULL, (rte_mempool_obj_cb_t *)obj_init, obj_init_arg,
			numa_id, 0);
if (mp == NULL && !g_enforce_numa && numa_id != SOCKET_ID_ANY) {
	mp = rte_mempool_create(name, count, ele_size, cache_size,
				0, NULL, NULL, (rte_mempool_obj_cb_t *)obj_init, obj_init_arg,
				SOCKET_ID_ANY, 0);
}

That cache clamp is a practical guardrail. If every lcore cache could hold too many objects, a pool might look empty to one core while many free objects sit in other cores' caches. That is not a malloc performance issue. It is pool sizing and cache behavior.

Memzones are named shared memory regions. They support cases where a component needs a named, aligned region rather than many small objects. SPDK asks DPDK for IOVA-contiguous memory unless the caller explicitly sets SPDK_MEMZONE_NO_IOVA_CONTIG.

Excerpt from lib/env_dpdk/env.c:

if ((flags & SPDK_MEMZONE_NO_IOVA_CONTIG) == 0) {
	dpdk_flags |= RTE_MEMZONE_IOVA_CONTIG;
}

if (numa_id == SPDK_ENV_NUMA_ID_ANY) {
	numa_id = SOCKET_ID_ANY;
}

mz = rte_memzone_reserve_aligned(name, len, numa_id, dpdk_flags, align);
if (mz == NULL && !g_enforce_numa && numa_id != SOCKET_ID_ANY) {
	mz = rte_memzone_reserve_aligned(name, len, SOCKET_ID_ANY, dpdk_flags, align);
}

if (mz != NULL) {
	memset(mz->addr, 0, len);
	return mz->addr;
}

Examples elsewhere:

lib/thread/thread.c:_thread_lib_init() creates g_spdk_msg_mempool for cross-thread messages.
lib/event/reactor.c:spdk_reactors_init() creates g_spdk_event_mempool for events.
lib/thread/iobuf.c:spdk_iobuf_initialize() creates shared iobuf backing pools through lower-level ring and memory helpers.

NUMA Is A Performance Feature And A Failure Mode

SPDK often runs with one or more reactors pinned to cores. Memory locality matters because a core polling an NVMe qpair or transport queue may touch buffers, descriptors, and completion state millions of times per second.

struct spdk_env_opts includes enforce_numa. In lib/env_dpdk/init.c:build_eal_cmdline(), this calls mem_enforce_numa(). In lib/env_dpdk/env.c:spdk_malloc(), spdk_zmalloc(), spdk_memzone_reserve_aligned(), and spdk_mempool_create_ctor(), allocation falls back to SOCKET_ID_ANY when allocation on the requested NUMA node fails unless NUMA is enforced.

That means NUMA can be both a performance feature and a failure mode:

Without strict NUMA, startup may succeed but memory may land on a remote socket.
With strict NUMA, startup or a later pool allocation may fail if the requested node lacks hugepages.
With uneven hugepage reservation across nodes, one reactor may have local memory while another pays remote-memory latency.

Beginner rule: if performance is unexpectedly uneven, inspect NUMA. If startup fails only with strict NUMA options, inspect hugepage distribution per NUMA node.

Edge Cases And Failure Modes

Core mask and lcore map both set: build_eal_cmdline() rejects the configuration before calling DPDK. DPDK also documents that only one of --lcores, -l, or -c should be used at a time.

Neither core mask nor lcore map set at env level: rejected. The event framework may set defaults before env init, but the env layer itself requires one CPU selection mechanism.

Malformed core syntax: DPDK may reject it inside rte_eal_init(). If startup logs show the EAL parameter list, inspect the final -c, -l, or --lcores value, not only the original application config.

--no-huge combined with hugepage-specific options: rejected by SPDK before EAL init.

--no-huge without explicit memory sizing: rejected by SPDK because the no-huge path requires a known allocation size.

iova-mode=pa with --no-huge: rejected. SPDK forces VA mode in that no-huge path.

--no-pci: valid for some tests and non-PCI tools, but it disables vtophys map creation. It is not a way to make PCI NVMe work with fewer permissions.

Root or privilege errors: app_setup_env() logs that the user may need root if spdk_env_init() fails and getuid() != 0. That message is intentionally broad. The real problem might be hugetlbfs permissions, /dev/vfio/$GROUP permissions, device binding, or memlock.

VFIO group not viable: if a device shares an IOMMU group with another device still bound to a host driver, DPDK/VFIO may refuse the group. scripts/setup.sh warns about this exact case.

Hugepages unavailable: DPDK may report no available hugepages, no mounted hugetlbfs for the requested size, or allocation failures. The cause may be configuration, permissions, page size mismatch, NUMA distribution, or fragmentation.

Reinitialization has special rules: spdk_env_init(NULL) is used after a prior spdk_env_fini() in the same process. Passing non-NULL options during reinitialization is rejected.

opts_size too small: newer fields may retain defaults. Both app and env options use opts_size to preserve ABI compatibility.

Misconceptions To Kill

"SPDK bypasses Linux, so Linux setup does not matter." It bypasses parts of the kernel I/O path, but it depends heavily on Linux hugepages, VFIO/IOMMU, PCI binding, and process permissions.
"A reactor mask is just an SPDK preference." It becomes an EAL CPU argument and determines where OS threads are launched.
"DMA-safe memory is just aligned memory." Alignment is necessary but not sufficient. The memory must be pinned, registered, translated, or mapped for the device path.
"If env init succeeds, all NVMe devices are ready." Env init means the platform is ready. Controllers still need probing, attachment, bdev creation, and subsystem config.
"Mempool exhaustion is like malloc slowness." In hot paths, exhaustion usually means a designed backpressure path, NOMEM retry path, or fatal configuration error.
"VFIO permission is per device." Traditional VFIO access is group-based. If the group contains multiple devices, the group is the permission and viability unit.

Diskengine Relevance

In an excloud diskengine-style deployment, SPDK often runs as an external daemon controlled by RPC. If the daemon never reaches RPC runtime state, diskengine cannot reconcile devices, volumes, or exports.

When diagnosing an early failure, classify it before chasing bdev code:

Env failure: EAL rejects arguments, hugepages unavailable, VFIO missing, memlock too low, PCI group not viable.
Startup failure: reactors or app thread fail after env init.
Subsystem failure: one subsystem init callback returns non-zero.
Config failure: startup or runtime JSON RPC fails.

This chapter covers the first class. The practical debug move is to capture the printed EAL parameter list, the effective user, scripts/setup.sh status, hugepage state under /sys/devices/system/node, /proc/meminfo, IOMMU boot parameters, and current PCI bindings before moving up into bdev configuration.

Prose Diagram: Address Translation Path

Imagine a write buffer as a card moving through five boxes:

The application has a C pointer, like 0x7f....
The pointer comes from spdk_dma_zmalloc(), so it belongs to SPDK/DPDK managed memory.
SPDK's memory map knows the virtual range.
spdk_vtophys() translates it to an address valid for the current IOVA mode.
A device or transport can use that address in a descriptor, SGE, or DMA mapping.

If the card starts from plain malloc(), it may fall out between boxes 2 and 3.

flowchart LR A["C pointer"] --> B["SPDK DMA allocator"] B --> C["DPDK hugepage / memory segment"] C --> D["SPDK mem map"] D --> E["spdk_vtophys()"] E --> F["IOVA / DMA address"] F --> G["Device descriptor"] X["plain malloc pointer"] -. may not be registered .-> D

Source Reading Exercise

Read these functions in order:

lib/event/app.c:spdk_app_start()
lib/event/app.c:app_copy_opts()
lib/event/app.c:app_setup_env()
lib/env_dpdk/init.c:spdk_env_init()
lib/env_dpdk/init.c:build_eal_cmdline()
lib/env_dpdk/init.c:spdk_env_dpdk_post_init()
lib/env_dpdk/memory.c:vtophys_init()
lib/env_dpdk/memory.c:spdk_vtophys()
lib/env_dpdk/env.c:spdk_dma_malloc_socket()
scripts/setup.sh:configure_linux_pci()
scripts/setup.sh:configure_linux_hugepages()
scripts/setup.sh:configure_linux()

Questions while reading:

Where is the default reactor mask chosen before env init?
Which options are copied from app opts into env opts?
What final EAL arguments are printed for your app?
What happens if DPDK returns EALREADY?
Which function initializes vtophys?
Which options disable or alter vtophys behavior?
Which setup script branch chooses vfio-pci?
Which setup script warning maps to VFIO group viability?
Which failures happen before any bdev module has a chance to run?

Primary References

SPDK System Configuration User Guide: https://spdk.io/doc/system_configuration.html
DPDK Environment Abstraction Layer programmer guide: https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html
DPDK Linux EAL parameters: https://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html
DPDK Linux system requirements and hugepage setup: https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html?highlight=hugepages
Linux kernel VFIO documentation: https://docs.kernel.org/driver-api/vfio.html

Reader Promise

Mental Model

Where The Env Fits In Startup

Source Anchors

The Two Option Structures

How EAL Arguments Are Built

No PCI, No Huge, Hugepage Options, And IOVA

What spdk_env_init() Actually Does

Hugepages And Why Normal malloc() Is Not Enough

Vtophys And IOVA

PCI Ownership And VFIO

scripts/setup.sh Is Runtime Preparation, Not Ceremony

Mempools And Memzones

NUMA Is A Performance Feature And A Failure Mode

Edge Cases And Failure Modes

Misconceptions To Kill

Diskengine Relevance

Prose Diagram: Address Translation Path

Source Reading Exercise

Primary References

What `spdk_env_init()` Actually Does

Hugepages And Why Normal `malloc()` Is Not Enough

`scripts/setup.sh` Is Runtime Preparation, Not Ceremony