Memory, Vulkan, and AMD Strix Halo

A machine with 128GB unified memory appears to offer a simple route to large local models: reserve memory for the GPU, load a quantized model, and run inference.

The construction record for this cluster is more complicated. It includes earlier ROCm allocation failures, a move to Vulkan through Mesa RADV, reported improvements after micro-batch changes, and later captures of kernel-level memory errors.

This article preserves those observations without treating them as a controlled comparison or a universal configuration recipe. The September 2026 baseline is an observed running deployment. Multi-hour stability and repeated-run failure rates belong to the next phase.

1. Memory Is More Than the Weight File#

On each node, the operator selected a 96GB BIOS UMA allocation from 128GB unified memory. DRM reports a 96 GiB GPU-visible VRAM region, while Linux reports roughly 30 GiB of conventional system RAM.¹

The layers are easier to understand as a map than as one number:

128GB physical unified memory
          |
          v
96GB BIOS UMA setting
          |
          +-- Linux-visible system RAM
          +-- GPU-visible VRAM
          |
          v
Driver / memory-management interfaces
          |
          v
Model weights + context state + compute / vision buffers

The model file's on-disk size is not a complete runtime-memory budget. The measurements below describe different interfaces and different capture moments. They should not be added together as if they were independent physical memory banks.

2. The 96 GiB DRM Baseline#

All three nodes report the same total through the DRM memory interface:¹

mem_info_vram_total     = 103079215104 bytes
mem_info_vis_vram_total = 103079215104 bytes

That byte count is exactly 96 GiB. Representative amdgpu boot messages are:

VRAM: 98304M
Detected VRAM RAM=98304M, BAR=98304M
98304M of VRAM memory ready

The raw M spelling is preserved inside the log excerpt. In explanatory prose, byte-derived binary values use GiB. The BIOS label remains 96GB because that is the operator-reported setting.

These are GPU-visible allocations in an integrated-memory system, not a claim that separate 96 GiB discrete VRAM chips exist.

3. Same UMA Setting, Different GTT and TTM Values#

The captured values are:

All columns come from the same collection of system-baseline records.¹

Server01 reports:

100000M of GTT memory ready
mem_info_gtt_total = 104857600000 bytes
pages_limit = 25600000
page_pool_size = 25600000

Server03 reports:

15603M of GTT memory ready
mem_info_gtt_total = 16361816064 bytes
pages_limit = 3994584
page_pool_size = 3994584

The GTT figures are approximately 97.66 GiB, 15.24 GiB, and 15.24 GiB. Earlier command output rounded these to 98G and 16G; the table retains the byte counts to avoid treating that display rounding as exact capacity.

Observed: identical BIOS UMA settings coexist with different GTT/TTM reports.

Not established: why those values differ. The machines differ in firmware, distribution, package builds, and potentially configuration history. The same short kernel release label does not establish identical kernel packages. No controlled A/B comparison isolated these variables.

The record does not justify attributing the difference specifically to Mesa, Ubuntu, or a single kernel setting. It also does not establish that the reported GTT total is an additional physically reserved pool on top of VRAM and system RAM.

4. Earlier ROCm Experiments#

The construction notes record an earlier large-model offload failure with messages including:²

ggml_backend_cuda_buffer_type_alloc_buffer:
allocating 29632.42 MiB on device 0:
cudaMalloc failed: out of memory

llama_model_load_from_file_impl:
using device ROCm0 (Radeon 8060S Graphics) - 29980 MiB free

The cuda spelling is preserved from the recorded message even though the experiment was described as using ROCm/HIP. It is not relabeled as evidence of NVIDIA hardware.

The defensible conclusion is narrow: the recorded ROCm configuration failed during a large allocation. The complete earlier ROCm version, build configuration, and workload trace are not attached to this publication baseline.

This does not establish a permanent “30GB ROCm limit,” a universal TTM one-third allocation rule, or a current comparison between ROCm and Vulkan. Those would require separate evidence.

5. Why the Deployment Moved to Vulkan#

The construction notes describe moving the large-model workloads to llama.cpp built with GGML_VULKAN=ON. The later system records identify Mesa RADV, and the runtime captures show the large models loaded through the resulting deployment.²¹³

The official repository is now published under ggml-org/llama.cpp. The following preserves the documented build pattern, with a recorded revision selected explicitly rather than implying that the latest branch reproduces the baseline.⁴

# Historical build pattern, not an instruction to upgrade a running node.
sudo apt update
sudo apt install -y \
  build-essential cmake vulkan-tools libvulkan-dev \
  glslc glslang-tools libshaderc-dev spirv-headers \
  mesa-vulkan-drivers

# Run in a separate workspace rather than over an existing installation.
git clone https://github.com/ggml-org/llama.cpp.git llama.cpp-vulkan
cd llama.cpp-vulkan

# Recorded Server01 source revision. Use the target node's recorded revision.
git checkout 64e9bceb2c3a856efed96feda784a50947049feb

cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j"$(nproc)"

Package repositories can change. These commands alone do not reconstruct the historical compiler, distribution packages, firmware, or driver. They are the build approach used in the notes, not a newly tested installation script or a full environment lockfile.

6. Current Vulkan and Runtime Identity#

The version records also distinguish the Vulkan instance from the device API:¹

The exact retained identities are:

LLM-Server01
  Kernel package build: #31~24.04.1-Ubuntu
  Mesa package: 25.2.8-0ubuntu0.24.04.2
  llama.cpp: 0.4.0-dev, build 10858
  Commit: 64e9bceb2c3a856efed96feda784a50947049feb

LLM-Server02
  Kernel package build: #31-Ubuntu
  Mesa package: 26.0.8-1ubuntu0.3
  llama.cpp: 0.4.1-dev, build 11047
  Commit: b23701f77d47dad9de834d59ebfcbe25c9e8b46f

LLM-Server03
  Kernel package build: #31-Ubuntu
  Mesa package: 26.0.8-1ubuntu0.3
  llama.cpp: 0.4.1-dev, build 11061
  Commit: b23efaa2ef147f547ee75cbf0c621d61904de80e

These are the identifiers reported locally. Referencing current upstream documentation does not replace the captured version history.

7. What LimitMEMLOCK=infinity Establishes#

All three captured service definitions include:

LimitMEMLOCK=infinity
LimitNOFILE=65535

The associated systemd properties report those values as active.⁵

LimitMEMLOCK controls the process resource limit for locked memory. It is permission to lock memory, not evidence that a process actually locked its model, removed every driver allocation limit, or gained additional physical RAM.⁶

The record contains no isolated comparison with and without this setting. Its justified status is part of the captured service configuration, not a proven prerequisite for all Vulkan inference. The launch commands also do not, by themselves, establish which memory-locking operations occurred internally.

8. Keep the Two Memory Measurement Series Separate#

The construction notes contain representative monitoring values attributed to radeontop / amdgpu_top observation:²

These retain the recorded MB unit label. The source does not attach a complete tool/version/time record to every snapshot, so the numbers are not silently converted into a different counter's unit system.

The later direct DRM readings are:

Their denominator is exactly 103,079,215,104 bytes = 96 GiB.¹

The earlier monitor denominator of 98,148 MB and the DRM total are not presented as identical measurements. Counter definitions, reservations, display units, and capture timing may matter; the record does not isolate their contributions.

Both series place Server02 closest to its reported ceiling. They do not establish a safe maximum occupancy or the peak allocation required by a future image or long-context request.

9. Model Residence and Submission Errors Are Separate Observations#

The captured Server02 runtime shows a loaded Qwen2.5-VL-72B service. Its kernel history also contains this repeated pair:⁵³

amdgpu_vm_validate() failed.
Not enough memory for command submission!

The first shown pair occurs around kernel monotonic time 41896.499 seconds, followed by a burst of similar messages. Those timestamps identify the event within the kernel log; they do not identify the application request that triggered it.

What is established: the driver reported the errors, and the later collection also found a running VLM process.

What is not established: whether the same VLM process was running during that burst, which prompt or image was active, whether another GPU client contributed, or whether GTT/VRAM occupancy was the direct cause.

Loading a model and successfully executing a workload are different tests. This record illustrates why both should be captured, but it is not a completed causal reconstruction of that particular incident.

10. Earlier Vulkan Device-Loss Reports#

The construction notes preserve earlier messages including:²

radv/amdgpu: The CS has been cancelled because the context is lost.
This context is innocent.
ggml_vulkan: device lost on Vulkan0
srv update_slots: decode() failed: vk::Queue::submit: ErrorDeviceLost

The notes associate those incidents with aggressive long-prompt or multimodal testing and report improvement after reducing micro-batch size.

That is useful historical information. It does not prove a GPU timeout as the sole cause, demonstrate elimination of every device-loss condition, or establish a long-run failure rate. A controlled reproduction with paired settings and workload traces is outside this baseline.

11. The Operational Batch Pattern#

The large-model launch commands use:

-b 2048
-ub 256

In the documented interface, -b is the logical maximum batch size and -ub the physical maximum micro-batch size.⁷

The gpt-oss-120B production pattern is:

-c 131072 -b 2048 -ub 256 --parallel 1 --flash-attn on

The first value is from the production script. The earlier sampled 256k process is retained as a separate experiment, not blended into the production record.⁸

The construction history explains why a smaller micro-batch was selected. It does not establish that 256 is universally optimal, or that the same value should be copied without testing another model or build.

12. Embedding and Reranking Settings Are Adjusted#

On Server03, the captured startup logs for the embedding and reranking services include:³

embeddings enabled with n_batch (2048) > n_ubatch (256)
setting n_batch = n_ubatch = 256 to avoid assertion failure

The effective batch setting is therefore different from the literal command line for those services.

Older RAG logs also existed on Server01, with different micro-batch and slot settings. The current process inventory shows only the 120B service there. Those historical files are not evidence of additional currently running services and are not merged with Server03's active configuration.

The practical rule is: record the process arguments and the matching startup interpretation, not just a script or any file that happens to be in the log directory.

13. Different Messages Describe Different Layers#

A warning or automatic correction is not necessarily a failure. Layer and severity should be identified before attempting a diagnosis.

14. Conditions of the Running Baseline#

The running deployment combines 96GB BIOS UMA, Vulkan/RADV, separated large-model nodes, the recorded service limits, 256-token micro-batches, distinct per-model logs, and unallocated memory at the captured moments.

Some are design choices. Some are settings confirmed by capture. Micro-batch improvement is reported in the construction history. Their individual contribution to stability was not isolated.

Calling these the conditions of the observed baseline is more accurate than calling each one a proven fix. Long-duration stability remains explicitly unmeasured.

15. What I Would Not Generalize#

I would not attribute the GTT difference to the distribution name alone. I would not describe an older ROCm result as its current behavior. I would not treat DeviceLost as a complete diagnosis or a successful load as a stability guarantee.

I would also not infer that an OS memory counter is a simple measure of how much model weight can be added. It belongs to a particular interface, allocation policy, and capture state.

16. Reproducibility Record#

A useful memory-related comparison needs hardware and BIOS identity, UMA setting, full OS/kernel package identity, Mesa and Vulkan details, exact runtime build, model and projector files, requested and effective context, logical and physical batch sizes, parallel slots, service limits, and workload details.

Where an error is central to a claim, the record should preserve the relevant application and kernel logs with correlated timestamps. Peak usage under the workload is more informative than an unrelated idle snapshot.

The current series contains substantial portions of that record. It does not pretend the missing historical test context has been recovered.

17. Current Interpretation#

The captures establish a deployment of large quantized models on this 96GB UMA configuration. The construction history explains the route taken to reach it.

The central lesson is not simply that “96GB is enough.” It is that capacity, the driver/runtime stack, workload requirements, and operational policy have to be considered together.

18. Open Questions#

The reason for the GTT/TTM differences remains unresolved. So does the exact workload behind the Server02 error burst. A new ROCm comparison would require a separately recorded experiment, and a safe sustained occupancy threshold for the VLM node would require longer workload tests.

These are future questions, not prerequisites for publishing an accurately bounded construction baseline.

19. Next: Context Is a Separate Question#

A model can be loaded while its service exposes less context than the command requested. The next report follows that distinction from model metadata through startup logs to actual rejected requests.

Continue with Part 3: Model Context vs. Runtime Context.

Evidence and Reading Notes#