Running a local LLM with Intel SYCL

This is the practical recipe used on this machine. It replaces Ollama's Vulkan inference path with llama.cpp's SYCL backend, which talks to the Intel GPU through Level Zero. The dashboard still works; only its model backend changed.

What was installed here

No Mesa upgrade was required. SYCL uses Intel's Level Zero/oneAPI compute stack, rather than Mesa's Vulkan inference path.

The average-user recipe

  1. Check the hardware and permissions. This route is intended for a supported Intel Arc, Flex, Max, or recent Intel iGPU. Confirm /dev/dri/renderD128 exists and that your user belongs to the render and video groups. Log out and back in after changing group membership.

  2. Install the Intel compute runtime and oneAPI components. On a supported Ubuntu release, Intel's APT instructions are the simplest route. Install Intel Deep Learning Essentials, then load its environment:

    source /opt/intel/oneapi/setvars.sh
sycl-ls

    sycl-ls should show an Intel GPU through Level Zero. We used a user-local offline installer because this host's Ubuntu version was newer than the installer's officially recognized list.

  3. Get a matching SYCL build of llama.cpp. Download the Ubuntu sycl-fp16-x64 archive from a llama.cpp release. The oneAPI runtime version must match the version used to build that archive; the release workflow records it. Alternatively, build from source:

    source /opt/intel/oneapi/setvars.sh
cmake -B build -DGGML_SYCL=ON -DGGML_SYCL_F16=ON \
 -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx
cmake --build build --config Release -j

  4. Choose a GGUF model that fits. Allow room for the model weights, KV cache, compute buffers, and desktop use. Quantized Q4 models are a sensible starting point. Larger context windows consume more memory and make prompt ingestion progressively slower.

  5. Start the server. A minimal equivalent is:

    source /opt/intel/oneapi/setvars.sh
export ONEAPI_DEVICE_SELECTOR=level_zero:gpu
./llama-server --model /path/model.gguf --device SYCL0 \
 --n-gpu-layers 999 --ctx-size 65536 --flash-attn on \
 --cache-type-k f16 --cache-type-v f16 \
 --host 127.0.0.1 --port 8080

    On this machine, scripts/run-sycl.sh contains the complete tuned command. systemd/llama-sycl.service makes it persistent.

  6. Test it before adding clients. With authentication enabled here:

    KEY=$(cat .runtime/llama-api-key)
curl -H "Authorization: Bearer $KEY" http://127.0.0.1:8080/v1/models

  7. Connect an OpenAI-compatible client. Use http://127.0.0.1:8080/v1, model ID qwen3.8, and the bearer key in .runtime/llama-api-key. OpenCode is configured this way globally on this machine.

LAN access on this machine

The server listens on all interfaces at port 8080. From another device on the same LAN, use:

http://192.168.2.176:8080/v1

Use the contents of .runtime/llama-api-key as the API key. Treat it like a password. The API is plain HTTP, so it is suitable only for a trusted LAN; use Tailscale, a reverse proxy with TLS, or an SSH tunnel across untrusted networks. UFW is disabled on this host. If another firewall is enabled later, allow TCP 8080 only from the local subnet—not from the public internet.

llama.cpp permits some read-only discovery routes, including the model list, without a key. Inference routes are protected: an unauthenticated LAN completion request was verified to return HTTP 401.

Operational notes

Primary references