Technology

RPC

5 synchronized stories carry this tag.

llama.cpp extends accelerator support and hardens failure paths

Signed prereleases b11202 through b11222 add F16 CUDA and larger-width SYCL fast Walsh-Hadamard transforms, CUDA support for Nemotron 3 Puzzle state size 96, tuned CUDA and HIP attention paths, an Adreno Vulkan argsort correction, Jinja dict support, safer grammar failure handling and cleaner RPC option parsing.

Why it matters: The sequence mixes new model and accelerator paths with a correctness fix and replacements for process aborts or configuration side effects. Those changes can affect output validity, server availability and performance on exact hardware.

Read storyOriginal source · github.com

llama.cpp adds embedded-GGUF and Qwen correctness fixes

The 17 September signed prerelease sequence expanded through b11028. Beyond the earlier fused-QKV, NVIDIA Vulkan and SYCL fixes, b11019 corrects wrong tensor data when a GGUF is embedded at an unaligned file offset, b11020 adds missing DeepSeek V3.2 and V4 message delimiters, b11026 repairs fused-MTP loading for Qwen3.5-MoE, and b11027 prevents RPC from selecting unsupported accelerator devices.

Why it matters: The later builds add silent-data-corruption, chat-boundary and model-loading reasons to retest the same date's canary. They also show why a morning release scan cannot be treated as the final artifact identity for a fast-moving runtime.

Read storyOriginal source · github.com

llama.cpp fixes an unauthenticated RPC remote-code-execution path

Later signed llama.cpp prereleases on 16 September added a documented fix for an unauthenticated, network-reachable use-after-free in RPC graph caching that could leak libc addresses and enable remote code execution. The sequence through b11007 also repaired int16 transpose corruption on SpaceMiT hardware and retained the earlier OpenCL, ROCm and Hexagon backend changes.

Why it matters: An exposed llama.cpp RPC server could turn a performance cache into a host-compromise boundary, while the SpaceMiT defect could silently corrupt most values in a 4-by-4 transpose. These are security and correctness reasons to replace the earlier 16 September…

Read storyOriginal source · github.com

llama.cpp fixes MiniCPM3 Metal and adds sparse Vulkan attention

Ten signed llama.cpp prereleases through b10985 fixed Android packaging, added CUDA 13.4 Windows builds, supplied a missing MiniCPM3 Metal flash-attention kernel, expanded OpenCL and OpenVINO paths, added sparse Vulkan flash attention for DeepSeek V4 and GLM while avoiding nondeterministic atomic addition, and limited RPC hash caching to weights.

Why it matters: The run combines concrete startup and correctness fixes with new accelerator paths. The MiniCPM3 kernel and Vulkan determinism change can affect whether a model runs and whether repeated inference is reproducible on the affected backends.

Read storyOriginal source · github.com

llama.cpp tunes Apple inference and RPC allocation paths

Five llama.cpp pre-release builds landed on 30 August in India. Builds b10696 and b10697 add flash-attention vector tuning for M3 Pro and M3 Ultra; b10698 removes Apple RDMA teardown error noise; b10699 adds an allocation-size expansion signal used by RPC; and b10695 updates the bundled cpp-httplib dependency.

Why it matters: Local and distributed inference behavior depends on hardware-specific kernels, teardown paths, allocation contracts, and bundled networking code. These small changes can affect stability and speed even when model weights do not change.

Read storyOriginal source · github.com