feat(win): fix silent CPU fallback, launcher suite, DirectStorage expert loads - #670
Open
khalilswdp wants to merge 4 commits into
Open
feat(win): fix silent CPU fallback, launcher suite, DirectStorage expert loads#670khalilswdp wants to merge 4 commits into
khalilswdp wants to merge 4 commits into
Conversation
5 tasks
Owner
khalilswdp
force-pushed
the
feat/windows-gpu-suite
branch
from
July 28, 2026 21:27
1e78fec to
58a241a
Compare
khalilswdp
force-pushed
the
feat/windows-gpu-suite
branch
from
July 28, 2026 23:03
58a241a to
d74d087
Compare
Owner
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On Windows every run silently fell back to CPU:
coli_cuda.dlllinked dynamiccudart, but CUDA 13.x keeps
cudart64_*.dllinbin\x64(not on PATH), soLoadLibraryfailed quietly. Fix:-cudart static(the DLL then imports onlyKERNEL32). The rest makes the fix visible and usable:
colilauncher: banner prints the real backend instead of a hardcoded"streaming CPU"; numeric env vars parse tolerantly (
CUDA_EXPERT_GB=autonolonger crashes);
--ramaccepts fractional budgets.--ramwouldbuy a second cache slot, instead of unconditionally blaming
PIN_GB.windows/*.ps1suite: Build (toolchain detect + doctor), Run (wizard,attaches to a running serve instead of starting a second engine), Chat, Web,
Warmup. Saved with a UTF-8 BOM — PowerShell 5.1 reads BOM-less files as
ANSI, which breaks the multilingual warmup prompts.
warmup.ps1:-Backend auto|gpu|cpu(the.coli_usagepin isbackend-flavoured),
-PromptFile, +40 multilingual/code prompts.COLI_DSTORAGE=1, default off): themiss path's coalesced ~19 MB slab pread can be served by a DirectStorage
host-destination DMA instead. Pure transport swap — bytes land in the slab
the expert cache already owns; cache/eviction/layout untouched; primary
replica only; any failure (no SDK at build, no
dstorage.dll, no adapter,request error) falls back to pread. The depot PR reuses this transport for
disk→VRAM. No performance claims yet — correctness-verified only.
Validation
make -C c checkfull pass at this tip; zero new warningsmake CUDA_DLL=1 colibri.execlean; DLL builds with and withoutDSTORAGE_HOMECOLI_DSTORAGE=0vs1byte-identical output; served loads reported at exit
.ps1scripts parse under Windows PowerShell 5.1 and PS 7box where the dynamic link failed silently
Compatibility
opt-in at build time (
DSTORAGE_HOME), optional in the loader;Linux/macOS unchanged