Heretek-AI/Strata
1
0 Fork 0
mirror of https://github.com/Heretek-AI/Strata.git synced 2026-11-07 17:19:46 -10:00
Code Issues Packages Projects Releases Wiki Activity

dsv41 (Windows HIP, integrated GPU): the shards' views are dropped after the load by default (DS4_NO_VIEW=0 turns it off, =1 forces it elsewhere)

Browse Source 
Unbuffered reads of a file that has a view in the process run at half the drive's rate on Windows (probe on Strix Halo: 6.5 GB/s with no view, 3.2 with one). Default on for an integrated GPU, where the CPU lane's misses read through the store's slots. Decode 9.1 -> 9.4-9.7 tok/s (code, drafter off, 2 interleaved pairs); the 32K prompt is unchanged; no failed reads in any run.
This commit is contained in:
Niko1221 committed 2026-10-30 18:43:29 +01:00
1 parent 19b5f873f2
commit 957038a1cd
1 file changed
+10 -1
Download Patch File Download Diff File Expand all files Collapse all files
src/dsv41/ds4_model.cu
+10 -1
View File
@@ -3183,7 +3183,16 @@ struct Model::Impl {
calibrate_paths();
init_store(); // (last: the store's arena is registered with the driver - page-locked memory the other pinned allocations must come before)
preload_store();
if (std::getenv("DS4_NO_VIEW") != nullptr && std::atoi(std::getenv("DS4_NO_VIEW")) != 0 && store && pk.gguf) {
// On by default for a Windows HIP build on an APU (Strix Halo: measured +7.5% decode, no change in the prompt); DS4_NO_VIEW=1 forces it
// on anywhere, DS4_NO_VIEW=0 turns it off
bool no_view = std::getenv("DS4_NO_VIEW") != nullptr && std::atoi(std::getenv("DS4_NO_VIEW")) != 0;
#if defined(_WIN32) && defined(STRATA_USE_HIP)
if (std::getenv("DS4_NO_VIEW") == nullptr && !devs.empty()) {
cudaDeviceProp vp{};
no_view = cudaGetDeviceProperties(&vp, devs[0].id) == cudaSuccess && vp.integrated != 0;
}
#endif
if (no_view && store && pk.gguf) {
pk.drop_views();
store->set_pfds(pk.pfds);
views = false;