software-engineering/spark-vllm-docker
2
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

Experimental mod to support gpu-memory-utilization-gb

Browse Source
This commit is contained in:
Eugene Rakhmatulin
2026-03-12 13:37:44 -07:00
parent 8fec9bed06
commit 8ae51192e5
2 changed files with 261 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
mods/gpu-mem-util-gb/run.sh Normal file
View File

@@ -0,0 +1,6 @@
#!/bin/bash
set -e
patch -p1 -d /usr/local/lib/python3.12/dist-packages < gpu_mem.patch \
&& echo "=====> You can now use --gpu-memory-utilization-gb parameter to specify reserved memory in GiB"