software-engineering/spark-vllm-docker
2
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

added -e parameter

Browse Source
This commit is contained in:
Eugene Rakhmatulin
2026-01-29 13:06:22 -08:00
parent b58ba7b19a
commit 7a81e90cd2
2 changed files with 9 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@@ -144,11 +144,13 @@ Don't do it every time you rebuild, because it will slow down compilation times.
For periodic maintenance, I recommend using a filter: `docker builder prune --filter until=72h` For periodic maintenance, I recommend using a filter: `docker builder prune --filter until=72h`
### 2026-01-26 ### 2026-01-29
Added `-e` / `--env` parameter to `launch-cluster.sh` to pass environment variables to the container.
Added an experimental build option, optimized for DGX Spark and gpt-oss models by [Christopher Owen](https://github.com/christopherowen/spark-vllm-mxfp4-docker/blob/main/Dockerfile). Added an experimental build option, optimized for DGX Spark and gpt-oss models by [Christopher Owen](https://github.com/christopherowen/spark-vllm-mxfp4-docker/blob/main/Dockerfile).
It is currently the fastest way to run GPT-OSS on DGX Spark, achieving 60 t/s on a single Spark. It's currently not working in a cluster configuration. It is currently the fastest way to run GPT-OSS on DGX Spark, achieving 60 t/s on a single Spark.
To use this build, first build the container with `--exp-mxfp4` flag. I recommend using a separate label as it is currently not recommended to use this build for models other than gpt-oss: To use this build, first build the container with `--exp-mxfp4` flag. I recommend using a separate label as it is currently not recommended to use this build for models other than gpt-oss:
@@ -183,7 +185,7 @@ Then, to run on a single Spark:
--max-num-batched-tokens 8192" --max-num-batched-tokens 8192"
``` ```
On a Dual Spark cluster (**CURRENTLY NOT WORKING**): On a Dual Spark cluster:
```bash ```bash
./launch-cluster.sh -t vllm-node-mxfp4 exec vllm serve \ ./launch-cluster.sh -t vllm-node-mxfp4 exec vllm serve \
@@ -521,7 +523,7 @@ The script attempts to automatically detect:
You can override the auto-detected values if needed: You can override the auto-detected values if needed:
```bash ```bash
./launch-cluster.sh --nodes "10.0.0.1,10.0.0.2" --eth-if enp1s0f1np1 --ib-if rocep1s0f1 ./launch-cluster.sh --nodes "10.0.0.1,10.0.0.2" --eth-if enp1s0f1np1 --ib-if rocep1s0f1 -e MY_ENV=123
``` ```
| Flag | Description | | Flag | Description |
@@ -531,6 +533,7 @@ You can override the auto-detected values if needed:
| `--name` | Container name (default: `vllm_node`). | | `--name` | Container name (default: `vllm_node`). |
| `--eth-if` | Ethernet interface name. | | `--eth-if` | Ethernet interface name. |
| `--ib-if` | InfiniBand interface name. | | `--ib-if` | InfiniBand interface name. |
| `-e, --env` | Environment variable to pass to container (e.g. `-e VAR=val`). Can be used multiple times. |
| `--apply-mod` | Apply mods/patches from specified directory. Can be used multiple times to apply multiple mods. | | `--apply-mod` | Apply mods/patches from specified directory. Can be used multiple times to apply multiple mods. |
| `--nccl-debug` | NCCL debug level (e.g., INFO, WARN). Defaults to INFO if flag is present but value is omitted. | | `--nccl-debug` | NCCL debug level (e.g., INFO, WARN). Defaults to INFO if flag is present but value is omitted. |
| `--check-config` | Check configuration and auto-detection without launching. | | `--check-config` | Check configuration and auto-detection without launching. |

2
launch-cluster.sh
View File

@@ -35,6 +35,7 @@ usage() {
echo " --name Container name (Optional, default: $DEFAULT_CONTAINER_NAME)" echo " --name Container name (Optional, default: $DEFAULT_CONTAINER_NAME)"
echo " --eth-if Ethernet interface (Optional, auto-detected)" echo " --eth-if Ethernet interface (Optional, auto-detected)"
echo " --ib-if InfiniBand interface (Optional, auto-detected)" echo " --ib-if InfiniBand interface (Optional, auto-detected)"
echo " -e, --env Environment variable to pass to container (e.g. -e VAR=val)"
echo " --nccl-debug NCCL debug level (Optional, one of: VERSION, WARN, INFO, TRACE). If no level is provided, defaults to INFO." echo " --nccl-debug NCCL debug level (Optional, one of: VERSION, WARN, INFO, TRACE). If no level is provided, defaults to INFO."
echo " --apply-mod Path to directory or zip file containing run.sh to apply before launch (Can be specified multiple times)" echo " --apply-mod Path to directory or zip file containing run.sh to apply before launch (Can be specified multiple times)"
echo " --check-config Check configuration and auto-detection without launching" echo " --check-config Check configuration and auto-detection without launching"
@@ -52,6 +53,7 @@ while [[ "$#" -gt 0 ]]; do
--name) CONTAINER_NAME="$2"; shift ;; --name) CONTAINER_NAME="$2"; shift ;;
--eth-if) ETH_IF="$2"; shift ;; --eth-if) ETH_IF="$2"; shift ;;
--ib-if) IB_IF="$2"; shift ;; --ib-if) IB_IF="$2"; shift ;;
-e|--env) DOCKER_ARGS="$DOCKER_ARGS -e $2"; shift ;;
--apply-mod) MOD_PATHS+=("$2"); shift ;; --apply-mod) MOD_PATHS+=("$2"); shift ;;
--nccl-debug) --nccl-debug)
if [[ -n "$2" && "$2" =~ ^(VERSION|WARN|INFO|TRACE)$ ]]; then if [[ -n "$2" && "$2" =~ ^(VERSION|WARN|INFO|TRACE)$ ]]; then