From 7a81e90cd2213f90071c539af8582f5d9ef90ca2 Mon Sep 17 00:00:00 2001 From: Eugene Rakhmatulin Date: Thu, 29 Jan 2026 13:06:22 -0800 Subject: [PATCH] added -e parameter --- README.md | 11 +++++++---- launch-cluster.sh | 2 ++ 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2d31268..dc6e7f6 100644 --- a/README.md +++ b/README.md @@ -144,11 +144,13 @@ Don't do it every time you rebuild, because it will slow down compilation times. For periodic maintenance, I recommend using a filter: `docker builder prune --filter until=72h` -### 2026-01-26 +### 2026-01-29 + +Added `-e` / `--env` parameter to `launch-cluster.sh` to pass environment variables to the container. Added an experimental build option, optimized for DGX Spark and gpt-oss models by [Christopher Owen](https://github.com/christopherowen/spark-vllm-mxfp4-docker/blob/main/Dockerfile). -It is currently the fastest way to run GPT-OSS on DGX Spark, achieving 60 t/s on a single Spark. It's currently not working in a cluster configuration. +It is currently the fastest way to run GPT-OSS on DGX Spark, achieving 60 t/s on a single Spark. To use this build, first build the container with `--exp-mxfp4` flag. I recommend using a separate label as it is currently not recommended to use this build for models other than gpt-oss: @@ -183,7 +185,7 @@ Then, to run on a single Spark: --max-num-batched-tokens 8192" ``` -On a Dual Spark cluster (**CURRENTLY NOT WORKING**): +On a Dual Spark cluster: ```bash ./launch-cluster.sh -t vllm-node-mxfp4 exec vllm serve \ @@ -521,7 +523,7 @@ The script attempts to automatically detect: You can override the auto-detected values if needed: ```bash -./launch-cluster.sh --nodes "10.0.0.1,10.0.0.2" --eth-if enp1s0f1np1 --ib-if rocep1s0f1 +./launch-cluster.sh --nodes "10.0.0.1,10.0.0.2" --eth-if enp1s0f1np1 --ib-if rocep1s0f1 -e MY_ENV=123 ``` | Flag | Description | @@ -531,6 +533,7 @@ You can override the auto-detected values if needed: | `--name` | Container name (default: `vllm_node`). | | `--eth-if` | Ethernet interface name. | | `--ib-if` | InfiniBand interface name. | +| `-e, --env` | Environment variable to pass to container (e.g. `-e VAR=val`). Can be used multiple times. | | `--apply-mod` | Apply mods/patches from specified directory. Can be used multiple times to apply multiple mods. | | `--nccl-debug` | NCCL debug level (e.g., INFO, WARN). Defaults to INFO if flag is present but value is omitted. | | `--check-config` | Check configuration and auto-detection without launching. | diff --git a/launch-cluster.sh b/launch-cluster.sh index 0956df3..9000560 100755 --- a/launch-cluster.sh +++ b/launch-cluster.sh @@ -35,6 +35,7 @@ usage() { echo " --name Container name (Optional, default: $DEFAULT_CONTAINER_NAME)" echo " --eth-if Ethernet interface (Optional, auto-detected)" echo " --ib-if InfiniBand interface (Optional, auto-detected)" + echo " -e, --env Environment variable to pass to container (e.g. -e VAR=val)" echo " --nccl-debug NCCL debug level (Optional, one of: VERSION, WARN, INFO, TRACE). If no level is provided, defaults to INFO." echo " --apply-mod Path to directory or zip file containing run.sh to apply before launch (Can be specified multiple times)" echo " --check-config Check configuration and auto-detection without launching" @@ -52,6 +53,7 @@ while [[ "$#" -gt 0 ]]; do --name) CONTAINER_NAME="$2"; shift ;; --eth-if) ETH_IF="$2"; shift ;; --ib-if) IB_IF="$2"; shift ;; + -e|--env) DOCKER_ARGS="$DOCKER_ARGS -e $2"; shift ;; --apply-mod) MOD_PATHS+=("$2"); shift ;; --nccl-debug) if [[ -n "$2" && "$2" =~ ^(VERSION|WARN|INFO|TRACE)$ ]]; then