software-engineering/spark-vllm-docker
2
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

Formatting

Browse Source
This commit is contained in:
eugr
2025-11-26 14:02:15 -08:00
parent cf8e411ad2
commit bdf16a0a34
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -181,7 +181,7 @@ docker exec -it vllm_node bash
All environment variables (NCCL, Ray, vLLM config) set by the startup script will be loaded automatically in this new session.
## 5.\. Using cluster mode for inference
## 5\. Using cluster mode for inference
First, start follow the instructions above to start the head container on your first Spark, and node container on the second Spark.
Then, on the first Spark, run vllm like this:
@@ -198,7 +198,7 @@ docker exec -it vllm_node
And execute vllm command inside.
## 5\. Benchmarking
## 6\. Benchmarking
Follow the guidance in [VLLM Benchmark Suites](https://docs.vllm.ai/en/latest/contributing/benchmarks/) to download benchmarking dataset, and then run a benchmark with a command like this (assuming you are running on head node, otherwise specify `--host` parameter):