Updated README
This commit is contained in:
35
README.md
35
README.md
@@ -53,9 +53,44 @@ docker run -it --rm \
|
|||||||
--ipc=host \
|
--ipc=host \
|
||||||
--privileged \
|
--privileged \
|
||||||
--name vllm_node \
|
--name vllm_node \
|
||||||
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||||
vllm-node bash
|
vllm-node bash
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Or if you want to start the cluster node (head or regular), you can launch with the run-cluster.sh script (see details below):
|
||||||
|
|
||||||
|
**On head node:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --privileged --gpus all -it --rm \
|
||||||
|
--ipc=host --shm-size 10.24g \
|
||||||
|
--network host \
|
||||||
|
--name vllm_node \
|
||||||
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||||
|
vllm-node ./run-cluster-node.sh \
|
||||||
|
--role head
|
||||||
|
--host-ip 192.168.177.11
|
||||||
|
--eth-if enp1s0f1np1
|
||||||
|
--ib-if rocep1s0f1
|
||||||
|
```
|
||||||
|
|
||||||
|
**On worker node**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --privileged --gpus all -it --rm \
|
||||||
|
--ipc=host --shm-size 10.24g \
|
||||||
|
--network host \
|
||||||
|
--name vllm_node \
|
||||||
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||||
|
vllm-node ./run-cluster-node.sh \
|
||||||
|
--role node
|
||||||
|
--host-ip 192.168.177.12
|
||||||
|
--eth-if enp1s0f1np1
|
||||||
|
--ib-if rocep1s0f1
|
||||||
|
--head-ip 192.168.177.11
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
**Flags Explained:**
|
**Flags Explained:**
|
||||||
|
|
||||||
* `--net=host`: **Required.** Ray needs full control over network ports; port mapping is insufficient for multi-node clusters.
|
* `--net=host`: **Required.** Ray needs full control over network ports; port mapping is insufficient for multi-node clusters.
|
||||||
|
|||||||
Reference in New Issue
Block a user