software-engineering/spark-vllm-docker
2
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

Updated Nemotron to support dual sparks

Browse Source
This commit is contained in:
Eugene Rakhmatulin
2026-03-12 13:30:15 -07:00
parent 6a323cc6f5
commit 8fec9bed06
2 changed files with 15 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
recipes/nemotron-3-super-nvfp4.yaml
View File

@@ -7,8 +7,7 @@ description: vLLM serving Nemotron-3-Super-120B using Marlin kernels
model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4
container: vllm-node
cluster_only: false
# This model can only run on single node (solo)
solo_only: true
solo_only: false
mods:
- mods/nemotron-super
@@ -17,7 +16,7 @@ container: vllm-node
defaults:
port: 8000
host: 0.0.0.0
tensor_parallel: 1
tensor_parallel: 2
gpu_memory_utilization: 0.7
max_model_len: 262144
max_num_seqs: 10
@@ -41,4 +40,6 @@ command: |
--load-format fastsafetensors \
--tool-call-parser qwen3_coder \
--reasoning-parser-plugin super_v3_reasoning_parser.py \
--reasoning-parser super_v3
--reasoning-parser super_v3 \
--tensor-parallel-size {tensor_parallel} \
--distributed-executor-backend ray