Commit Graph

5 Commits

Author SHA1 Message Date
sonusflow
3baca14eb1 Move recipe to 4x-spark-cluster/ and add UMA memory optimizations
- Move qwen3.5-397b-int4-autoround.yaml to recipes/4x-spark-cluster/
  per maintainer request (multi-node recipes in separate directory)
- Add PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to recipe env
- Optimize Ray for GB10 UMA (128GB shared CPU/GPU memory):
  - Disable Ray dashboard (saves ~1.2 GiB per node)
  - Limit Ray object store to 1 GiB (default 30% of RAM = 33 GiB)
  - Disable pre-started idle workers (saves ~8 GiB on head node)
  - Set --num-cpus 2 and --disable-usage-stats on all nodes
- Net effect: ~40+ GiB freed across 4-node cluster for model/KV cache
2026-03-11 07:29:45 +00:00
Eugene Rakhmatulin
1025243316 Added launch_cluster script to simplify launching cluster on nodes. 2025-12-18 13:10:57 -08:00
eugr
4c976375c5 Added missing dependencies; added dashboard support for Ray clusters 2025-11-24 21:13:06 -08:00
eugr
f5141974ae Fixed cluster script and small fix for Dockerfilewq 2025-11-24 15:45:04 -08:00
eugr
3ecca4d2b7 Updated Dockerfile to include 2 levels of cache busters, added the cluster script and README. 2025-11-24 15:21:08 -08:00