LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training
Ranking
Overall
77
Content
95
Popularity
34
Observed public metrics from 1 member.
Merged summary
TL;DR - LazyTrain optimizes checkpointing, activation placement, recomputation, and data transfers for LLM training on memory-constrained hardware. It improves throughput and enables larger batches by reducing exposed communication and optimizer-state overhead.
- Uses mixed-integer scheduling to coordinate CPU, GPU, and NVMe resources.
- Combines 8-bit optimizer states with fast gradient clipping in a Hybrid 8-bit operator.
- Improves sustained TFLOPS by approximately 1.24Ă— over matched H800 baselines.
- Trains Qwen3.6-27B at 1,361 tokens/s with batch size 72 and 68.84 GB peak GPU memory.
Sources (1)
LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - LazyTrain optimizes checkpointing, activation placement, recomputation, and data transfers for LLM training on memory-constrained hardware. It improves throughput and enables larger batches by reducing exposed communication and optimizer-state overhead.
- Uses mixed-integer scheduling to coordinate CPU, GPU, and NVMe resources.
- Combines 8-bit optimizer states with fast gradient clipping in a Hybrid 8-bit operator.
- Improves sustained TFLOPS by approximately 1.24Ă— over matched H800 baselines.
- Trains Qwen3.6-27B at 1,361 tokens/s with batch size 72 and 68.84 GB peak GPU memory.