#gpu

Articles tagged with #gpu

Pushing the Limits: Extreme Inference Speedup of Qwen 3.8 27B on NVIDIA B300 (100 to 10k+ tok/s)
How tensor parallelism, MTP speculative decoding, prefix caching and suffix decoding move Qwen 3.8 27B from 104 tok/s to nearly 10,000 tok/s on our stack, and where Fireworks AI went further.
Oct 7, 202622 min read4
Benchmarking Qwen 3.8 27B Across Inference Providers: Together, Fireworks, Doubleword, and g factor
Qwen 3.8 27B throughput from concurrency 1 to 64: Together, Fireworks FP8, Doubleword, vanilla vLLM, and MTP4 speculative decoding, with latency and repeat counts.
Oct 1, 202619 min read1
Distributed Training & Inference: From CPUs and GPUs to a Cluster
An illustrated guide to distributed training: CPUs vs GPUs, CUDA, data, tensor and pipeline parallelism, FSDP vs DeepSpeed ZeRO, Ray, and inference.
Sep 29, 202617 min read
AsyncGRPO: Eliminating GPU Idle Bubbles in Environment-Heavy RL Post-Training
How async GRPO decouples rollout generation, CPU gym execution, and policy updates to remove 80% GPU idle time in environment-heavy agent RL.
Sep 27, 202612 min read2