DeepSeek V4 Flash on a Single AMD MI300X
- ID
- 10786
- Status
- summarized
- Published
- 04 Aug 2026, 6:00 PM
- Fetched
- 06 Aug 2026, 6:35 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://github.com/ryanzhou/deepseek-v4-flash-mi300x
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.0
- Created
- 07 Aug 2026, 1:33 PM
- Tags
- Audience
- developersai_ml_learnersai_agent_users
What happened
A GitHub repo documents running DeepSeek-V4-Flash-0731 (304B parameters) on a single AMD MI300X GPU in production, achieving 168.6 tok/s single-stream decode and ~8K tok/s prefill with vLLM ROCm nightly. The entire model fits in the MI300X's 192 GB HBM without quantization or offload, but required custom patches for FP8 format differences, MoE routing, speculative verification, and kernel tuning.
Why it matters
If you're cost-sensitive about serving large open models, the MI300X's 192 GB HBM at roughly half the list price of an H100 makes single-card deployment of 300B+ models viable—worth evaluating before defaulting to NVIDIA. The repo's patches for AMD's non-standard FP8 (fnuz E4M3 vs OCP) are a concrete warning that porting NVIDIA-targeted vLLM recipes to MI300X is not drop-in.
Discussion angle
Is the MI300X now a practical alternative to H100 for self-hosting large open models in Southeast Asia, where AMD GPU availability and ROCm support maturity may differ from US hyperscaler environments?