Search Ctrl+K

Change language Switch ThemeSign In

[](https://www.bestblogs.dev/en/explore/brief "Daily Brief")[](https://www.bestblogs.dev/en/explore/newsletter "Weekly Picks")[](https://www.bestblogs.dev/en/explore/topics "Topics")[](https://www.bestblogs.dev/en/worldcup "World Cup Special")

[](https://www.bestblogs.dev/en/settings "Settings")[](https://www.bestblogs.dev/en/docs "Help Center")

Narrow Mode

78

本地部署 GLM-5.2 的门槛太高了，根本玩不起！

The author found that local deployment of GLM-5.2 requires massive VRAM (436G for the quantized version, 704G for the FP8 version), with slow inference speed and poor concurrency on H20, making the hardware barrier extremely high. Follow·

Yesterday·426 words (about 2 min)

·View Source →

AI Summary & Key Points

Summary

This article is an author's hands-on record of deploying GLM-5.2 on a compute server. Two versions were tested: unsloth's UD-Q4_K_XL quantized version (436G gguf file, 4 H20 GPUs with 560G VRAM total) and Zhipu's official FP8 quantized version (704G weights, 8 H20 GPUs with 1.1T VRAM total). The quantized version achieved only 20-30 tokens/second and could not support concurrency; the FP8 version could not enable 1M context with 1.1T VRAM, supporting only 2.5 concurrent users with 256K context, and multiple Claude Code connections would cause stuttering. The author notes that its cache architecture is based on DeepSeek 3.2, resulting in significantly lower VRAM utilization efficiency compared to DeepSeek 4 and Qwen 3.5/3.6, concluding that without hardware at the H200/B300 level, it's not worth attempting.

Main Points

* 1. The VRAM requirement for local deployment of GLM-5.2 is extremely high, far exceeding what conventional enterprises can afford.

The UD-Q4_K_XL quantized version requires 4 H20 GPUs (560G VRAM), and the FP8 version requires 8 H20 GPUs (1.1T VRAM), making it unfeasible for most teams.

* 2. Inference speed on H20 is unsatisfactory, with weak concurrency.

The quantized version achieves only 20-30 tokens/sec, and the FP8 version 50 tokens/sec, but with 256K context it only supports 2.5 concurrent users, causing noticeable stuttering with multiple calls.

* 3. The cache architecture is based on DeepSeek 3.2, resulting in lower VRAM efficiency than the latest competitors.

The author inferred from vLLM logs that VRAM utilization efficiency is much worse than DeepSeek 4 and Qwen 3.5/3.6, which is the underlying reason for the high deployment barrier.

Sign in to highlight text and take notes as you read. Sign in now

智谱最近发布的 GLM-5.2 口碑很好，于是想在算力服务器上试一下，结果发现，门槛太高了，根本玩不起！

前后尝试了两个版本，分别是：

1. unsloth 的UD-Q4_K_XL 量化版本) <https://huggingface.co/unsloth/GLM-5.2-GGUF/tree/main/UD-Q4_K_XL>

2. 智谱官方的FP8 量化版本) <https://huggingface.co/zai-org/GLM-5.2-FP8>

先说一下 UD-Q4_K_XL 量化版本，下载下来的 gguf 文件共 436G ，4 张 H20 （共 560G 显存），编译最新的 llama.cpp 来运行，结果发现只有 20 ～ 30tokens/秒，更别说并发访问了，基本没法用；

然后是 FP8 量化版本，权重文件共 704G ，8 张 H20 （共 1.1T 显存），下载最新的 vllm 来运行，结果如下：

1. 在上下文类型也是 fp8 的情况下，8 张 H20 ，1.1TB 显存，居然无法开启 1m 上下文；

2. 将上下文长度设置为 384k 之后，vllm 启动提示 1.3 个并发，将上下文长度设置为 256k ，vllm 启动提示 2.5 个并发；

3. 输出大概有 50tokens/秒，吐字速度算还可以；

4. 3 个 claude code 同时连接使用，就能感觉到明显卡顿；

从 vllm 的启动日志看，glm-5.2 的缓存架构还是基于 deepseek 3.2 的，显存利用效率比 deepseek4 甚至 qwen3.5/3.6 差很多！

以上只是对本地部署测试 glm-5.2 的初步印象，劝大家如果没有 h200/b300 级别的装备的话，还是算了吧！😂

Key Quotes

> I advise everyone that if you don't have h200/b300 level equipment, just forget it! 😂

Tags

LLM

Model Deployment & Inference

AI Hardware & Chips

Performance Evaluation

Related Articles

* GLM-5.2 Released and Open-Sourced: Focused on Coding and Long-Horizon Tasks * LLM Training You Didn't Know: Principles, Paths, and New Practices - Tw93 * From OpenClaw to FastClaw: How to Design a Great Multi-Agent Architecture * A Comprehensive Guide to Microsoft Build 2026: The 'Agent-First' Era Arrives with Seven In-House Models * Two Years of Real Trading +135%: The Multi-Agent Investment Research Framework I Built with Claude Code * Skill: Tips on How Vibe Coding Improves Code Quality and Efficiency * AI R&D Automation: Wiki Knowledge Base + Skill Pack * 69. Interview with Yuandong Tian: The Real Problems of LLMs, Shifts, the AI Flood, and the Path Not Taken * Skill: Tips on How Vibe Coding Improves Code Quality and Efficiency * vibe-coding-template: After Losing a Codex Conversation File, I Organized a Set of Agent Long-Term Collaboration Templates to Quickly Embed AGENTS.md, Task Prompts, Code Review, Knowledge Explanations, and Web-Search Workflows into Projects

Make your daily reading actually fit you.A daily brief built from the sources you follow. Get started free HomeDiscoverWorld CupSettings