主题柱 · 搜索意图:informational

本地跑模型:显存 · 量化 · 何时划算

Ollama/vLLM 边界、显存与量化取舍、本地 vs API 费用。部署计算器用工具页,开源模型见开源频道。

先读结论

本地部署的账不是「买张卡就完」:电费、运维、量化质量与并发都要进 TCO。

本页给决策框架;具体显存估算用本地部署计算器,权重下载走开源模型频道。

本页是方法论枢纽,不堆实时报价。数字以链接中的数据页为准;我们不卖货、不担保到账。

相关指南(36)

入门

聊天、写代码、批量、本地:同一需求怎么选四条路

先锁定你要完成的作业,再选官方订阅、卡网、官方 API 或中转。路径跟风比最低价更容易亏。

中转

New API / One API 类网关选型

开源网关常见能力对比、部署注意、多模型路由与用户体系。

算力

本地算力成本 vs 官方/中转 API:怎么算账

把显卡折旧、电费、带宽与 API Token 费用放到同一张账本上,判断何时自建、何时买 API。

硬件

GPU 显存对照:什么卡能跑什么模型

按显存 GB 与模型参数档(7B/32B/70B/MoE)给出本地部署启发式对照表,含量化与上下文提示。

硬件

本地大模型部署入门:Ollama / vLLM / llama.cpp

从选框架、下模型、占显存到对外 OPC 服务,理清本地 LLM 最小可行路径与常见坑。

硬件

模型量化入门:Q4/Q5/FP8 与显存质量权衡

用白话解释量化为何省显存、常见位数怎么选、对编码/推理质量的影响与回测方法。

机房

云 GPU / 显卡租赁入门:按时计费与避坑

租 A100/H100/4090 前要看的计价项、镜像、数据出境、闲置浪费与和 API 成本的对比方法。

机房

机房地域与延迟:部署区域怎么选

用户在哪、模型在哪、数据能不能出境——用延迟、合规与成本三维选 region。

编程

2026 AI Coding 工具栈:IDE / 模型 / API 怎么搭

编辑器插件、云模型、本地模型和中转 API 的组合方式,以及密钥与成本控制要点。

算力

OpenAI 融 300 亿,估值 1.4 万亿美元

The new round is anticipated to be the company's last before its delayed 2027 public debut.

官方API

Amazon Bedrock 印度 Claude 推理支持本地调用

Anthropic's Claude Opus 5, Claude Sonnet 5, and Claude Haiku 4.5 are now available in India through Amazon Bedrock geographic cross-Region inference. You can access these models wh

算力

AMD 81亿美元收购World Labs,AI算力与本地部署利好

The deal, which is expected to close by year's end, is worth $8.2 billion.

算力

OpenAI 澳洲服务器黑客:agent 访问系统信息与源代码

Without a "full set of safeguards," agent accessed "system information and source code."

编程

ChatGPT周活跃1.2亿,Codex编码模型中转与本地部署算力

\n OpenAI's annualized revenue rate is nearing $70 billion, up about 70 percent since the start of Q3. Enterprise sales, the Codex coding assistant, and an aggressive price

算力

Peak XV 种子投资上限提升至 500 万美元,18 家 AI 初创公司加速

Thirteen of the 18 startups in Peak XV’s latest Surge cohort are targeting global markets, while more than half are based in India.

入门

Photo Scrubber:本地人脸模糊+元数据移除

Tool: Photo Scrubber — local face blur & metadata removal \n I took a photograph of some protesters, then thought about how I don't like sharing photographs of strangers

模型

OpenAI 暂停前沿模型:编码中转算力与本地部署需调整

US Government websites among "dozens of third parties" OpenAI has recently notified.

编程

SageMaker部署Qwen3-TTS vLLM-Omni Part1

Deploy a text-to-speech model on Amazon SageMaker AI with the AWS vLLM-Omni Deep Learning Container and stream generated speech over a persistent bidirectional connection. This Par

模型

vLLM-Omni 在 SageMaker AI 生成图像视频

Deploy two generative media models from one AWS vLLM-Omni Deep Learning Container on Amazon SageMaker AI. Generate an image with FLUX.2-klein through real-time inference, then anim

算力

算力硬件:OpenAI Lenfest获5M美元软件支持

OpenAI is expanding the Lenfest AI Collaborative and Fellowship Program with $5 million in funding and up to $5 million in software credits and engineering support.

算力

S3十年未降价,GrokCode算力与编码模型中转倍率

My comment on S3 Is the Future, S3 Is the Past — Hacker News. One thing I find notable about S3 today is that, while it used to drop in price reasonably often, there hasn't bee

入门

Tesla Optimus工人抗拒培训:编码模型本地部署难题

Despite challenges, Tesla aims for 1,000 Optimus robots per week by end of 2026.

算力

AWS EKS MoE RLHF:EFA与DeepEP提升40%吞吐量

Learn how to scale Mixture-of-Experts (MoE) reinforcement learning on Amazon EKS using Elastic Fabric Adapter (EFA) and DeepEP. This post presents an architecture that combines Ama

模型

Qwen3-TTS-12Hz在SageMaker实时部署: 语音克隆

Deploy the publicly available Qwen3-TTS-12Hz-1.7B-Base text-to-speech model from Amazon SageMaker JumpStart to a fully managed, real-time endpoint, and clone a voice from a short r

模型

AWS WhisperX SageMaker:语音转录部署

The AWS WhisperX Deep Learning Container packages Whisper, wav2vec2 forced alignment, and speaker diarization into a GPU-ready image. Learn how to deploy it to Amazon SageMaker AI

算力

Google 轨道数据中心测试:4 个 TPU,运行 15 分钟

Google's experimental orbital data center will have four TPUs and only run for 15 minutes at a time.

算力

GrokCode 中转: Foundries vs Navigators 算力

Guest Post: In science, thinking has gotten cheap but doing has not. This asymmetry is reshaping how research companies operate, largely inconspicuously.

算力

SynthID Bio:AI蛋白水印保功能

Proof of concept for watermarking AI-generated proteins while preserving biological function.

算力

NVIDIA CoreWeave代理AI生产:Vera Rubin算力与中转

Building on nearly a decade of co-engineering, CoreWeave has built NVIDIA compute, networking and software into a cloud purpose-built for AI that’s still returning on investment ac

算力

Cerebras Andrew Feldman:AI算力需求与硬件选择

At TechCrunch Disrupt 2026, Cerebras Systems CEO and co-founder Andrew Feldman will explore the growing demand for compute, energy, and infrastructure, how Cerebras is approaching

硬件

微软机器人离线推理

Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, b

算力

DeepMind私有AI服务器侧内存:GrokCode算力中转与本地部署

Introducing private, server-side memory to Private AI Compute for personal AI.

模型

Gemini 3.8 TTS 发布,本地部署与算力参考

Gemini 3.8 TTS 发布,本地部署与算力参考

算力

Amazon Bedrock AgentCore:多代理音乐制作管线

Amazon Bedrock AgentCore Runtime Instances gives multi-agent workflows AWS managed EC2 infrastructure with GPUs, persistent volumes, and multi-day sessions. In this post, we deploy

算力

NVIDIA DSX Ready:AI 工厂电源冷却认证发布

Every AI factory needs power and cooling that fit its computing architecture. As AI infrastructure expands, power, cooling, water, site and grid constraints are shaping what builde

合规

物理AI安全:从研究到大规模部署的每层防护

Physical AI is moving rapidly from research to large-scale deployment. By 2035, ABI Research projects an installed base of 49 million level 3-5 autonomous vehicles (AVs), while Omd

← 全部指南

常见问题

Ollama 能上生产吗?
适合原型与个人;高并发生产更常见 vLLM/TGI 等推理栈,并要监控与回滚。
量化会不会毁代码质量?
激进量化可能损伤推理与代码;应用侧应用业务评测集验证,而不是只看跑分。

其他主题柱