Tag: inference optimization

Scaling Open-Source LLMs: Hardware, Serving Stacks, and Playbooks for 2026

Scaling Open-Source LLMs: Hardware, Serving Stacks, and Playbooks for 2026

Learn how to scale open-source LLMs in 2026. Explore hardware needs for gpt-oss-120b, the role of SLMs, and professional serving stacks using vLLM and SGLang.

How to Choose Batch Sizes to Minimize Cost per Token in LLM Serving

How to Choose Batch Sizes to Minimize Cost per Token in LLM Serving

Learn how to choose optimal batch sizes for LLM serving to cut cost per token by up to 87%. Discover real-world results, batching types, hardware trade-offs, and proven techniques to reduce AI infrastructure costs.

Categories

Archives

Recent-posts

Calibration and Outlier Handling in Quantized LLMs: How to Keep Accuracy When Compressing Models

Calibration and Outlier Handling in Quantized LLMs: How to Keep Accuracy When Compressing Models

Jul, 6 2025

How Generative AI Transforms Sales Battlecards, Call Summaries, and Objection Handling

How Generative AI Transforms Sales Battlecards, Call Summaries, and Objection Handling

Jul, 21 2026

Code Generation with LLMs: Boosting Productivity and Managing the Limits

Code Generation with LLMs: Boosting Productivity and Managing the Limits

Apr, 21 2026

Speculative Decoding Guide: Speed Up LLM Inference with Draft and Verifier Models

Speculative Decoding Guide: Speed Up LLM Inference with Draft and Verifier Models

Apr, 25 2026

When to Use Reasoning Models: Managing Think Token Costs in LLMs

When to Use Reasoning Models: Managing Think Token Costs in LLMs

Aug, 10 2026

PCables AI Interconnects

Menu