Tag: batch size

How to Choose Batch Sizes to Minimize Cost per Token in LLM Serving

How to Choose Batch Sizes to Minimize Cost per Token in LLM Serving

Learn how to choose optimal batch sizes for LLM serving to cut cost per token by up to 87%. Discover real-world results, batching types, hardware trade-offs, and proven techniques to reduce AI infrastructure costs.

Categories

Archives

Recent-posts

Multi-GPU Inference Strategies for Large Language Models: Tensor Parallelism 101

Multi-GPU Inference Strategies for Large Language Models: Tensor Parallelism 101

Mar, 4 2026

How Domain Experts Turn Spreadsheets into Applications with Vibe Coding

How Domain Experts Turn Spreadsheets into Applications with Vibe Coding

Feb, 18 2026

Emergent Capabilities in Generative AI: What We Know and What We Do Not

Emergent Capabilities in Generative AI: What We Know and What We Do Not

Aug, 25 2026

Testing and Monitoring RAG Pipelines: Synthetic Queries and Real Traffic

Testing and Monitoring RAG Pipelines: Synthetic Queries and Real Traffic

Aug, 12 2025

Bias in Large Language Models: Sources, Measurement, and Mitigation

Bias in Large Language Models: Sources, Measurement, and Mitigation

Mar, 18 2026

PCables AI Interconnects

Menu