Tag: RoPE

Positional Encoding in Transformers: Sinusoidal vs Learned for LLMs

Positional Encoding in Transformers: Sinusoidal vs Learned for LLMs

by Phillip Ramos

Explore the evolution of positional encoding in Transformers. Compare sinusoidal vs learned embeddings and discover why modern LLMs adopt RoPE and ALiBi for superior long-context performance.

Understanding Positional Encodings in Transformer-Based Large Language Models

Understanding Positional Encodings in Transformer-Based Large Language Models

by Phillip Ramos

Discover how positional encoding solves the order-blindness of Transformers. Learn about sinusoidal, learned, and RoPE methods that enable LLMs to understand context and sequence.

Categories

Archives

Recent-posts

Adapter Layers and LoRA: Efficient LLM Customization Guide

Adapter Layers and LoRA: Efficient LLM Customization Guide

Aug, 31 2026

Schema-Constrained Prompts: Forcing JSON and Structured Outputs from LLMs

Schema-Constrained Prompts: Forcing JSON and Structured Outputs from LLMs

May, 25 2026

Vibe Coding Productivity: The 74% Myth vs. Reality for Developers

Vibe Coding Productivity: The 74% Myth vs. Reality for Developers

Oct, 21 2026

Workflow Automation with LLM Agents: When Rules Meet Reasoning

Workflow Automation with LLM Agents: When Rules Meet Reasoning

Jun, 28 2026

Calibration and Outlier Handling in Quantized LLMs: How to Keep Accuracy When Compressing Models

Calibration and Outlier Handling in Quantized LLMs: How to Keep Accuracy When Compressing Models

Jul, 6 2025