A thoughtful AI collaborator for writing, analysis, coding, and long-form work.
Full evaluationGlossary · Updated Sep 2, 2026
Inference
Running a trained model to produce outputs, as opposed to training it; the compute that powers every response.
Definition
Inference is the act of generating an answer, image, or transcript from a model. Its cost and speed depend on model size, hardware, and optimisations such as quantisation and caching. Providers price inference per token or per generation, and self-hosters pay for GPU time. Latency and throughput are inference properties.
Why it matters when choosing a tool
Inference economics decide whether an AI feature is affordable at your volume and whether it responds fast enough for the user experience.
Where you will meet it
AI Coding & Development, AI Agent & Chatbot Builders
Related terms
Tools where this matters
Reviewed products in the related categories
An AI-native code editor for repository-aware agents, edits, review, and automation.
Full evaluation
Anthropic's agentic coding tool that works in the terminal, IDE, desktop app, and browser to plan and execute multi-step changes.
Full evaluation
Unified TypeScript library for streaming, structured outputs, and agent loops across AI providers
Full evaluation
Prototyping environment and developer API platform for Google's Gemini models
Full evaluation
A source-available workflow automation platform with native AI agent nodes, self-hostable or cloud.
Full evaluation