Inference is what happens each time you use a trained AI model. You send in a prompt, image or data, the model processes it and returns a result. It contrasts with training, which is the earlier, far more expensive process of building the model.
During inference the model's learned settings are fixed and simply applied to your input. For a language model, that means generating a reply one token at a time. It runs on specialised hardware, either in a provider's cloud or on your own equipment.
For operators, inference is the ongoing cost and speed of AI. Every customer chat, product description or forecast run is an inference call. Providers typically charge for it by usage, and the model size, input length and response length all affect both price and how quickly the answer arrives.
Questions to ask a vendor include where inference runs (which country, which cloud), whether your data is retained or used for training, what happens during outages, and how performance holds up at peak times such as a Friday dinner rush or a sale launch. Some tasks can run on smaller on-device models for speed and privacy.
Understanding inference separates the one-off effort of building AI from the daily running cost, which is the figure that actually affects your margins.