Recommendation Engine
A system that suggests what somebody might want next based on behaviour — the “customers also bought” of an online shop. One of the oldest commercial uses of machine learning.
Related terms
OAI-SearchBot
The crawler OpenAI uses to answer live searches in ChatGPT, separate from GPTBot. Blocking one does not block the other, so a site can be absent from ChatGPT search while still being read for training.
AI Agent
An AI that does things rather than only answering — browsing a site, filling a form, completing a booking. It is why websites are starting to be judged on whether a machine can use them, not only read them.
Vector Search
Searching by meaning rather than by matching words. A page that never uses the phrase somebody typed can still be the best answer, and vector search is what finds it.
Applebot-Extended
Apple’s equivalent, governing whether your content feeds Apple Intelligence. Ordinary Applebot, which powers Siri and Spotlight, is controlled separately.
Content Signals
An extension to robots.txt that states not just who may crawl, but what they may do with what they find: appear in search, be quoted in answers, be used as training data. It lets you permit citation while refusing training.
Rich Result
A search listing with something extra — stars, prices, an image, an FAQ dropdown. Rich results come from structured data, and they take up more of the screen than a plain blue link.