LLM, agent, prompt, RAG: 7 technical AI terms to follow any conversation in 2026

By the LetzAgents team · Published on 27 May 2026

For the past eighteen months, these words have come up in meetings, in the business press, with consultants. LLM, prompt, agent, token, RAG, fine-tuning, hallucination. Seven terms that determine your ability to take part in a technology decision. This glossary gives you precise definitions without developer jargon, in under fifteen minutes.

One useful clarification: this is a technical glossary, not legal. It does not cover the AI Act, GDPR, the Cloud Act, the CNPD or DPIA. For that, see our legal AI glossary (AI Act, GDPR, DPIA) published in early May. If you are truly starting from scratch, first read our four-step beginner path published on 22 May, then come back here. With Mistral now deployable on-site through AI4LUX, these seven terms are no longer Californian theory: they describe building blocks that fiduciaries, family offices and law firms on the place financière are installing in-house.

1. LLM (Large Language Model): a statistical engine that predicts the next word

An LLM is a program trained on vast quantities of text to predict, word after word, the most probable continuation of a sentence. When you type "Dear Sir, further to your letter of…" into ChatGPT, the LLM calculates what comes statistically next, from the billions of examples it saw during training (source: Vaswani et al., Google Research, 2017, the foundational Transformer paper).

The executive-to-executive analogy: an oversized spellchecker. Your Word spellchecker suggests one word, the LLM suggests a thousand in a row. Everyday consumer products: ChatGPT (OpenAI), Claude (Anthropic), Mistral Le Chat, Gemini (Google).

The nuance many AI sales reps forget: an LLM does not understand, it predicts. No notion of truth, no representation of the world, no intent. That is why an LLM can produce a perfectly formed sentence that is factually wrong, and why a human must always validate the output, especially in a regulated profession.

2. Prompt: the question (or instruction) you give the model

A prompt is the instruction you write for the LLM: a question ("Summarise this contract in five points"), an instruction ("Rewrite this email in a more formal tone"), a context. It is the equivalent of the brief you would give to an intern you are trusting with a task for the first time.

The analogy holds exactly: vague brief, vague output. "Write me a sales email" produces a generic message. "Draft a follow-up email to a Luxembourg client about an invoice unpaid for forty-five days, firm but courteous, formal address, signed by the partner" produces something usable. Three pieces of practical advice: give the context (who is speaking, to whom, for what purpose), specify the expected format, give an example if you have one.

The nuance that saves time: a good prompt does not fix a bad model. If the task exceeds what the LLM can do, no prompt will rescue the result.

3. AI agent: a program that does, not one that answers

An AI agent is a program that executes a sequence of actions to achieve an objective. A chatbot answers. An agent does. This is the most common confusion in meetings, and the nuance is not cosmetic: it determines what you buy and what it costs.

A concrete field case: a Luxembourg brokerage firm entrusts an AI agent with prospect qualification. The agent reads incoming emails, checks LinkedIn profiles, looks up the CRM, drafts a summary, schedules a follow-up. Another case: the AI morning briefing for sales and account managers, where an agent compiles each morning's market watch and sends an eight-line summary before 8 a.m. At the other end, an AI phone agent takes calls 24/7, qualifies the caller, books an appointment.

The professional nuance: the agent does not replace the human, it produces a draft. The sales rep reviews the qualification, the manager reviews the briefing, the receptionist confirms the appointment. The agent removes the mechanical time, not the responsibility.

4. Token: the unit of measure of an LLM

A token is a piece of a word, neither a letter nor a whole word. In English, one token equals roughly three-quarters of a word (source: official OpenAI tokenizer). "Accounting" is two tokens, "the" is one token.

Why this is worth understanding: because it is the billing unit. When a vendor sells API access to its model, it charges per token consumed. When a modern model is described as handling "200,000 tokens of context" (source: official Anthropic documentation, Claude 3), that means it can ingest roughly one hundred and fifty pages in one go, which changes what it is able to handle (an entire client file, an 80-page contract, a full year of accounts).

The budget nuance: more tokens consumed equals higher cost. A private AI installed on your premises shifts that calculation: cost is no longer per token, it is a pooled infrastructure cost. For intensive use, that is the heart of the question how much does private AI cost for a Luxembourg SME.

5. RAG (Retrieval-Augmented Generation): making the LLM answer using your documents

RAG is the building block that connects an LLM to your document base so it answers with your documents and not with its general knowledge (source: Lewis et al., Meta AI Research, NeurIPS 2020, the foundational RAG paper). It is probably the most useful concept in the glossary, because it is what you most often buy when someone talks to you about "enterprise AI".

The firm analogy: an assistant who, before answering, consults your binders, your client contracts, your internal procedures, then formulates an answer grounded in those documents. Without RAG, the LLM answers from memory. With RAG, it reads your documents first.

A case from the field: an accounting firm installs an internal chatbot that answers questions about internal procedures. The same mechanism powers automated document processing, with the AI reading stacks of files. Commercial nuance: RAG is not fine-tuning. RAG reads on the fly. Fine-tuning retrains the model. Two objectives, two cost profiles.

6. Fine-tuning: adapting a model to a domain, permanently

Fine-tuning is the retraining of an existing model on specific data so it absorbs a style, vocabulary or way of reasoning particular to a profession.

The practical rule: in the vast majority of SME cases, it is RAG, not fine-tuning (source: Databricks guide RAG vs Fine-tuning, 2024). If a vendor sells you fine-tuning for a modest case (FAQ chatbot, document assistant), ask why it is not RAG. Often, they are selling what they know how to do, not what serves you.

7. Hallucination: when the model invents with confidence

A hallucination is a false answer produced by the LLM with the same confidence as a true answer: invented case law, fabricated figure, wrong date, statute that does not exist. It is not an occasional bug, it is a structural feature: because the model predicts the most probable next word with no internal verification mechanism, it can produce a perfectly plausible string with no anchor in reality.

A well-known example: in Mata v. Avianca (United States District Court, Southern District of New York, June 2023), an American lawyer filed a brief citing six decisions generated by ChatGPT (source: Mata v. Avianca court docket, CourtListener). None of the six existed. In a profession where the accuracy of information engages your professional liability, that is a risk to understand.

The operational nuance: hallucination cannot be eliminated 100%. It is reduced by three combined levers: RAG (which grounds the answer in your documents), systematic human verification of everything that comes out (citation, figure, date), and software guardrails that flag answers that look "too clean". The rule sums up in one sentence: verify figures and citations before using them, always.

What now?

You have the seven words. You distinguish a chatbot from an agent, you understand why RAG is more relevant than fine-tuning in most SME cases, you can follow a conversation between an AI consultant and your IT director. To project these terms onto a concrete case, book a call.

About LetzAgents

LetzAgents designs and deploys sovereign AI agents for businesses in Luxembourg: fiduciaries, family offices, law firms and SMEs in regulated sectors. European hosting, GDPR and AI Act compliance, operational partnership with European AI models (Mistral in particular). Our team combines AI engineering with knowledge of the Luxembourg financial place.

Learn more about LetzAgents

FAQ

1. What is the concrete difference between a chatbot and an AI agent?

A chatbot answers a question with text. An agent receives an objective and executes a sequence of actions (read an email, check a database, schedule an appointment, write a summary). The chatbot answers and waits for the next question. The agent runs a process. That is what changes the scope and the cost of a deployment.

2. RAG or fine-tuning, which one to pick for an SME?

For the vast majority of SMEs, it is RAG. RAG connects the LLM to your document base and produces an answer grounded in your documents, updated in real time. Fine-tuning retrains the model, costs more, and is only justified for very large volumes and a strong proprietary style. If a vendor proposes fine-tuning for a modest case, ask why it is not RAG.

3. Can an LLM really invent case law or an official figure?

Yes, and it is a structural risk called hallucination. The model produces a plausible but false answer, with the same confidence as a true one. The cause is mechanical: the LLM predicts the most probable next word without checking truthfulness. In regulated professions (legal, fiduciary, health), the rule is to verify every citation and every figure by hand before publishing.

4. What does "200,000 tokens of context" mean in a product sheet?

It means the model can read roughly 150 pages of text in one go. The token is the unit of measure of an LLM (about three-quarters of a word in English) and the billing unit at most AI API vendors, so a direct cost variable.

5. Does this glossary replace the legal AI Act / GDPR glossary?

No, it complements it. This glossary is technical. The legal AI glossary (AI Act, GDPR, Cloud Act, NIS2, DORA, DPF) governs the legal use of AI in Luxembourg. Both vocabularies are needed to steer an AI project in a regulated profession.

6. How can you concretely train on these seven terms in Luxembourg?

The free Elements of AI Luxembourg programme, run by the Ministry of Connectivity, the Digital Learning Hub and INFPC, offers a training module open to all every year (source: Digital Learning Hub Luxembourg, official Elements of AI page). The 5th edition closed on 22 May 2026. Subsequent editions are announced on dlh.lu. To go further on use cases specific to your profession, book a call to discuss your use case.