KNOWLEDGE / 11
AI, LLM, RAG and agents
LLM solution design, retrieval architectures, agents, and their practical testing.
Questions and practice
Open a question to see the answer, examples, and exercises.
What does RAG add to an LLM?Junior
Answer
RAG retrieves relevant external material and adds it to the generation context; it does not guarantee a correct or complete answer.
Examples
- An assistant may answer from a current specification and still interpret it incorrectly.
Practice exercises
- Compare answers with correct, missing, and conflicting retrieval context.
What are tokens and a context window in an LLM system?Junior
Answer
A model processes text as tokens, while the context window limits the combined size of instructions, history, retrieved data, and output. More context does not guarantee a better answer: irrelevant text increases cost, latency, and the chance of losing important signals. Select and structure context deliberately.
Examples
- A long log dump can obscure acceptance criteria even when everything technically fits in the window.
Practice exercises
- Allocate a prompt budget among instructions, task data, retrieved evidence, and output, explaining priorities.
What do embeddings do in a retrieval system?Junior
Answer
An embedding maps text to a vector representation where proximity roughly reflects semantic similarity. Retrieval finds candidates, but similarity does not guarantee factual relevance, freshness, or access permission. Metadata filters, lexical search, and reranking often complement vector search.
Examples
- A password-reset query may retrieve a semantically similar but obsolete guide; a version filter removes it.
Practice exercises
- Create five queries with expected documents and identify failure cases for similarity-only retrieval.
When is an agent unnecessary?Middle
Answer
When the action sequence is known and a normal workflow with clear rules and validators can perform it.
Examples
- A fixed parse, validate, then publish pipeline does not need autonomous next-step selection.
Practice exercises
- Convert one deterministic agent scenario into a workflow and compare the risks.
How do chunking and reranking affect RAG quality?Middle
Answer
A chunk should preserve a complete idea and enough local context without mixing too many topics. Retrieval selects candidates quickly, while a reranker orders them more precisely for the query. Measure retrieval quality separately because a generator cannot reliably repair missing evidence.
Examples
- A section-boundary chunk keeps a rule with its exceptions, while a fixed-size split may separate them.
Practice exercises
- Compare two chunking strategies on ten queries using recall@k and top-ranked context quality.
How does structured output differ from executing a tool call?Middle
Answer
Structured output constrains generated data to a schema but does not prove that values are true. A tool call is the model’s proposal to invoke a capability with arguments; the application controls actual execution. Before a side effect, validate, authorise, ensure idempotency, and require human approval for risky actions.
Examples
- Valid JSON containing deleteUserId still cannot authorise deletion without permission checks and confirmation.
Practice exercises
- Design validation and approval for a tool that changes production configuration.
Why do embeddings not replace permissions?Senior
Answer
Vector similarity measures relevance; access must be checked separately before context reaches the model.
Examples
- A semantically similar document belonging to another customer does not become an authorised source.
Practice exercises
- Design retrieval with tenant filtering and test a cross-tenant access attempt.
How should an autonomous agent’s authority be constrained?Senior
Answer
Give an agent a minimal tool set, scoped credentials, budgets, and explicit stop conditions. Classify decisions by reversibility and impact: read-only work may be automated more broadly, while destructive or external actions require approval. An audit trail should connect intent, context, tool call, result, and human decision.
Examples
- An agent may prepare a patch and tests, while merge and production deployment remain separate human gates.
Practice exercises
- Create an authority matrix for a coding agent: allowed, approval-required, and forbidden actions.
Why is prompt injection a system problem rather than only a prompt problem?Senior
Answer
A model has no reliable security boundary between instructions and untrusted content. Protection must surround the system: least-privilege tools, data provenance, isolation, allowlists, output validation, approval, and monitoring. A sentence in a system prompt cannot guarantee that a retrieved document will not influence a decision.
Examples
- A document saying “send secrets” has no access to a tool capable of reading secrets, regardless of the model’s response.
Practice exercises
- Threat-model a RAG agent: untrusted sources, assets, tool boundaries, abuse cases, and mitigations.