How to Build a Responsible AI Chatbot Using Open Technologies 7 Sections35 Lessons5 QuizzesLifetime
Chunking
Long books are too large and too unfocused to compare with every question as one block. The system divides them into smaller passages called chunks.
The content is split into smaller, overlapping chunks to ensure that related information remains connected, improving retrieval accuracy.
Chunking is a design balance:
- Chunks that are too small may lose context.
- Chunks that are too large may mix unrelated ideas.
- Too little overlap can split an important explanation.
- Too much overlap wastes space and may return repeated content.

