Studiehjälpen
A RAG assistant that would rather say "I don't know" than guess about your benefits.
The problem
Why it exists
I moved to Sweden in 2019 and later combined YH studies with parental leave and freelance income. Working out what that meant took hours of cross-referencing CSN, Försäkringskassan and Skatteverket pages that all affect each other but never link the answers together.
An ordinary chatbot is the wrong tool for this. It answers confidently from half-remembered training data, and wrong answers about benefits have real consequences. So the goal is a transparent assistant, not a clever one. Every answer shows where it came from, and the system refuses when it doesn't have a good enough source.
How it works
The planned query loop
Phase 1 of 6 is done: schema, embedding model and vector search, checked against a live database. This loop is what the next phases build.
Step 1 of 6 · Client
Question
Someone asks a question, maybe a follow-up to the last one.
Decisions
What I chose, and what I didn't
Retrieval instead of live browsing
A vector search takes milliseconds where fetching pages takes seconds. Five targeted chunks cost far fewer tokens than whole pages. And logging which chunk produced which answer means the system can be evaluated at all.
Rejected
A model plus web search. Fine for one-off questions, but it can't be scoped, cited precisely, or measured against a test set.
Hybrid search, not vectors alone
Embeddings capture meaning but blur exact tokens, and this corpus is full of them: SGI, fribelopp, form numbers, kronor amounts. Keyword search runs alongside, and Postgres already has Swedish full-text search built in.
Rejected
Pure vector search. Simpler, and measurably worse on exactly the questions this corpus gets.
pgvector instead of a vector database
One database holds the relational data, the vectors and the full-text index. No second service to run, back up or keep in sync.
Rejected
Pinecone or Qdrant. They're better at large scale, and unnecessary overhead here. I'd switch at millions of vectors, or when vector search starts competing with normal queries.
No RAG framework
The core of a RAG pipeline is a few hundred lines: chunking, embedding, search, prompt assembly. If I write each step myself, then when retrieval gets worse I'm debugging my own chunker, not a framework's internals.
Rejected
LangChain or LlamaIndex. They get you to a demo faster. They also hide the parts I wanted to learn.
By the numbers
What it adds up to
- questions in the evaluation set
- 25
- database for rows, vectors and full-text search
- 1
- phases done
- 1 / 6
- RAG frameworks
- 0
What I learned
Most RAG failures are retrieval failures that look like generation failures. So the evaluation scores two things separately: did the right chunk come back, and was the answer faithful to it. It runs again after every change to chunking, embedding or retrieval.