rag-motion

AI training advice usually sounds confident but can't say where it came from, and it ignores the details a coach cares about: what equipment someone has, how experienced they are, whether they're 17 or 70. I wanted an assistant that answers only from sources a coach would trust, cites every claim, respects a client's limits, and then builds a full training program the same way.

• →A knowledge base of the Physical Activity Guidelines for Americans, 8 open-access research papers, coaching notes, and a 60-exercise catalog, stored in PostgreSQL with pgvector
• →Ingestion that splits each document by its own structure into 150–450 word passages, keeping the source, chapter, pages, and license on every passage
• →Search that turns a question into an embedding and pulls the closest exercises and passages, with SQL filters for equipment and age group read from the question
• →Answers written only from the retrieved evidence, with every claim cited and every citation checked in code. Advice from the model's general knowledge is labeled separately.
• →A tool-calling agent that interviews a client, searches exercises and sources, and submits a full program, which code checks against 18 rules before accepting it
• →LangSmith tracing on every request: what was retrieved, the prompt, the answer, the cost, and the timing

1. 1

   Read the limits: a small model call pulls equipment, age group, and experience level from the question

2. 2

   Expand gym slang: a dictionary turns shorthand like RDL and BSS into full exercise names

3. 3

   Retrieve: pgvector finds the closest exercises and passages, filtered in SQL to what the client can use

4. 4

   Answer: Gemini writes from that evidence only and cites every claim

5. 5

   Check: code verifies every citation before the answer is shown

A 55-question test set with expected, acceptable, and forbidden answers for each question, drafted with AI and reviewed by me against my exercise science training. Retrieval is scored with recall and MRR. Answers are checked in code for valid citations, honest coverage claims, and forbidden recommendations, with an AI judge used only for a few labeling rules. Every before-and-after comparison is averaged over 3 runs, because single runs proved too noisy to trust. The program agent has its own evaluation: 14 fictional clients, scored by the same code checks the agent has to pass.

citations valid

40/40

citations valid

exercise recall after adding filters

0.80 → 0.83

exercise recall after adding filters

slang questions finding the right exercise

89% → 100%

slang questions finding the right exercise

• ✓Every answer cites its sources, and code verifies each citation
• ✓Equipment and age-group limits are enforced: no limit-breaking answers across 3 runs
• ✓The program agent has passed every check on all 90 runs since the structure fix
• ✓Answers cost about $0.0004 each

• →Structured output checks the shape of an answer, not whether it's true. One model followed an instruction hidden inside a document.
• →Search can't enforce rules on its own. Limits belong in the query, not the prompt.
• →An answer can't be better than what search returns, so I measure retrieval separately from answers.
• →A research tool once invented numbers from a PDF it couldn't read. Checking against the real text caught it.

• Python
• FastAPI
• PostgreSQL + pgvector
• Voyage embeddings
• Gemini via OpenRouter
• LangSmith
• pytest
• Docker