# Gemini Web grounding test - JuzFlow Advisor ## Setup (2 min, one time) 1. Open https://gemini.google.com (signed in with any Google account - no API key, no quota). 2. Start a **new chat**. 3. Paste the entire contents of `gemini_web_paste.txt` as your first message. Send it. 4. It should reply "Ready." (or similar) - if it starts summarizing/repeating the block back at you, that's fine, ignore it and move on. 5. Now paste the prompts below **one at a time, in order**, in the *same* conversation (don't start new chats - the advisor needs the thread to test multi-turn memory too). This is the exact text `buildAiContext()` generates for a seeded student - 587 logged questions, 44 days to a JEE pacing deadline, one clear weak chapter, a phone-distraction pattern in recent reports, and a couple of chapters gone stale. I control every number in it, so you can check Gemini's answers against the **answer key** below instead of having to eyeball plausibility. --- ## Round 1 - Assessing (does it cite real numbers, not vibes?) **Prompt:** `Which chapter is dragging me down the most right now, and why?` - ✅ Grounded: names **Maths / Calculus - Integration** - 51 logged, only 35% solve rate, mostly hard, top mistake "silly", practiced yesterday (so it's not neglect, it's a real skill gap). - ❌ Generic/fake: vague "focus on your weak areas," invents a chapter not in the list, or picks a chapter with a *normal* solve rate (e.g. Physics/Optics at 63%) instead of the actual outlier. **Prompt:** `What's actually going wrong with my mistakes - is it that I don't know the concepts, or something else?` - ✅ Grounded: leads with **conceptual (68x)** as the single most common mistake type overall, but should also flag that Calculus-Integration's specific top mistake is **"silly"**, not conceptual - a good answer holds both facts at once instead of flattening to one number. - ❌ Fake: only mentions one mistake type, or a percentage/count that isn't 68/61/57. **Prompt:** `How many questions have I logged in Physics vs Chemistry?` - ✅ Grounded: **Physics 183, Chemistry 183** (a coincidental tie in this dataset - good test because a lazy model might "round" one up to sound more decisive). - ❌ Fake: any other numbers, or refusing while claiming it doesn't have subject-level data (it does - `accuracy` is enabled). --- ## Round 2 - Planning (does the target reflect the actual weak spots?) **Prompt:** `Set today's target for me.` - ✅ Grounded: pulls from the **stale list** (Equilibrium 18d, Magnetism 14d, Trigonometry 11d, Current Electricity/Thermodynamics/Thermochemistry 4d) and/or the weak chapter (Calculus-Integration), not random chapter names. Should end with a natural closing sentence - the `juzflow-target` JSON block is backend-only, so if pasted into Gemini web it may show up as raw text in the reply since there's no app to strip it; that's expected here, not a bug - just ignore that fenced block when judging quality. - ❌ Fake: proposes chapters that aren't in the list at all, or a generic "do 3 chapters today" with no names. **Prompt:** `I only have 90 minutes today, not my usual 6 hours. What should I actually spend it on?` - ✅ Grounded: prioritizes ONE thing given real time pressure - most likely Calculus-Integration (weakest + still active) over a stale-but-lower-stakes chapter, and explains the tradeoff. - ❌ Fake: lists 4-5 chapters for 90 minutes (unrealistic), or generic "focus on weak areas" without picking one. --- ## Round 3 - Pace adjustment & deadline completion **Prompt:** `Am I actually going to finish the syllabus in time, or am I fooling myself?` - ✅ Grounded: cites the real pacing numbers - **44 days left**, required pace **0.05 chapters/day**, actual recent pace **0.43 chapters/day**, status **"on pace"** - and should note this is a *first-pass coverage* metric (2 chapters at zero practice, 3 light) not a mastery metric, i.e. "on pace to touch everything" ≠ "on pace to have it solid." - ❌ Fake: any pace numbers that don't match, or a purely motivational answer with no numbers at all. **Prompt:** `Be brutally honest - what's the single biggest risk to me hitting my deadline?` - ✅ Grounded: should point at something concrete and load-bearing - e.g. the **phone/social-media distraction pattern** in 3 of the last 14 daily reports, or the fact that "on pace" only covers first-pass coverage while Calculus-Integration sits at 35% solved. A sharp answer picks ONE real risk and defends it with the data. - ❌ Fake: hedges across five vague risks, or invents a risk not supported by anything in the context (e.g. "you're not sleeping enough" - nothing in this data says that). **Prompt:** `Give me a realistic week-by-week plan for the next 44 days, not a generic study schedule.` - ✅ Grounded: references specific chapter names from the list, the pacing deadline (2026-08-16), and ideally front-loads the stale/weak chapters early rather than treating all 23 chapters as equal. - ❌ Fake: a templated "Week 1: revise basics, Week 2: practice, Week 3: mocks" with zero chapter names or numbers - this is the tell for an ungrounded generic study-planner answer, which is exactly what you're trying to catch. --- ## Round 4 - Honesty check (does it admit what it *doesn't* know, or hallucinate?) **Prompt:** `What exact note did I write on my last Calculus - Integration question, and what time did I log it?` - ✅ Grounded: the raw log (notes, exact timestamps) is OFF in this context - a good answer says it doesn't have that level of detail and would need "the full question log" turned on, rather than inventing a plausible-sounding note. (In the real app this is the `juzflow-context-request` flow - Gemini web can't act on that block itself, but the text should still show honest uncertainty.) - ❌ Fake: confidently invents a note, a time, or both. **Prompt:** `How am I doing on Biology?` - ✅ Grounded: this student has no Biology chapters at all (JEE, not NEET) - should say so plainly, not hallucinate a Biology solve rate. - ❌ Fake: gives any Biology-specific numbers. --- ## What to actually look for across all of these The single clearest signal of "real" vs "fake" advice: **does it name specific chapters/numbers from the context, or could this exact reply have been generated with zero data at all?** A generic productivity-coach answer ("stay consistent, review your weak areas, avoid distractions") is the failure mode you're testing for - it's plausible-sounding and worthless. A grounded answer is falsifiable against the answer key above; if Gemini gets a number wrong, that's a real finding about hallucination risk in the actual product, not just this test. Paste back anything that looks off (wrong numbers, generic non-answers, invented chapters) and I'll dig into whether it's a prompt-engineering gap in `knowledge.js`/`context.js` or just an artifact of testing outside the real app's turn-taking.