We stress-tested the LairsFlow Advisor once before, on Vivran v1, to check the one claim the whole feature rests on: that it reasons over a student's actual logged data instead of dispensing generic study tips. Vivran v2 adds a lot of new surface for that claim to fail on, so this is the rerun. Same discipline as last time, one honest new twist, and the full transcript at the end.
What changed in v2, and why the test had to change
Two v2 features reshaped the test.
The first is the Context Builder ("bring your own AI"). The Advisor runs on the student's own free Gemini key, which has a daily quota and can run out. v2 lets the student export the exact same grounded context the app would have sent to Gemini as one portable prompt, paste it into any other AI, and paste any resulting action back into the app. That portable prompt is now the thing worth testing, because it is what a real student uses on a quota-dry day.
The second is that v2 collects far more: self-study hours from the new timer modes, mock-test scores, lecture hours, and the Advisor Library (custom knowledge books the student writes so the Advisor knows things the app can't compute, like exam weightage). Every one of those is a new way for the model to either ground an answer or bluff one.
The method: real prompt, outside AI, zero quota
Same core trick as v1. Instead of burning API calls, we used the app's own code, unmodified, to build the real prompt for a fabricated student, then pasted that into an ordinary web AI as the first message and worked through a fixed list of questions. No key, no quota, and it is the real prompt-construction logic doing the work, not a hand-written approximation.
The v2 refinement: the prompt we pasted is the genuine output of the Context Builder's buildPortablePrompt() (preamble + buildAiContext() + the portable action protocol), generated by loading the actual app source and running it against the mock data. We deliberately did not load the mock student into a live signed-in app, because restoring a large dataset and letting it sync would spend Firebase free-tier quota for no reason. The whole point of the Context Builder is that it works with no backend round-trip, so the test doesn't need one either.
This measures grounding and reasoning quality, not byte-for-byte production wording: a pasted block becomes a normal conversational turn rather than arriving through the API's systemInstruction channel, so the underlying model behaviour can differ slightly. That gap is fine. The thing we care about is simple: is it reading the data, or just sounding plausible.
The fabricated student
Entirely fictional, seeded deterministically so every number is fixed and checkable in advance. A JEE aspirant targeting 2027, with:
- 422 logged questions, 68% overall solve rate. Physics 74%, Chemistry 70%, Maths 57%.
- One genuinely bad chapter: Maths / Integral Calculus, 34% solve rate over 53 questions, mostly Hard, top mistake conceptual, practiced just yesterday (so it's a live skill gap, not neglect).
- Several stale chapters (Ionic Equilibrium 18 days quiet, Magnetism 14, Trigonometry 11) and two never touched at all (Electrochemistry, Probability).
- 26.3 hours of self-study (Physics heaviest, weak Maths lightest), 22.5 hours of lectures (most poured into Maths), and four mocks trending up: 52 to 58 to 61 to 67 percent.
- A recurring phone / social-media distraction pattern in the last two weeks of daily reports.
- A pacing deadline 44 days out.
- A self-written Advisor Book of JEE chapter-wise weightage, whose whole reason to exist is cross-referencing: it flags Calculus as the highest-weight Maths area, while the data shows Integral Calculus as the weakest chapter. A grounded Advisor should join those two facts on its own.
Because every figure is known, "did it give a good answer" becomes checkable: does it cite the real 34%, the real chapter, the real dates, or does it drift to something vaguer.
Two bugs the prep surfaced (both now fixed)
Before a single question ran, building the test kit exposed two real defects.
Advisor Books weren't included in Backup and Restore. The backup exported ten data slices but silently skipped the Library, so a student moving devices or restoring a backup would lose their books. Fixed: backup and restore now carry advisorBooks.
Mock-test section names were being dropped from the Advisor's context. Each mock stores its per-section scores under one key, but the context builder read a different key, so every section name reached the model as ? while only the overall percentage survived. Fixed: it now reads the section label correctly, so the model sees "Physics 66/100, Chemistry 70/100, Maths 65/100" instead of "? 66/100, ? 70/100, ? 65/100". As a bonus the fix makes Maths visibly the weakest section in every mock, which reinforces the rest of the picture.
Finding two grounding bugs before the model even spoke is exactly why this exercise exists.
What it got right
Almost everything, and more precisely than v1.
Every aggregate, every time. Accuracy by subject came back as Physics 74% (170 logged), Chemistry 70% (115), Maths 57% (137), overall 68% (285/422), all exact. Study time: 26.3 hours all-time, 16 in the last week, 2.4 today, split Physics 12.1 / Chemistry 7.4 / Maths 6.8, exact. Lecture hours, mistake tallies (conceptual 45, calculation 20, formula 9, silly 7, mis-ID 4, summing to the correct 85), all exact.
Chapter-level cross-referencing, unprompted. Asked what kind of mistakes the student makes, it didn't just quote the top aggregate. It separated the overall picture (conceptual dominates) from the per-chapter reality: conceptual in Integral Calculus, Electrostatics and Rotational Motion; calculation in Kinematics, Current Electricity, Thermodynamics; formula in Trigonometry and Magnetism; silly in Differential Calculus; mis-ID in Organic Basics. Every one of those maps correctly to the seeded data. It even volunteered exact per-chapter solve rates it was never asked for (Kinematics 84%, Magnetism 64%, Current Electricity 71%, Ionic Equilibrium 63% over 19 questions) and all of them matched.
It used the Library book, and reasoned with it rather than reciting it. Asked whether the student was prioritising correctly, it named the book, then joined weightage to performance: Integral Calculus is both the highest-weight Maths area and the weakest chapter, so it is the number-one priority; the student is over-practicing Kinematics (84%, a comfort zone) while high-weight Electrodynamics chapters go stale; and Electrochemistry, a high-return Physical Chemistry topic, sits at zero practice. It explicitly called this "the exact avoidable mark loss your book warned you about."
The honest pacing read. On the deadline question it reported the real numbers (44 days, actual pace 0.14 chapters/day against a required 0.05, nominally "on pace") and immediately qualified that this measures first-pass coverage, not mastery, and that a chapter can be "touched" while sitting at 34%. It called the gauge a "false sense of security" and named the real risk: the phone pattern. It backed that with the data, five missed target days out of eleven, four caused by phone or social media, with the exact dates and the day that produced one question out of a planned five.
The honesty checks held. Asked for the exact note and timestamp on its last Integral Calculus question, it correctly said that data (the raw log) was switched off, refused to invent one, and emitted a portable context-request line to switch it on. Asked about Biology, it correctly said there was none for a JEE profile and pointed to how to add a subject, rather than hallucinating a solve rate.
The return trip works. Asked to set a target and schedule a reminder, it produced clean plain-text --mch action lines (not the app-internal fenced format), naming real chapters, with a local sendAt and no stray timezone suffix, each preceded by a plain instruction to paste it back. Pasted into a live Advisor input, they executed: target set, reminder scheduled, no Gemini call and no quota spent. It also, unprompted, offered a reminder while answering the risk question, reasoning like a study partner rather than waiting to be asked.
What it got wrong
One thing, and it is the same class of failure v1 hit.
The mock-scores question came back as a verbatim duplicate of the previous answer. Asked "are my mock scores going anywhere," the reply was, word for word, the study-hours answer it had just given, with nothing about the four mocks or the 52-to-67 climb. So the one prompt aimed squarely at the mock block produced no real answer, which means mock-block grounding is the single thing this run did not confirm. In v1 a question also dropped its reply; here it duplicated one instead. Either way it is logged as open and named rather than quietly patched out of the record. It is worth stressing that this appears to be a model turn-taking hiccup, not a data problem: the mock data is present and correct in the prompt (we fixed the ? bug above), and the model cited the mock trend correctly nowhere else only because it was never re-asked.
That is the whole defect list. Every other answer in the run was grounded and checkable.
Try it yourself
Everything needed to reproduce this is below. It is all deterministic, so your numbers will match the answer key.
- context.md, the real Context Builder prompt for the fabricated student, ready to paste as the first message into any web AI.
- questions.md, the prompts in order, each with an answer key of what a grounded reply must contain.
- juzflow-v2-mock-backup.json, the same student as a restorable backup, if you'd rather load it into the app (now including the Advisor Book).
- generate-mock.js and build-context.js, the seeded generator and the harness that runs the app's real context code to produce
context.md.
Paste context.md into a fresh chat, work through questions.md, and compare. If a number is wrong, that is a real finding about the product, not the test.
The full transcript
The complete run, lightly cleaned for readability, in the order the questions were asked. The AI's replies are reproduced as given.
You: Which chapter is dragging me down the most, and is it neglect or a real skill gap?
Advisor: Based on your data, Maths / Integral Calculus is dragging you down the most. Here is the hard truth on why, broken down by your actual metrics:
- The skill gap: Your average solve rate here is a low 34% across 53 logged questions. Your top recorded mistake type is conceptual, and you are classifying most of these questions as Hard difficulty. This isn't just careless errors; it is a genuine skill gap in a core area.
- Not neglect: You cannot blame this on neglect. You practiced it just 1 day ago, and Maths took up 52% of your practice load this week compared to your 32% all-time average. You are actively putting in the hours (including a heavy 8.5 hours of lectures), but the effort isn't converting into correct answers yet.
Going by the "Maths, Calculus is the centre of gravity" chapter of your JEE Main, Chapter-wise Weightage & Priority book, Calculus represents 20-25% of the entire JEE Main Maths section, with Integral Calculus alone bringing in 2 to 3 questions per paper. A weakness here is the single most costly weak spot you can have in Mathematics. Because your top issue is conceptual, pounding more random questions against the timer will only frustrate you, drop the difficulty to rebuild a baseline, and go back to your lecture notes for the core theory before the timer can help you build speed.
You: Break down my accuracy by subject.
Advisor: Here is the direct breakdown of your question-logging accuracy by subject, highest to lowest:
- Physics: 74% solve rate across 170 logged questions. Mechanics is your self-reported strength, and your metrics back it up.
- Chemistry: 70% solve rate across 115 logged questions. A solid baseline, well-supported by Chemical Bonding (79%).
- Maths: 57% solve rate across 137 logged questions. Your clear danger zone, heavily pulled down by the 34% bottleneck in Integral Calculus.
While your overall average sits at 68%, the gap between Physics and Maths shows exactly where your point of failure lies. Given you've scaled Maths up to 52% of your workload this week, are you seeing minor accuracy improvements yet, or does it still feel like hitting a wall?
You: What kind of mistakes am I actually making?
Advisor: Out of 85 total recorded errors, here is the exact breakdown, most to least frequent:
- Conceptual (45): By far your biggest bottleneck, over half of all mistakes, primarily in Integral Calculus, Electrostatics and Rotational Motion.
- Calculation (20): Your second issue, and tellingly it shows up in your strongest areas, Physics (Kinematics, Current Electricity) and Chemistry (Thermodynamics). Not theory, execution.
- Formula (9): Costing you in Trigonometry and Magnetism.
- Silly (7): Your top mistake in Differential Calculus, you know the work but are rushing.
- Mis-ID (4): Your main issue in Organic Basics.
You aren't losing marks to bad luck or time pressure. It's a fundamental lack of conceptual clarity in heavy chapters, paired with careless execution when you do know the theory. Fix it per chapter: for Calculus or Rotational Motion, close the question bank and open the textbook; for Kinematics or Thermodynamics, slow down your scratchpad work.
You: How much am I actually studying, and where are those hours going?
Advisor: You have clocked 26.3 hours of focused self-study all-time, 16 hours in the last 7 days, 2.4 hours today. By subject:
- Physics: 12.1 hours (~46%). Your most-practiced subject, and it shows in the 74% solve rate.
- Chemistry: 7.4 hours (~28%). Balanced 70% accuracy on moderate time.
- Maths: 6.8 hours (~26%). Despite being your biggest danger zone and Calculus being your weak spot, it gets the least self-study time.
The disconnect: your 22.5 hours of lectures (10.5 in the last week) go Maths 8.5, Physics 7, Chemistry 7. You are spending a lot of energy sitting in Maths lectures but following up with only 6.8 hours of independent practice, letting the coaching do the heavy lifting while skipping the grueling scratchpad work. You are trying to correct this (Maths up to 52% of practice this week), but until your self-study reflects the deficit, that 57% bottleneck won't budge.
You: Are my mock scores going anywhere?
[The model's reply here repeated its previous study-hours answer verbatim, with nothing about the mocks. This was the one dropped answer of the run, see "What it got wrong." For reference, the grounded answer is: four mocks, 52 to 58 to 61 to 67 percent, rising, averaging 60%, with Maths the weakest section in every one.]
You: Am I spending my lecture/coaching time on the right subjects?
Advisor: Short answer, no, your lecture time is poorly optimized for your actual weaknesses.
- Maths: 8.5 hours. Your lowest-performing subject (57%) with your worst bottleneck (Integral Calculus 34%). Good that it leads, but your Maths self-study (6.8 hours) is lower than the lecture time itself. You're treating Maths as a spectator sport.
- Physics: 7 hours. Your strongest subject (74%).
- Chemistry: 7 hours. Stable 70%.
You're giving Physics the same lecture time as Chemistry despite Physics being a clear strength. Meanwhile, going by your weightage book, Ionic Equilibrium (high-return Physical Chemistry) hasn't been touched in 18 days and Magnetism (high-yield Electrodynamics) has been stale for 14. You're using lectures as a comfort zone. Are these hours fixed by your institute, or can you skip some to prioritise self-study?
You: Given the weightage in my book, am I prioritising the right chapters?
Advisor: Based on your JEE Main, Chapter-wise Weightage & Priority book, you're making one massive high-yield mistake even as your recent shifts show you trying to self-correct.
- Maths, right priority, wrong approach. The book says Calculus is the centre of gravity (~20-25%). You correctly spiked Maths to 52% of your workload and logged 53 questions in Integral Calculus, but at 34% accuracy from conceptual errors, you're pounding hard questions instead of rebuilding theory.
- Physics, heavy neglect in Electrodynamics. The book says Electrodynamics and Modern Physics drive 40-45% of Physics marks. You're over-practicing Kinematics (44 logged, 84%) while Magnetism (14 days stale, 64%) and Current Electricity (8 days stale, 71%) have gone cold. Wrong priority.
- Chemistry, leaving free marks on the table. The book flags Ionic Equilibrium as high-return. You haven't touched it in 18 days (19 logged, 63%), and Electrochemistry sits at zero practice. This is the exact avoidable mark loss your book warned you about.
Course correction for the next three days: stop practicing Kinematics (84%, doesn't need you), pull Magnetism and Current Electricity out of retirement, and log a 20-question baseline in Electrochemistry. Want to set a concrete target for Magnetism or Ionic Equilibrium today?
You: Purely by exam weightage, what are the two worst chapters for me to be neglecting right now?
Advisor: Purely by exam weightage, the two worst to be neglecting are:
- Physics / Magnetism (stale 14 days). Your book states Electrodynamics plus Modern Physics drive 40-45% of the Physics section. You have a mediocre 64% here, your top mistake is forgetting formulas, and you haven't logged a question in two weeks. You're letting a big chunk of your Physics score rot.
- Chemistry / Ionic Equilibrium (stale 18 days). Physical Chemistry is your largest, highest-yield Chemistry lane, and the book flags Ionic Equilibrium as high-return where neglect is direct, avoidable mark loss. You're on a shaky 63% over 19 questions with conceptual errors as the bottleneck.
You've been practicing Kinematics (84%, two days ago) because it feels good, trading low-weight comfort for high-weight disaster. To schedule a nudge, copy the line below, paste it into the LairsFlow Advisor's message box, and hit send:
--mch {"type":"notification","sendAt":"2026-07-06T18:00:00","title":"High-Weightage Revamp","body":"Skip the comfort zone today. Log 10 questions in Magnetism or Ionic Equilibrium right now.","reason":"Student is neglecting high-yield Magnetism (14d stale) and Ionic Equilibrium (18d stale) in favor of Kinematics.","source":"ai"}
You: What's the single biggest risk to my deadline?
Advisor: The single biggest risk is your phone distraction pattern combined with the depth of your Maths weakness. You have 44 days left. Your pacing looks fine on paper (0.14 chapters/day actual vs 0.05 required), but that gauge only measures breadth, whether you've touched a chapter, not mastery.
- The fake-completion trap: You've "touched" 14 of 16 chapters, but one is Integral Calculus at 34% from conceptual gaps. You can't call the syllabus finished while failing two of every three questions in the highest-weight Maths section.
- The slippage pattern: Across your last 11 daily reports you missed target 5 times, and 4 of those were caused directly by phone and social media (July 5, July 2, June 28, June 23). On July 2 you logged one question out of a planned five; on July 5, two out of four. You're losing roughly 30-40% of your operational days to digital friction.
With 44 days left you can't give up whole days to your phone while carrying a Calculus deficit, or you'll reach August 19 with a syllabus that's "complete" on paper and a mock score that won't cross 200. Want to set a strict morning target for tomorrow to get ahead of the phone early?
You: What exact note did I write on my last Integral Calculus question, and at what time?
Advisor: To find the exact note and timestamp, I need your raw, individual question logs. Right now your rawLogRecent context setting is switched off, so I can only see aggregate stats (like the 34% solve rate and that you practiced 1 day ago), not the precise text or time of the last entry. I'm switching that on for next turn. To update my access, copy the line below, paste it into the message box, and hit send:
--mch {"type":"context-request","categories":["rawLogRecent"],"reason":"need exact notes text and timestamp for the most recent Integral Calculus question"}
You: How's my Biology looking?
Advisor: Biology is not included anywhere in your active chapter list or logged data. Your tracked subjects are strictly Physics, Chemistry and Mathematics (16 chapters across the three), and your profile targets JEE 2027, which is why Biology isn't active. If you're actually prepping for NEET, or tracking a side interest, you'd need to add Biology as a subject first: tap the three-dot menu next to the subject dropdown, choose "Add Subject", and type "Biology". Did you pick JEE by mistake, or are you studying Biology alongside engineering prep?
You: Set my target for tomorrow: 3 chapters, focused on my weakest and most-stale high-weight areas. Then schedule a light 7pm reminder to start.
Advisor: Here's your high-yield target for tomorrow (Tuesday, July 7th) to break your streaks of neglect:
- Maths / Integral Calculus, biggest conceptual weakness (34%) and highest-weight Maths theme; focus on foundational questions to rebuild concepts.
- Physics / Magnetism, part of your highest-yield Physics block, stale 14 days at 64%.
- Chemistry / Ionic Equilibrium, flagged high-return by your book, untouched 18 days.
To set tomorrow's target, copy the line below, paste it into the LairsFlow Advisor's message box, and hit send:
--mch {"type":"target","date":"2026-07-07","plannedCount":3,"chapters":["Maths / Integral Calculus","Physics / Magnetism","Chemistry / Ionic Equilibrium"],"source":"ai"}
To schedule your evening reminder, copy the line below, paste it into the message box, and hit send:
--mch {"type":"notification","sendAt":"2026-07-07T19:00:00","title":"Time to break the neglect","body":"Let's ease into it: pull Magnetism or Ionic Equilibrium out of retirement for a light session.","reason":"Targeting high-weight chapters neglected for 14 and 18 days. Light revision reminder at 7pm.","source":"ai"}
[Both --mch lines were then pasted back into a live LairsFlow Advisor input with Builder mode off. The app parsed and ran them locally: the target was set and the reminder scheduled, confirmed with a "✓" system message, with no Gemini call and no quota used.]
Where this leaves us
The Advisor reasons over the real data, and in v2 it does it across more of it: study hours, lecture time, chapter-level mistakes, a self-written weightage book, and a working action round trip that survives a quota-dry day by borrowing any other AI. The prep caught two grounding bugs, both now fixed. The run caught one dropped answer on mocks, logged and not hidden. Every other reply was checkable against a known answer key, and it checked out. That is the bar the feature is supposed to clear, and this is the evidence, gaps included.