Overview§

It is tempting to ask a chatbot to “summarize Chapter 4” in a single prompt, but a passive summary feels clear while leaving you unprepared for an exam—genuine learning happens in the act of creating a study guide and testing what you actually know. In this lab, you and your partner will work in driver and navigator roles to build, test, and refine active study guides for real courses, steering an AI toward self-testing and misconception-hunting, then comparing a commercial chatbot (Google Gemini) with a small open-weight model (NRP’s gemma-small). Along the way you’ll confront the “novice verifier” dilemma: how can a student who is not yet an expert tell whether an AI’s study guide is correct?

Acknowledgments§

Much of this lab’s design is adapted from Geoffrey Challen’s Understanding AI course (UIUC CS 199) — including the idea of learning by building a study guide, the illusion of competence framing, and the prompting tips in Step 4.

Learning Outcomes§

By the end of this lab, you will be able to:

  1. Set concrete, measurable learning goals and direct an AI to build active, self-testing study materials for them — practice questions, misconception checks, or an oral-exam simulation — rather than passive summaries.
  2. Recognize the illusion of competence by actively self-testing on AI-generated materials and using your mistakes to guide a revision.
  3. Compare how a large commercial chatbot (Google Gemini) and a small open-weight model (NRP’s gemma-small) handle the same study-guide task — both building and critiquing — and articulate where the smaller model falls short.
  4. Verify AI-generated content — by checking it against authoritative course materials and by prompting two different AI models to critique the same study guide — and judge whether it can be trusted.
  5. Weigh the tradeoffs of using AI for study and discuss the ethical implications of AI in high-stakes professional judgment.
  6. Collaborate effectively in driver and navigator roles.

Driver & Navigator Roles§

You already practiced the core mechanics in Lab 1 — the driver types and thinks aloud, the navigator keeps the running log, tracks the “In your report:” items, and confirms the share links. Those basics don’t change here, so we won’t repeat them. What changes: each of you also becomes the other’s study coach and intellectual sanity check, which adds a few study-guide-specific duties.

You’ll build two separate study guides — one for Partner A’s topic (Topic A) and one for Partner B’s topic (Topic B). Both topics are set up together in Part 1; after that, each partner drives the guide for their own topic:

On top of the Lab 1 basics, these are the duties that are new or adapted for this lab:

Prerequisites§

Before starting this lab, ensure you have:

  1. Completed Lab 01: Intro to AI & LLMs (including active accounts for Google Gemini and the NRP chatbot).
  2. Access to course materials (such as a textbook, syllabus, lecture slides, or instructor notes) for at least one course you are taking this semester (other than this AI course). Both partners must have access to their respective course materials during class.

Setup§

  1. Open both chatbots (as in Lab 1): Google Gemini at https://gemini.google.com/ with your school Google account active in the lower-left corner, and the NRP chatbot at https://nrp-openwebui.nrp-nautilus.io/ as you did in Lab 1, being sure to select the gemma-small model.
  2. Create your shared Google Doc:
    • Create a new Google Doc named Lab 02 Notes — [Partner A First & Last] and [Partner B First & Last].
    • Share it with your partner (edit).
    • Use Google Docs’ Tabs feature (not browser tabs) to split your work into three tabs:
      • Tab 1: Lab Log & Reflections — your running log. Every “In your report (Tab 1):” entry from all four parts goes here. Keep this tab open and write here as you go.
      • Tab 2: Study Guide 1 — [Partner A’s Topic] (built with Google Gemini) — paste the finished guide here.
      • Tab 3: Study Guide 2 — [Partner B’s Topic] (built with NRP’s gemma-small) — paste the finished guide here.
  3. Capture every AI chat as an accessible share link. Every conversation with an AI in this lab — the main study-guide sessions and any quick assist — must appear in your report (Tab 1) as an accessible share link. Each step’s “In your report” line names its required link. Make every link accessible by giving your partner and your instructor access (must be done manually in the NRP chatbot) — the instructor opens them when grading. If you start a brand-new chat, its link counts too.
  4. Reinforced from Lab 1 — know your models’ difference: NRP models have no web search; they answer only from what they learned in training. Gemini can search the web, and may do so automatically. This matters most in Part 3, where each model’s information source may affect its accuracy.
  5. Respect your professor’s course materials. Lecture slides, custom problem sets, homework and exam questions, and instructor notes are your professor’s intellectual property. Without explicit permission, do not paste that material into any AI service (Gemini, NRP, or otherwise). You can use without permission: topic and concept names, your own notes, and published textbook material you’re assigned to read (short, paraphrased excerpts). If a step calls for an instructor’s sample or exam question, get permission first or describe it in your own words. When in doubt, ask.

Instructions§

This lab has four parts that build on each other. Partner A drives first while Partner B navigates; you swap once Study Guide 1 is finished. As in Lab 1, keep your shared document open the whole time and let the navigator record every prompt in the log — the In your report (Tab 1): lines tell you exactly what to write down, so if you’ve recorded all of them, you haven’t skipped anything.


Part 1: Topics, Goals, and Active Study Design§

In this part, Partner A will be the Driver (typing into Google Gemini) and Partner B will be the Navigator (recording notes).

Step 1: Choose your courses and topics§

Why: A study guide is only useful if it helps you learn something you don’t already understand. Testing yourself on concepts you mastered weeks ago creates a false sense of security.

  1. Partner A: Pick a course you are currently enrolled in this semester and an upcoming or recent topic (Topic A) that you have not yet fully mastered.

    Examples: Introductory Microeconomics — “Price Elasticity of Demand and Supply”; Introduction to Psychology — “Operant Conditioning vs. Classical Conditioning.”

  2. Partner B: Pick a course you are currently enrolled in this semester and an upcoming or recent topic (Topic B) that you have not yet fully mastered.

  3. Verify the unfamiliarity rule: Because each of you will study and test your own topic, make sure you have not already mastered the one you picked. If you already feel like an expert on it, the study session won’t teach you much—choose a different topic (e.g., if you picked an intro-psychology concept you already know well, pick one you’re still working through).

  4. Confirm that you have your course materials (textbook chapters, lecture slides, or class notes) open and ready for both topics.

In your report (Tab 1):

Step 2: Formulate three concrete learning goals per topic§

Why: Asking an AI to “make a study guide” without specifying learning goals produces a disorganized grab-bag of trivia. Clear, measurable learning goals force the AI to target specific competencies.

A learning goal describes what a student should be able to do (e.g., “Calculate the net force on an object on an inclined plane,” “Distinguish between positive and negative reinforcement,” or “Explain the difference between mitosis and meiosis”).

  1. For Topic A, brainstorm 3 concrete learning goals.
  2. For Topic B, brainstorm 3 concrete learning goals.
  3. Optional AI assist: If you struggle to articulate specific goals, you may query Gemini. Ask for 4–5 so you have extra candidates to choose from:

    “I am studying [topic] in my [course name] class. What are 4 or 5 concrete, testable learning goals or skills an instructor would expect students to master for a midterm exam? Give them as action statements (e.g., explain, calculate, compare).”

  4. Refine the suggestions into 3 distinct goals for each topic.

In your report (Tab 1):

Step 3: Debate active vs. passive study structures§

Why: Reading a clear summary creates the illusion of competence—the mistaken feeling that because an explanation makes sense while you read it, you can recall and apply it on a test. Genuine learning comes from pulling the answer from your own memory and explaining it in your own words—solving problems, not just re-reading a summary that already looks right.

  1. Partner debate: Spend two minutes discussing your past study habits. When have you read over notes or study guides and felt confident, only to struggle on the exam? What study activities actually made you think?
  2. Now, prompt Google Gemini (Partner A driving) to explore active study architectures for Topic A:

    “I am preparing an active study guide for [Topic A] based on these learning goals: [paste Topic A goals]. Instead of summarizing the material, suggest 3 distinct structural formats for a study guide that maximize active learning and self-testing. Give the pros and cons of each format.”

  3. Read and consider Gemini’s suggestions.
  4. Choose the overall structure you want to use for Study Guide 1.

In your report (Tab 1):


Part 2: Generation, Self-Testing, and Revision§

In this part, you will build Study Guide 1 using Google Gemini, test yourselves, revise it, and then swap roles to build Study Guide 2 using NRP’s gemma-small.

Step 4: Build Study Guide 1 with Gemini (Topic A)§

Why: Standard AI prompts may generate generic, superficial questions. To create an effective study guide, you must apply targeted prompt engineering.

Partner A drives using Google Gemini. For each of the 3 learning goals for Topic A, generate 3 to 5 questions or tasks.

You must apply at least two of the following advanced prompting techniques (adapted from Geoffrey Challen’s course):

Assemble your generated study guide into Tab 2 of your document.

In your report:

Step 5: Test yourselves on Study Guide 1§

Why: A study guide is a diagnostic instrument. Topic A is Partner A’s course, so Partner A is the person who actually needs to learn it—which means Partner A is the one who must take the test under exam-like conditions to find the holes in their own knowledge.

  1. Close your lecture notes and textbooks!
  2. Partner A takes the test: Partner A (the topic owner) attempts to answer or solve each question for Topic A from memory. Partner B acts as the “examiner,” reading the questions aloud or pointing to them in the document, and does not reveal answers.
  3. Partner A works through every question. Where it helps, Partner B can prompt Partner A to explain their reasoning, but never gives away the answer.
  4. Navigator (Partner B) records the results: Which questions did Partner A answer easily? Which caused hesitation? Which were answered incorrectly or not at all? These mistakes are the roadmap for the revision in Step 6.
  5. Optional: Partner B may also try the most difficult questions, purely to test themselves—but the focus stays on Partner A’s own mastery of Topic A.

In your report (Tab 1):

Step 6: Revise Study Guide 1 based on what you missed§

Why: A study guide that tests what you already know is wasted time. True learning occurs in the gap between what you know and what you don’t.

  1. Retire the mastered goal: Identify any learning goal(s) where you answered all questions easily. Discard or archive this goal. You do not need to study it further right now.
  2. Double down on the struggle: Identify the learning goal or specific questions where you stumbled during Step 5.
  3. Prompt Gemini to either:
    • Break that difficult goal into two deeper subgoals, or
    • Generate 3 new targeted questions specifically addressing the errors or confusion you demonstrated in Step 5.
  4. Add your revised, sharpened study guide to Tab 2.

In your report:


Checkpoint — swap roles!

Partner B is now the Driver (operating the NRP chatbot with gemma-small). Partner A is now the Navigator (managing notes and tracking).


Step 7: Build, test, and revise Study Guide 2 with NRP’s gemma-small (Topic B)§

Why: In Lab 1, you observed differences in raw text generation between commercial chatbots and open-weight models. Now you will see how a small open-weight model (gemma-small, 12 billion parameters) handles complex pedagogical instructions compared to Gemini.

Repeat Steps 4–6 with these modifications:

In your report:


Part 3: Checking Correctness§

A fundamental challenge of using AI for education is the novice verifier dilemma: generative AI can make subtle or confident errors, but a student studying the topic is not yet an expert capable of spotting every mistake. In this part, you will use two distinct strategies to verify correctness without being an expert.

Step 8: Spot-check against authoritative course materials§

Why: When in doubt, students often turn to a general Google or web search. However, generic web searches lack the pedagogical context and specific terminology expected by your professor. Authoritative course materials (your textbook, syllabus readings, or lecture slides) represent the “ground truth” for your course.

  1. Pick one question and answer from Study Guide 1 (Gemini) and one question and answer from Study Guide 2 (gemma-small).
  2. Open your textbook, lecture slides, or instructor notes for each respective course.
  3. Find the exact concept, formula, or historical event in your course materials and compare it against the AI’s question and implied answer.
  4. Check for three potential issues:
    • Factual accuracy: Did the AI get the facts, definitions, or steps right?
    • Scope & context: Did the AI frame the concept the way your course frames it, or did it introduce irrelevant outside jargon?
    • Misleading simplicity: Did the AI oversimplify a nuance that your textbook emphasizes?

In your report (Tab 1):

Step 9: Elicit AI critiques — and compare models§

Why: Ask an AI, “Is this right?” in the same conversation where it created the guide and it usually answers, “Yes, looks great!” Two habits cause this: you haven’t told it to look for problems, so it defaults to agreement; and its own draft is still in its context (the conversation history it reads before every reply), which nudges it to quietly defend what it already wrote.

We break both. We prompt the AI to adopt the persona of an expert critic whose specific job is to find flaws, and — the subtle part — we start a brand-new chat with an empty context and paste the guide in as a plain document. With no memory of writing it, the model can’t protect its own output; it reviews the guide the way a neutral stranger would, often catching flaws it would otherwise wave through.

To make the comparison fair, have both models critique the same guide. The two critiques become apples-to-apples, so any difference in sharpness reflects a difference between the models, not between the guides.

  1. Pick the guide to critique. As a pair, choose whichever of your two study guides you feel is weakest — the one with the thinnest questions, the most confusion during self-testing (Steps 5/7), or the most problems in your Step 8 spot-check. There’s no wrong answer; name it and briefly note why you picked it.
  2. Critique the chosen guide with Gemini (in a fresh chat): Start a brand-new chat in Google Gemini so its context is empty — do not continue any earlier conversation. Send the following critique prompt, pasting the full text of your chosen guide in place of the bracketed placeholder:

    “Adopt the persona of a demanding university professor who is an expert in [the topic of the chosen guide]. Below is a study guide I am preparing. Treat it as someone else’s work that you are reviewing — do not assume it is correct, and do not flatter me. Find any factual errors, ambiguous wording, or conceptual holes. More importantly, critique the pedagogical design: which questions are too easy, which encourage rote memorization rather than deep thinking, and what critical exam topics were omitted? — [paste the full text of the chosen guide here]”

  3. Critique the same guide with gemma-small (in a fresh chat): Start a brand-new chat in the NRP chatbot with gemma-small selected so its context is empty. Send the exact same critique prompt with the same guide pasted in place of the placeholder.
  4. Analyze the results:
    • Did either model identify genuine weaknesses or mistakes in the guide?
    • How did the critique from Gemini compare to the critique from gemma-small? Did the smaller model offer meaningful critique, or was it superficial?

In your report (Tab 1):


Part 4: Reflection and Ethical Implications§

Work together collaboratively on this section. Either partner may drive or you may share typing in your document.

Step 10: Metacognition: Learning vs. the “illusion of competence”§

Why: Generating a study guide with AI can feel productive even when no learning occurred in your brain. Taking time to reflect on what you actually retained—and what the AI’s help cost you—separates real learning from artificial productivity.

Discuss with your partner:

  1. Did the act of defining learning goals, steering the AI prompts, and taking the practice tests actually advance your understanding of your chosen topic?
  2. What concrete evidence do you have of your learning? (For example: can you explain a concept right now without looking at the AI’s output?)
  3. How did your experience comparing active questioning versus passive reading illustrate the illusion of competence?
  4. What, if anything, did you give up by having the AI generate the questions instead of writing them yourself? Name one part of the process where the AI did work that would have been learning for you to do.

In your report (Tab 1):

Step 11: The professional expertise dilemma§

Why: In Step 8 you couldn’t fully verify the AI’s study guide because you’re still learning the topic, and in Step 10 you named work the AI did that would have been valuable learning if you had done it yourself. Now scale both up: in fields like medicine and law, licensed experts must review everything AI produces and take full responsibility for it. That rule sounds safe, but it raises two dilemmas that are the professional-scale versions of what you just experienced.

Your task:

  1. Pick one critique to debate with your partner. (Note they pull opposite ways: A worries AI destroys the training ladder, B wonders whether that ladder is still necessary.)
  2. Build your case first, without the AI. One of you argues in favor of the critique, and the other argues against it. Each of you writes down 2–3 points of your own, shares them with each other, and pushes back on what you hear.
  3. Then query Google Gemini as a sparring partner to test your arguments:

    “We are debating this ethical question regarding AI and professional expertise: [summarize Critique A or Critique B]. Here are our arguments in favor: [paste your points]. Here are our arguments against: [paste your partner’s points]. Poke the biggest holes in each argument, and give us one strong reason on the side we’re weakest in that we haven’t considered. Focus on concrete real-world consequences.”

  4. Revise your positions based on the AI’s critique. If the AI finds a hole you can’t defend, concede it and rebuild around something stronger.

In your report (Tab 1):

Step 12: Reflect on teamwork and the lab itself§

Why: Collaborative teamwork is an essential professional skill. And zooming out to evaluate the lab’s own design—what worked and what didn’t—is how you (and your instructor) learn what actually helped you learn.

Discuss with your partner:

In your report (Tab 1):


Deliverables§

Submit your work on Canvas following the instructor’s specific directions. Your submission is the three-tab Google Doc you built during the lab:

  1. Tab 1: Lab Log & Reflections — the running log. Each step’s “In your report (Tab 1):” callout tells you exactly what to record; if you’ve captured every one, you’ve captured all of this tab’s deliverables.
  2. Tab 2: Study Guide 1 — [Topic A] — the full initial draft and revised draft created with Google Gemini.
  3. Tab 3: Study Guide 2 — [Topic B] — the full initial draft and revised draft created with NRP’s gemma-small.

Extensions§

Looking to explore further? Try any of these optional challenges: