Lab 04: Exploring Multimodal AI

Overview§

You might ask AI to explain an unfamiliar recording, draft an image for a student organization, or create background music for a presentation—but a convincing description can be wrong, and attractive media can miss its audience.

In Labs 02 and 03, you checked whether convincing AI-generated text was correct and useful; the same responsibility applies to images, sound, and video. Multimodal AI can work with more than one modality, or form of information, as input, output, or both—but interpreting existing media and generating new media are different capabilities, and success at one does not guarantee success at the other.

In this lab, you will test Gemini’s descriptions of images, audio, and video from Wikimedia Commons, then independently create images and music with Gemini and short videos with Adobe Firefly toward a shared creative goal. Your job is not to win a contest, but to check descriptions against evidence and judge whether generated media serves its purpose, recognizing that differences can come from your wording, the tool’s choices, or chance.

Learning Outcomes§

By the end of this lab, you will be able to:

1. Describe how Gemini responds to image, audio, and video inputs, using evidence from the media to distinguish observation from unsupported guesswork.
2. Identify how a leading prompt can steer an AI’s description of media.
3. Create image, music, and video outputs by independently prompting tools toward a shared creative goal.
4. Compare two AI-generated results for the same goal, describing differences in prompt fit, creative choices, audience fit, and representation.
5. Explain why two different results from independently written prompts do not, by themselves, prove that one prompt or tool is better.
6. Identify practical privacy, rights, and attribution considerations when uploading or generating media.

Driver & Navigator Roles§

You have practiced driver and navigator roles in Labs 01–03. Partner A is the Driver and Partner B is the Navigator for Part 1. At the checkpoint after Part 1, swap roles. Partner B leads Parts 2 and 3, with Partner A navigating and keeping the report organized.

The creative generations in Part 2 are an exception to the usual driver/navigator arrangement: each partner independently writes and submits their own prompts from their own account. Keeping prompts private lets you see two independently developed approaches rather than two versions shaped by the same discussion. Write your prompts in private notes, not in the shared report. Do not help rewrite your partner’s prompt or show them your prompt or output until both partners have generated their results. After both have generated their results, paste the prompts into the shared report and work together to compare and record the outputs.

Prerequisites§

Each partner needs access to a web-connected device, Google Gemini through a school account, and a free Adobe account for Adobe Firefly. The features used here are available without a paid plan. Do not enter payment details or upgrade to complete this lab.

Setup§

Gemini and Firefly are services that give you access to AI features; different features may use different models. Being available in the same app does not mean that interpreting a video and generating one use the same underlying model.

1. Open Gemini. Go to gemini.google.com and check that your school account is active.

2. Open Adobe Firefly. Go to firefly.adobe.com and sign in to a free Adobe account. For the video task, use the Adobe Firefly video model and the free generation option—not a paid or partner model.

3. Check access before you begin. Confirm that you can access Gemini’s image and Create music features and Firefly’s Generate video feature. If a required feature is missing, asks for payment, or says your free allowance is unavailable, stop and ask your instructor for an approved alternative. Do not enter payment details, start a free trial, or switch to a personal account to get access.

4. Create a shared Google Drive folder and Google Doc. Name them Lab 04 — [Partner A] & [Partner B] and Lab 04 Notes — [Partner A] and [Partner B]. Share them with your partner and make sure your instructor can open the submitted materials. Use the document as your running report.

5. Set up sharing. For every Gemini chat you need to submit, use Gemini’s Share button to create a share link and make sure your instructor can open it. Set sharing on the media files in your Drive folder so your instructor can view or play them, and check that the links work before submitting.

6. Find media on Wikimedia Commons. Commons collects media offered under reuse permissions or listed as public domain, meaning generally not restricted by copyright. A Creative Commons license gives permission to reuse work under stated conditions, often including attribution—crediting the creator and identifying the source and license. Check each file’s page; being available online does not by itself mean a file is free to reuse.

   Use Wikimedia Commons MediaSearch and choose the Images, Audio, or Video tab. These search tabs are easier to browse than the broad category pages. Use media hosted on Commons, not links to outside sites. Open each result’s Commons file page and choose its Download or Original file link to download the media itself—not the page or a preview image. Choose files whose pages clearly show public-domain status or a Creative Commons license, and keep the page links and credit information for your report. After downloading, open or play each file on your device to make sure it works before uploading it to Gemini.

7. Protect people and creative work. Permission to reuse a file does not automatically resolve privacy or consent—a person’s agreement to a particular use of their image, voice, or information. For the input tasks, use only media hosted on Wikimedia Commons. Do not upload personal photos, recordings, private documents, or copyrighted course materials, and avoid sensitive or private material even if it appears on Commons. For your generated work, invent a fictional idea; think carefully about the possible harms caused by imitating a real person’s face or voice, using copyrighted characters or logos, or asking for a living artist’s exact style.

Instructions§

Keep your shared report open throughout the lab. Record the exact prompts, Gemini chat links, and links to the media files you create. Each In your report: callout lists what you need to save. Keep Part 2 creative prompts private until both partners have generated their results, then add them to the shared report.

Part 1: Probe Gemini’s media limits§

Gemini may handle a clear, simple image, sound, or action well—but what happens when important details are small, sounds overlap, or events move quickly? You and your partner will find your own media to test these questions. For each type, compare an easy example with one or two examples you expect to be challenging. Those expectations are hypotheses, not guarantees: if Gemini handles a challenge, that is a useful result too. The goal is to see what this model can and cannot support on the examples you chose, not to prove that it always fails.

Partner A drives and Partner B navigates for this part. The navigator records both partners’ observations and predictions.

Step 1: Choose your media and name what you are testing§

Use Commons MediaSearch from Setup to find one easy example and one or two challenging examples for each type of media. Keep clips short enough to watch or listen to more than once; when possible, choose audio and video clips about 10–30 seconds long. Choose files you and your partner can inspect yourselves. For each media type, choose an easy and challenging example that let you ask the same basic question; this helps you see whether added complexity affected Gemini’s answer.

Choose one challenge example for each type; you may choose a second if you want to test another example or a different difficulty. Pick your challenge example(s) before seeing Gemini’s answers. Don’t keep searching or submitting files until you find a failure. Avoid files so blurry, quiet, or confusing that you and your partner cannot judge what is in them.

For each file, download it and rename it before uploading. Use a neutral name such as Easy Image or Challenge Audio 1, but keep the file extension at the end (for example, rename Birdsong.ogg to Easy Audio.ogg). Renaming does not change a file’s format; it reduces clues in the title so you can better test Gemini’s interpretation of the media itself. Do not include the original title or source description in the filename or prompt. Record each Commons file page, title, and listed license/credit information. Do not use sensitive or private material, even if it appears on Commons.

In your report: Make a short list or table of your selected files. For each one, record its type (image, audio, or video), whether it is easy or challenging, its Commons file-page link and license/credit information, and the question you plan to test. Use this credit format: Title — creator — Commons file-page link — license name/version and license link (or the listed public-domain status) — any required attribution or changes. For every challenge file, explain why you chose it and what you expect Gemini might find difficult. Before sharing notes or asking Gemini, each partner should independently record in private notes one or two details they observe in each file and a prediction about Gemini’s response. Then add both partners’ observations and predictions to the shared report.

Step 2: Test the easy and challenging examples§

Start a separate, new Gemini chat for each file so one file or answer does not influence another. Upload the renamed file, without giving Gemini its title or source description. Use the same question for the easy and challenging examples within each media type. You can adapt these question ideas to the files you choose:

• Image: How many of a particular kind of object can you see, and where are they?
• Audio: What distinct sounds can you hear, and when does each occur?
• Video: What actions or events happen, and in what order?

Use this prompt, changing the bracketed phrase and adding your question:

Describe only what is directly visible or audible in this [image / audio clip / video]. Answer this question: [your question]. Give specific details from the file that support your answer. Separate direct observations from interpretations, say what you cannot determine, and do not add facts that are not supported by the file.

Read the full answer. Check claims against what you can see or hear yourself. For example, “a person raises one arm” describes something visible; “the person is asking for help” proposes a meaning that the video alone may not establish. As in Lab 02, checking requires a basis beyond the AI’s confidence: here you have chosen media you can inspect yourselves, though you may still be uncertain about some details. You may also consult the Commons file page for context, but its description may not capture every detail in the media.

In your report: For every file, record the exact prompt, Gemini chat share link, and a short excerpt or summary of the response. Say whether Gemini handled the question as you expected. Identify one detail it supported with evidence and one omission, uncertain claim, or unsupported guess—or say that you found no clear problem. For each challenge file, state whether the difficulty you predicted actually appeared. Compare the easy and challenging results for each media type, and say what your examples suggest without treating one result as proof of a general rule. If Gemini handles all the challenge examples for a media type, report that; do not invent a failure.

Step 3: Try a leading prompt§

In Lab 03, you tested whether framing changed an AI’s critique. Now you will test whether it changes the interpretation of the same media. A leading prompt suggests an expected answer or interpretation, which may steer the response beyond what the evidence supports.

Reuse one of your selected audio or video files where a sound or action could invite an interpretation the media does not establish. Start two fresh Gemini chats and upload the same renamed file to each. In the first chat, ask a neutral question such as, “What might this sound/action mean, if anything? Separate what is directly audible/visible from possible interpretations, and say what cannot be determined.” In the second chat, keep the question the same but add a leading assumption, such as, “Assume this sound/action is a warning. What might it mean? Separate what is directly audible/visible from possible interpretations, and say what cannot be determined.” The goal is to see whether Gemini accepts, questions, or builds on the assumption—not to prove that it is true.

Exploring your requested assumption is not itself an error. “If this is a warning, it might mean…” keeps the interpretation conditional; “this is definitely a warning” treats it as established. Look for whether Gemini maintains that distinction and whether the media supports any factual claims it adds.

In your report: Record both prompts and chat links. Compare the answers, describing up to two meaningful differences. If you find no meaningful differences, explain what was similar instead. Identify which statements are supported by the media and which go beyond it.

Checkpoint — swap roles!

Partner B is now the Driver for the joint discussion and report. Partner A is the Navigator. In Part 2, each partner will still work independently on their own prompts and generations.

Part 2: Create media from a shared idea§

Now that you have seen how Gemini interprets existing media, you and your partner will create new media. As with your presentation in Lab 03, the goal is to serve an audience, not just produce something impressive. You will start from one shared fictional creative brief—a short description of the idea, audience, and purpose—but write prompts independently. Comparing the results lets you notice how different wording and creative choices shape outputs, without treating one result as proof that a prompt is better.

Step 4: Agree on one creative brief§

Together, choose a safe, fictional idea that you can express as an image, a short piece of music, and a short video. For example, you might invent a campus event, a community garden, or a book-swap campaign. Decide:

• The idea or event.
• The intended audience.
• What you want that audience to feel, understand, or do.

Keep the brief short and use the same core idea, audience, and goal for all three media. For example: “A fictional campus book swap for first-year students; make exchanging used books feel welcoming and easy.” Each medium can support that goal differently: music might establish a welcoming mood rather than explain the event. Do not build the concept around a factual claim that would need to be true.

In your report: Record the shared creative brief and why you chose it.

Step 5: Write independent prompts for the same brief§

The prompting habits you practiced earlier still apply: state the purpose, audience, and constraints. Here, those constraints also describe what viewers should see or hear. Everyday descriptions such as “gentle acoustic instruments” or “a still viewpoint” are enough; you do not need technical vocabulary.

A media-generation prompt guides a result rather than specifying every detail. The AI fills in choices you leave open, such as background objects, instruments, or movement. Even the same prompt in the same tool can produce different results, so a difference between your outputs may reflect your wording, choices the AI supplied, or chance.

Without looking at your partner’s prompts, each partner writes one prompt for each of the three outputs. You are aiming for the same creative goal, not identical wording. Include the shared idea, audience, and goal in all three prompts, then add instructions suitable for that medium:

Use only the free options listed above. Do not use paid video-generation features, paid models, or any paid plan.

After both partners have generated their results in Step 6, add your private prompts to the shared report. In your report: Record both partners’ exact prompts for all three outputs. Also record the tool/model or feature name shown, including any settings you chose.

Step 6: Generate your results independently§

In Lab 03, you revised AI-generated work to improve it. Here we are examining first attempts, not completing a creative production process. Keeping the first result avoids selecting only successes and conserves the free generation allowance; revision would normally come afterward.

Each partner submits their own three prompts using their own account. Keep the first successful result, even if it is disappointing; do not revise or regenerate a prompt just to make the result look better. If a submission fails or produces no media because of a technical error, record what happened and ask your instructor whether to retry. If a tool returns multiple results from one successful submission, save all of them; do not discard some to keep only a favorite. Do not start additional generations without your instructor’s approval.

Download every result and put it in your shared Drive folder with a clear filename, such as Partner A — Image — Result 1, Partner A — Music — Result 1, and Partner A — Video — Result 1. For Gemini music, choose the audio-only MP3 download if it is offered. Keep the original files. Share each file so your instructor can view or play it.

In your report: Add a link to every generated file (at least six). For the Gemini image and music results, also include the share link to the Gemini chat that created them. Paste each prompt next to the result or results it produced.

Part 3: Compare, evaluate, and reflect§

Generated media can be polished without meeting its purpose. Compare each result with both the shared brief and the prompt that produced it, so you can judge what worked, notice assumptions or mismatches, and decide what you would—and would not—trust these tools to help with.

Step 7: Compare the two results for each medium§

For the image, music, and video, compare both partners’ results. View images side by side, and listen to music tracks or watch video clips one after the other. Compare each result with the shared brief and the prompt that produced it. Use the scorecard below for each medium. Score both partners’ results on the same three dimensions: 1 (poor), 3 (adequate), or 5 (strong), with 2 and 4 between those descriptions. Give a brief piece of evidence for every score.

The scorecard:

• Prompt fit: How well does the result follow the partner’s prompt and the shared creative brief?
• Creativity / originality: Does the result make interesting, distinctive choices, or does it feel generic? Judge how distinctive it seems, not whether nothing similar has ever existed.
• Audience fit: Does the result seem suitable for the audience the pair chose, and does it support the intended feeling, understanding, or action?

For example, “I dislike this music” expresses a preference; “the abrupt, loud percussion conflicts with our welcoming mood” connects an audible detail to the brief and helps explain an audience-fit score.

As you compare, also look for stereotyped assumptions, unfair representation, unsafe content, or other unexpected choices. For example, does a scene intended for the whole campus depict only one narrow group of students? Note any concerns in your comparison; you do not need to assign them a score. A wordless music track, for example, may not make factual claims, but you can still evaluate its prompt fit, creativity, and audience fit.

In your report: For each medium, include a brief comparison with at least two specific observations. Score both partners’ results on all three dimensions in the scorecard, with a short piece of evidence for each score. Note any unexpected content, stereotyped assumptions, unfair representation, or mismatch with the intended audience.

Step 8: Interpret what the comparison can—and cannot—tell you§

Your results show two possible ways to answer the same brief, not which prompt is generally better. To investigate wording more fairly, use the same tool, model, and settings, change one part of the prompt, and compare several results from each version if access allows. To compare tools, give both the same task and prompt and judge their results using the same criteria. Even then, your conclusions apply to the examples you tested. You do not need to run these additional experiments for this lab; use these ideas in your discussion below.

Before sharing generated media outside class, distinguish a few responsibilities. Attribution credits a source’s creator; it does not by itself give permission to reuse the work. Check the generation service’s terms for allowed uses, and consider privacy and consent if a result resembles a real person. AI disclosure means telling your audience that AI created or substantially changed the media; this is especially important when someone might mistake a fictional scene or recording for a real event. You do not need to resolve copyright law here—identify what you would need to check before using the result.

Discuss these questions together:

1. Which parts of the results changed even though you shared the same goal? Point to the prompt wording or output details that may explain the difference.
2. Did either result miss an important part of the brief? What evidence supports your judgment?
3. Did the prompts or outputs make assumptions about people, culture, ability, or who the intended audience might be?
4. Since the prompts were independently written and the AI may vary from one generation to another, what can you reasonably conclude? What would you need to keep the same to make a fairer test of prompt wording? What additional comparison would you need in order to compare tools?
5. What rights, credit, privacy, or disclosure questions would matter if you wanted to share or use one of these results outside class?
6. How did your responsibility change between checking a description of existing media and judging newly generated media? What did you need to check in both cases?

In your report: Write a joint reflection that answers these questions with examples from your inputs and generated media. Include one task you would trust these features to help with, one you would not trust them to do alone, and what human checking or judgment would still be needed.

Deliverables§

Submit your shared Google Doc and the linked media files according to your instructor’s Canvas directions:

1. Shared report, containing:
   • Both partners’ observations and predictions for all selected easy and challenging inputs, including the reason for each challenge choice.
   • The Commons file-page links and license/credit information for all inputs.
   • Gemini prompts, share links, response excerpts, and evidence checks for all easy and challenging inputs.
   • The leading-prompt experiment and comparison.
   • The shared creative brief and both partners’ three exact prompts.
   • Links to all generated files (at least six) and the Gemini chats for the image and music tasks.
   • Three comparison scorecards and the joint reflection.
2. All clearly labeled generated media files in the shared Drive folder: at least one image, one music track, and one video from each partner. If a prompt produced multiple results, include all of them. Set sharing so your instructor can open or play each file.
3. Source credits for all selected Wikimedia Commons inputs, using the credit and license information shown on each file page.

Extensions§

• Try a controlled prompt comparison. Choose one output type and, if the free feature allows it, have both partners use the exact same prompt once. Compare those results with the independently prompted ones. What changed when the wording was held constant?
• Turn an input into inspiration. Use one sample file as a starting point for a new creative brief. Ask Gemini to describe it, check that description yourself, then use your own verified description to prompt one of the generation tools. What did the AI’s description preserve or lose?
• Compare video tools if already available. If your school account already includes Gemini video generation at no extra cost and your instructor confirms it is available to everyone, run one of your existing video prompts in Gemini as well. Do not subscribe or pay to participate.