The Vault Comes Alive — From a Single Label
In EP 01, I described the Family Vault like this:
"Financial records, legal documents, Eunsoo's drawings, family photos, our favorite restaurants and recipes."
Back then it was a concept. Now it was time to actually build it — and the first thing I reached for was, unexpectedly, wine.
Photograph a wine label
The scenario is simple. You photograph the label of a wine you liked. That's it. Pepper does the rest.
Technically, here's what happens:
- The photo goes to Gemini multimodal — not OCR, a model that understands the image directly.
- The prompt is a sommelier persona. From the label it pulls
wine_name/vintage/region/varietal/producer, and summarizes the producer's story in three lines. - The original photo is saved to the Drive
/winefolder, namedwinename_date. - I add the rating (half-star steps) and notes.
Why multimodal. Reading a label works with plain OCR, but "what this wine actually is" doesn't come out of OCR. Pulling the letters off an Antinori label is one thing; attaching the context — "Tuscany's flagship producer, the originator of the Super Tuscans" — is another. A single multimodal call does both. And for an obscure wine with no label knowledge? I strengthened the prompt to generate a description from region and varietal instead.
When the first wine row appeared in the Table Editor, and the app list showed a label thumbnail + rating + sommelier notes — it hit the same way the first table did back in EP 03. The moment the "Family Vault" from the documents became something you could hold.
Then recipes, restaurants, contacts
I repeated the same pattern per type.
Recipe — a dish photo, a category (Korean/Japanese/Western/Dessert…), a rating. No LLM here. A recipe is me saving something I already know, not something for Pepper to analyze. For each type I decided separately: does this need AI?
Place — restaurants. Here the input UX was the whole game. In Korea you hit "share" in Naver Maps. Pepper smart-parses that shared text — the URL, the place name, and even the city from the third line. country="Korea" is set automatically. Google Maps is manual entry. The list collapses in two levels: Country → City.
Contact — with account details, optionally.
As the Vault grew type by type, that sentence from EP 01 was turning into data. "The more the Vault accumulates, the deeper Pepper knows us."
Making Pepper read what you upload
Piling things into the Vault and Pepper knowing them are different things.
So I added Drive auto-embedding. When you upload a file, Gemini multimodal reads its contents and stores them as an embedding in pgvector. Then Layer 3 from EP 09 (embedding-based Vault search) can search the contents of documents too. And a daily 3 a.m. cron sweeps Drive to sync new files.
An upload hook (embed right after upload) plus a cron (catch what was missed) — this two-path structure bit me twice later. A cron creating phantom rows from orphaned files whose app-side save never finished, and an embedding-similarity filter cutting off all the correct answers. Both in EP 17.
When the ground you're standing on disappears
While filling the Vault I also attached a few capabilities. naver.place (a place deep link), naver.shopping (lowest-price product search). The picture: Eunsoo asks for something, and it links straight to the cheapest listing.
Not long after I built it, Naver announced it was shutting down its Search Shopping API (late July). naver.shopping died. I replaced it with a deep link.
There's a lesson in that. Pepper doesn't run only on code I wrote. Naver, Google, Kakao — it sits on top of platforms I don't control. Kakao Maps went paid; Google's models post shutdown notices generation after generation (like the Gemini 2.0 sunset in EP 04). The more capabilities there are, the larger the surface area for the ground under me to shift.
So capabilities had to default to "fail quietly, fall back" from the start. This principle connects to the next episode's theme — except the next episode is the opposite story. The one where failing quietly becomes the disaster.