Project Overview
Type: Internal GenAI Insight System + Advanced Prompt Engineering
Team: Fay Cai, Iris Bierlein.
Role: AI product management & design
Tools: Google Sheets + Apps Script + OpenAI API
Timeline: 02/2025 — 04/2025
A lightweight, scalable insight engine designed for the NYU Library UX team to understand chaotic and unstructured user feedback through semantic clustering, theme mapping, and longitudinal trend analysis—without the need for a separate product UI.
Impact & Outcome
• 90%+ reduction in manual tagging
• 2 new UX interventions launched
• Shifted UX team’s workflow from reactive to proactive
• Recurring themes surfaced across time
• Callout quote by UX Director Lisa Gayhart:
“This system made it so persuasive and powerful for us to advocate for UX changes to the leadership team.”
Problem
The UX team collected large volumes of user feedback from different cohort each quater—but relied on:
• Manual categorization (inconsistent)
• Basic sentiment scoring (often inaccurate or vague)
• No system for recurrence detection (hard to spot chronic vs. emerging issues)
Design Goal
To guide a large language model (LLM) to infer intent, emotion, and behavioral patter from unstructured user feedback using principles from cognitive and behavioral science, and to design a lightweight, maintainable internal tooling system that extracts:
• Recurring unmet needs
• Suggestions and support patterns
• Latent behavioral insights
—all without introducing a new product interface or dashboard, but instead leveraging
existing workflows (Google Sheets + Apps Script) to empower the UX team with actionable,
psychologically-aware intelligence.
User
Story
"As a UX manager, I need a simple, private, low-maintenance system that helps me understand what users are consistently struggling with—without reading every single feedback—so I can prioritize fixes and advocate for changes more efficient and consistently within the organization."
Solution: GenAI-Powered Insight System (Inside Google Sheets)
Input: A raw feedback sheet (timestamped, anonymous entries)
Automation Pipeline:
1. Google Apps Script monitors new entries and connects them to OpenAI API using add-on menu
2. Custom-Engineered prompt categorize entries into:
• Semantic Category (Complaint, Suggestion, Positive Support)
• Precise Topic (e.g., “bathroom,” “wifi,” “staff,” etc.)
• Behavioral Intent & Psychological Cue (optional advanced prompt layer)
• Data Visualization
3. The GenAI response is parsed and appended back into the Sheet
Output & Interface:
The Google Sheet becomes a live insight system, with:
• Categorized feedback for filtering and searching
• Precise topic tagging for theme mapping
• Auto-generated cohort analysis (e.g., quarter-over-quarter breakdowns)
• Dynamic heatmaps showing issue frequency and persistence
• Trend pivots for identifying chronic vs. emerging problems
Why this Approach Works:
• No New Interface: Uses a tool the UX team already knows and trusts
• Scalable Intelligence: Every new entry is semantically and behaviorally analyzed
• Actionable by Design: Insight is organized and immediately usable—no additional parsing needed
• Batching optimizations will reduce token consumption costs.
Key Design Principles
• Behavior-First Prompting: Aligns LLM reasoning with human affect models
• Workflow-Native Design: No new tools, just smart layers on top of existing ones
• Time-Sensitive Insight: Uses cohort mapping to distinguish chronic vs. transient issues
• Low-Friction Collaboration: Designed for non-technical UX stakeholders
Add-on Menu through Google App Script Allow non-technical team member easy to operate the system
Stack Overview
Component
Role
Google Sheets
Central UI and database for feedback
Apps Script
Connects Sheets to GenAI via API, handles automation
OpenAI API (GPT-4)
Processes each comment and returns: semantic category, precise topic, behavioral intent, etc.
Custom Prompt Engineering
Structured prompts instruct GPT to output structured insights (not open-ended summaries)
Advanced Prompt Engineering:
Cognitive Design Layer— Prompt as Behavioral Scaffold
To move beyond sentiment and extract actionable insight, I embedded behavioral psychology principles directly into the prompt. This allowed LLM to:
• Detect user frustration signals
• Identify latent needs (not just surface complaints)
• Categorize intent through a behavioral lens
Behavioral Dimensions Used in the Prompt:
Behavioral_Intent: Based on intent inference from behavioral economics and affective computing.
Psychological_Cue: Inspired by self-determination theory, BJ Fogg’s behavior model, and affective UX research.
Insight_Summary: Designed to reflect latent needs, not just surface issues (e.g., “seeking control over environment”).
Example Insights Generated by the AI-powered System
Information Architecture & System Flow (Automated)
Key Takeaways
• AI is most powerful when it augments existing habits—not replaces them.
• UX strategy isn’t just about screens—it’s also about mental models and workflows.
• Lightweight tools can create deep change when aligned with user behavior.
• Insight design goes beyond visualizing data—it interprets it, prioritizes it, and transforms it into clear, actionable understanding—turns feedback from noise into navigation—helping UX teams prioritize interventions and advocate for systemic change.