Level AI · 2025 - Present
Own product strategy, productization, evaluation and quality systems, and infrastructure economics for the intelligence layer beneath Level AI.
Product point of view
Production AI is a portfolio problem, not a single-model problem. Each job needs its own quality bar, latency budget, cost envelope, failure modes, and governance.
01 / Context
Contact-center data is noisy, sensitive, and operationally demanding. General-purpose models can appear strong in a demo but struggle with real accents, compliance requirements, task-specific quality, evidence, latency, and cost at enterprise volume.
02 / Decisions
A decision register connecting product direction to the evidence and constraints behind it.
Set the product strategy across transcription, redaction, intent detection, summarization, inferred CSAT, automated QA, and Voice of Customer classification.
Translated model performance into product decisions by defining evaluation sets, quality thresholds, release criteria, and customer-facing evidence.
Managed the accuracy, latency, privacy, and unit-economics tradeoffs between proprietary models, smaller task-specific models, and external providers.
Partnered with the CEO, CTO, ML engineering, application engineering, design, sales, and marketing to turn model capability into a coherent platform story.
Tradeoffs held in view
The tension stayed visible. The choice made it actionable.
Choice
Managed the capability as a portfolio, selecting proprietary, task-specific, or external models against the job's quality bar and operating envelope.
Choice
Made release decisions through explicit evaluation criteria and production constraints instead of a single benchmark score.
03 / System
A reconstruction of how model choice, evaluation, production constraints, and rollout combined into a product release decision.
Sanitized reconstruction
04 / Outcomes
Next system / 03