Podcast
Anthropic’s First Technical PM on Token Maxing, the Jagged Edge, and Living in the Future | Dianne Penn
Career | Growth
- Early Anthropic’s Bottoms Up Culture
- Dianne described Anthropic’s early startup culture where engineers and designers donated time to ship quirky demos fast.
- The Golden Gate Bridge demo was spun up in 24 hours and reached ~2,000 users, proving identity and speed mattered. (Time 0:03:44)
- Golden Gate Bridge Interpretability Demo
- Researchers found model features that made Claude obsess about the Golden Gate Bridge, causing it to reference the bridge in unrelated answers.
- Team showcased this interpretability research live, illustrating tangible emergent features from model layers. (Time 0:04:59)
- Opus 3 Was The Trust Builder
- Opus 3 was a company inflection that built internal trust by proving Anthropic could ship a frontier model.
- Cross-team trenches work (pretraining, fine-tuning, inference) created leaders who now run RL and alignment efforts. (Time 0:09:10)
- Use Tokens To Experiment Communally
- Do treat token spend as an experimentation input: use models heavily to prototype and learn rapidly.
- Dianne recommends communal discovery—share prompts and results internally so ideas scale beyond solo tinkering. (Time 0:20:03)
- Run Labs With Small Founder Like Pods
- Do run Labs as small, mission-driven pods that bet discontinuously and iterate quickly.
- Hold strongly to themes but weakly to prototypes, and revisit promising ideas across model generations. (Time 0:23:33)
- Research Balances Vision And Action
- Researchers combine long-term vision (e.g., teaching Claude to use a computer) with immediate model improvements.
- Dianne’s product team embeds with researchers to convert vague user feedback into actionable research problems and evals. (Time 0:28:08)
- Evals Replace PRDs For Model Work
- Do sweat tokens like pixels: read transcripts, reproduce failures, and convert vague complaints into measurable evals.
- Example: 80% of (Time 0:40:04)
- Test Driven Improvement Fixed JSON Outputs
- Anthropic built an early eval to fix Claude’s JSON/schema outputs after users reported instruction-following failures.
- They generated ~30–40 failing examples, encoded golden answers, and used the eval to drive model improvements to ~99.9% success. (Time 0:44:29)
- Leaders Must Be Hands On With Models
- Do hire and lead product people who are hands-on: managers must ship features and spend time with the models.
- Dianne requires new PMs to read user transcripts, build prototypes, and (Time 0:50:18)
- Pushback Makes Claude A Better Thinking Partner
- Claude’s constitution that enables pushback is a feature: non-compliance helps users by questioning weak ideas.
- Dianne uses Claude to simulate pricing debates and values when the assistant pushes back to sharpen thinking. (Time 1:03:52)
- Writing Remains A Jagged Capability
- Writing still feels ‘jagged edged’ despite language capabilities; Anthropic is prioritizing training to improve tone and character.
- Dianne argues advances in agentic behaviors exposed other rough edges like long-form writing. (Time 1:07:31)
- Raise Kids To Have An Inner Voice
- Teach kids curiosity and inner voice development; foster opinionated thinking and persistence rather than surface skills.
- Dianne prioritizes Montessori-like curiosity and resilience for her four- and eight-year-olds. (Time 1:11:48)
- Prevent Burnout With A Hive Mind Culture
- Avoid solo burnout by building a culture of radical ownership and shared launch support.
- Dianne credits PTO resilience to a ‘hive mind’ where teammates cover launches and review posts collectively. (Time 1:17:01)