Skip to content

Podcast

Enterprises Are Concerned About AI Costs, Governance and Trust

The Enterprise AI Show

Source ↗ ← All highlights
  • Enterprises Use A Buffet Of Models
    • Enterprise AI projects rarely standardize on one vendor; teams use a buffet of Anthropic, OpenAI, Microsoft, Google and others during experimentation.
    • Brian Gracely observed many groups still operate shadow AI and test multiple models before choosing a direction, causing a mixed-tool landscape. (Time 0:04:21)
  • Teams Expect Model Costs To Jump
    • Rising model costs are a top concern; initial contracts are often below true token cost and teams expect prices to rise 3–5x.
    • Organizations plan semantic routing, lower-cost models, and API filtering to control future spend. (Time 0:10:18)
  • Design For Model Flexibility Upfront
    • Build flexibility into architecture now by enabling routing to cheaper models and adding filtering at API gateways.
    • Brian Gracely noted teams are already implementing semantic routing and small LLM fallbacks to manage token bills. (Time 0:11:08)
  • Governance Is Nonnegotiable In Regulated Firms
    • Governance moved from optional to central: regulated firms now demand auditability, data controls, and guardrails for production GenAI.
    • Financial services teams raised governance as a top concern alongside cost and indemnification requirements. (Time 0:11:53)
  • Metrics Vary And Confuse Stakeholders
    • Measurements exist but lack a shared language; teams measure productivity variably (percent gains, hours saved) making cross-team comparisons hard.
    • Brian Gracely highlighted inconsistent reporting where same metrics trigger different stakeholder reactions about job elimination vs. unlocked innovation. (Time 0:14:02)
  • Explain AI Gains In Business Terms
    • Translate technical gains into business terms for nontechnical stakeholders, clarifying whether hours saved reduce headcount or enable new work.
    • Brian Gracely advised preparing clear, non-AI explanations of value to avoid misinterpretation of time-savings metrics. (Time 0:14:55)
  • Trust Is The Hard Problem After Governance
    • Trust is the next gating factor after cost and governance; enterprises struggle to make inherently nondeterministic GenAI outputs feel reliable.
    • Firms invest in guardrails, red teaming, and control mechanisms to reduce hallucination and drift for critical financial use cases. (Time 0:17:31)
  • Use Case Fragmentation Mirrors Maturity Gaps
    • Use cases remain fragmented and domain-specific; some teams have clear roadmaps while others still can’t see practical applications.
    • Brian Gracely observed a wide maturity gap where some groups are AI-first and others remain on the sidelines uncertain how to apply the tech. (Time 0:20:21)
  • Adoption Follows Centralize Then Re-decentralize Wave
    • Adoption cycles show centralize then decentralize then semi-controlled decentralization as teams scale and costs or governance force reorganization.
    • Brian Gracely described a wave where shadow AI centralizes for control, matures, then re-decentralizes on shared platforms and processes. (Time 0:22:18)
  • Security Tools And Risks Aren’t Universally Known
    • Not everyone has access to advanced cyber-focused models like Mythos or GPT-5.5-Cyber, creating steep knowledge distribution gaps.
    • Brian Gracely noted security teams that had seen these tools understood fears better, while others remained unaware of risks. (Time 0:25:51)