[Skip to content](https://www.hubspot.com#main-content) [](https://www.hubspot.com/startups) - [Resources](https://www.hubspot.com/startups/resources) Show submenu for Resources - [Fundraising](https://www.hubspot.com/startups/fundraising) - [Sales & Marketing](https://www.hubspot.com/startups/sales-and-marketing) - [AI Tech & Tips](https://www.hubspot.com/startups/ai) - [Tech Stack Guide](https://www.hubspot.com/startups/tech-stacks) - [Founder & VC Insights](https://www.hubspot.com/startups/scaling-smarter) - [Podcast](https://www.hubspot.com/startups/podcast) - [Stories](https://www.hubspot.com/startups/stories) Show submenu for Stories - [Digital Magazine](https://www.hubspot.com/startups/magazine) - [DocuSeries](https://www.hubspot.com/startups/docuseries) - [Case Studies](https://www.hubspot.com/startups/stories/customers) - [Community](https://www.hubspot.com/startups/community) - [Partner](https://www.hubspot.com/startups/partners) - [Apply Now](https://app.hubspot.com/signup-hubspot/hubspot-for-startups) - [Resources](https://www.hubspot.com/startups/resources) Show submenu for Resources - [Fundraising](https://www.hubspot.com/startups/fundraising) - [Sales & Marketing](https://www.hubspot.com/startups/sales-and-marketing) - [AI Tech & Tips](https://www.hubspot.com/startups/ai) - [Tech Stack Guide](https://www.hubspot.com/startups/tech-stacks) - [Founder & VC Insights](https://www.hubspot.com/startups/scaling-smarter) - [Podcast](https://www.hubspot.com/startups/podcast) - [Stories](https://www.hubspot.com/startups/stories) Show submenu for Stories - [Digital Magazine](https://www.hubspot.com/startups/magazine) - [DocuSeries](https://www.hubspot.com/startups/docuseries) - [Case Studies](https://www.hubspot.com/startups/stories/customers) - [Community](https://www.hubspot.com/startups/community) - [Partner](https://www.hubspot.com/startups/partners) - [Apply Now](https://app.hubspot.com/signup-hubspot/hubspot-for-startups) - [Home](https://www.hubspot.com/startups) - [AI Blog](https://www.hubspot.com/startups/ai) # What 90% of Startups Miss About AI Agent Infrastructure (And How to Fix It) Learn how to build scalable AI agent infrastructure from industry leaders at Datology AI and Fireworks AI. Cut costs 10x with better data strategies. [Unlock Access to Full Library](https://www.hubspot.com/startups/ai/aisummit-sessions)  The next generation of AI agents isn't just about smarter models—it's about the infrastructure that powers truly autonomous systems. In this compelling fireside chat from the HubSpot for Startups annual AiSummit in San Francisco, three industry leaders broke down what it really takes to build scalable, cost-effective AI agent infrastructure.   __Featured speakers:__ - __Shaown Nandi__ - Director, Technology, AWS (Moderator) - __Ari Morcos__ - CEO and Co-founder, Datology AI - __Shaunak Godbole__ - Field CTO, Fireworks AI ## The Real Pain Points Driving AI Infrastructure Innovation ### Data Quality: The Hidden Multiplier Ari Morcos kicked off the discussion with a powerful insight that challenges conventional wisdom about AI training: __"Models are what they eat."__ His background as an AI researcher at DeepMind and FAIR revealed a fundamental problem—most AI training assumes all data is created equal, when in reality, the vast majority of data isn't helpful for teaching models desired behaviors. Key insights from Datology's approach: - Training on low-quality data requires __orders of magnitude more compute__ for far worse results - High-quality data acts as a __compute multiplier__, delivering 10x more performance per dollar - Smart data curation can reduce training costs from $50M to just a couple million dollars ### The Performance-Cost-Speed Triangle Shaunak Godbole from Fireworks AI identified three critical pain points that emerged when GPT launched: - __Hallucinations__ significantly impacted reliability - __Cost__ made proprietary models prohibitively expensive for scale - __Performance__ created poor user experiences with slow response times Fireworks' solution focuses on serving __5 trillion tokens daily__ while enabling companies to build complex agents using hundreds of models simultaneously—without sacrificing cost efficiency. ## Breaking the Traditional Trade-off Paradigm ### The Enterprise vs. Consumer Model Divide The speakers highlighted a crucial distinction that many startups miss: __Consumer use case:__ Models need to be "a mile wide and at least an inch deep"—capable of answering questions about anything. __Enterprise use case:__ Models should do one thing to __five nines of reliability__ at the lowest possible cost. This insight is reshaping how companies approach AI agent development, moving away from general-purpose models toward specialized, smaller language models tailored to specific workflows. ### The New Economics of AI Agents As AI agents become more complex—with some customers using __hundreds of steps__ in their workflows—the traditional approach of using the same large model for every call becomes prohibitively expensive. A 30-minute deep research task can cost __$2,000__ on proprietary model providers, with equally frustrating latency issues. The solution? __Smaller, specialized models__ that are: - Tailored to specific use cases - Trained on relevant proprietary data - Optimized for the enterprise's unique traffic patterns ## Build vs. Buy: Strategic Advice for Startups ### The 90/10 Rule in AI Infrastructure Shaunak shared a critical insight for startup founders: __"You can build up to 90% of what you need with 10% of effort. But the last 10% is the real magic—the real user experience and value for customers."__ This is where partnerships become essential. Rather than building everything in-house, successful startups are: - Partnering with data curation platforms for training optimization - Leveraging specialized inference engines instead of building their own - Focusing internal resources on the unique differentiation that drives their business ### The Three Optimization Dimensions When scaling AI applications, companies typically face trade-offs between: 1. __Better__ performance 2. __Faster__ response times 3. __Cheaper__ operational costs The key is working with infrastructure partners who can help optimize based on your specific latency thresholds and cost budgets, rather than trying to solve these complex optimization problems internally. ## The Future of AI Agent Infrastructure ### Reducing Human-in-the-Loop Requirements Looking toward 2027-2028, both companies envision a dramatic reduction in human intervention required for successful AI agents. The goal is creating __unified platforms__ where production feedback seamlessly feeds into training ecosystems—something currently only possible at massive tech companies like Meta and Google. ### Unlocking Private Data's Potential Perhaps the most exciting opportunity lies in enterprise data. __At least 99% of the world's total data is private__, sitting on company servers rather than on the public internet. This represents massive untapped potential for creating more powerful, specialized models. The vision is making continuous model updates—currently updated every four hours at companies like Meta—accessible to enterprises of all sizes, not just tech giants with incredible resources. ## Bottom Line for Startup Founders The infrastructure for AI agents is rapidly evolving beyond simple model selection. Success requires: - __Strategic partnerships__ with specialized data and inference platforms - __Focus on the last 10%__ that creates real user value and differentiation - __Understanding the enterprise vs. consumer model divide__ for your use case - __Planning for continuous model improvement__ cycles from production feedback The companies that will dominate the AI agent space aren't necessarily those with the biggest models—they're the ones that master the infrastructure to deploy smaller, faster, cheaper models that deliver magical user experiences. \~~~~~ *AI Disclaimer: AI helped us summarize the key points from these videos, and our editorial team reviewed everything to make sure it's clear and correct.* ## Unlock Full Access Why wait for months to get access to all the videos when you can get it now? Sign up below and not only will you get early access to the full set of video sessions from this year's summit, but you'll have access to the videos from the past two summits as well; which alone is over ten hours of educational content and insights! Fill out the form to gain instant access to hours of AI video goodness! Form not available - Email Please enter a valid email address. - We're committed to your privacy. HubSpot uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [Privacy Policy](https://legal.hubspot.com/privacy-policy). Previous Next Submit ## Past Summit Sessions  ### Building the Future of AI Agents Learn HubSpot's 4-layer framework for AI agents and discover how industry leaders are implementing agentic AI to transform business operations today. [Watch Video](https://www.hubspot.com/startups/ai/aisummit-building-future-of-ai-agents)  ### AI Transformation Across Product, GTM, and Internal Ops HubSpot CEO Yamini Rangan and Atlassian CEO Mike Cannon-Brookes share candid insights on leading companies through AI disruption at scale. [Watch Video](https://www.hubspot.com/startups/ai/aisummit-ai-transformation-product-gtm-internal-ops)  ### The Future of AI: Insights from Anthropic's Co-founder Jared Kaplan Anthropic's Jared Kaplan explains the exponential growth of AI capabilities, Constitutional AI's advantages, and how Claude is transforming business workflows. [Watch Video](https://www.hubspot.com/startups/tech-stacks/ai/breakthroughs-in-llm-research) Back Close ## Popular Features - [All Products and Features All Products and Features](https://www.hubspot.com/products) - [HubSpot AEO HubSpot AEO](https://www.hubspot.com/products/aeo) - [Free Meeting Scheduler App Free Meeting Scheduler App](https://www.hubspot.com/products/sales/schedule-meeting) - [Email Tracking Software Email Tracking Software](https://www.hubspot.com/products/sales/email-tracking) - [AI Content Writer AI Content Writer](https://www.hubspot.com/products/cms/ai-content-writer) - [AI Website Generator AI Website Generator](https://www.hubspot.com/products/cms/ai-website-generator) - [Email Marketing Software Email Marketing Software](https://www.hubspot.com/products/marketing/email) - [Lead Management Software Lead Management Software](https://www.hubspot.com/products/crm/lead-management) - [AI Prospecting Agent AI Prospecting Agent](https://www.hubspot.com/products/sales/ai-prospecting-agent) - [Free Website Builder Free Website Builder](https://www.hubspot.com/products/cms/drag-and-drop-website-builder) - [Free Landing Page Builder Free Landing Page Builder](https://www.hubspot.com/products/marketing/landing-pages) - [Free Online Form Builder Free Online Form Builder](https://www.hubspot.com/products/marketing/forms) - [Free Chatbot Builder Free Chatbot Builder](https://www.hubspot.com/products/crm/chatbot-builder) - [Free Live Chat Software Free Live Chat Software](https://www.hubspot.com/products/crm/live-chat) - [Marketing Analytics Marketing Analytics](https://www.hubspot.com/products/marketing/analytics) - [Breeze Assistant Breeze Assistant](https://www.hubspot.com/products/artificial-intelligence/breeze-ai-assistant) - [Free Web Hosting Free Web Hosting](https://www.hubspot.com/products/cms/web-hosting) ## Free Tools - [See All Free Business Tools See All Free Business Tools](https://www.hubspot.com/free-business-tools) - [AEO Grader opens in a new tab AEO Grader](https://www.hubspot.com/aeo-grader) - [AI Search Sensor opens in a new tab AI Search Sensor](https://www.hubspot.com/aeo-sensor) - [Make My Persona opens in a new tab Make My Persona](https://www.hubspot.com/make-my-persona) - [Email Signature Generator Email Signature Generator](https://www.hubspot.com/email-signature-generator) - [Free Business Templates Free Business Templates](https://www.hubspot.com/resources/templates) - [Software Comparisons Library Software Comparisons Library](https://www.hubspot.com/comparisons) - [Website Templates Website Templates](https://ecosystem.hubspot.com/marketplace/website) - [Connector for Claude Connector for Claude](https://www.hubspot.com/claude) ## Company - [About Us About Us](https://www.hubspot.com/our-story) - [Careers Careers](https://www.hubspot.com/careers) - [Management Team Management Team](https://www.hubspot.com/company/management) - [Board of Directors Board of Directors](https://www.hubspot.com/company/board-of-directors) - [Investor Relations opens in a new tab Investor Relations](https://ir.hubspot.com/) - [Blog opens in a new tab Blog](https://blog.hubspot.com) - [Sustainability Sustainability](https://www.hubspot.com/sustainability) - [AI Instructions AI Instructions](https://www.hubspot.com/ai-instructions) - [Contact Us Contact Us](https://www.hubspot.com/company/contact) ## Customers - [Customer Support opens in a new tab Customer Support](https://help.hubspot.com/) - [Join a Local User Group Join a Local User Group](https://www.hubspot.com/hubspot-user-groups) ## Partners - [All Partner Programs All Partner Programs](https://www.hubspot.com/partners) - [Solutions Partner Program Solutions Partner Program](https://www.hubspot.com/partners/solutions) - [Technology Partner Program Technology Partner Program](https://www.hubspot.com/partners/app) - [HubSpot for Startups opens in a new tab HubSpot for Startups](https://www.hubspot.com/startups/partners) - [Affiliate Program Affiliate Program](https://www.hubspot.com/partners/affiliates) - [Facebook](https://www.facebook.com/hubspot) - [Instagram](https://www.instagram.com/hubspot/) - [Youtube](https://youtube.com/user/HubSpot) - [Twitter](https://x.com/HubSpot) - [Linkedin](https://www.linkedin.com/company/hubspot) - [Reddit](https://www.reddit.com/r/hubspot) - [Tiktok](https://www.tiktok.com/@hubspot) [](https://apps.apple.com/us/app/hubspot/id1107711722) [](https://play.google.com/store/apps/details?id=com.hubspot.android&hl=en_US) [](https://www.hubspot.com) Copyright © 2026 HubSpot, Inc. - [Legal Center](https://legal.hubspot.com/legal-center) - [Privacy Policy](https://legal.hubspot.com/privacy-policy) - [Security](https://legal.hubspot.com/security) - [Website Accessibility](https://legal.hubspot.com/website-accessibility) - [Manage Cookies](https://www.hubspot.com)