Chatbots, Agents, Copilots, and Robots: The Future of AI Agents
Discover insights from AI leaders at Glean, Perplexity, LlamaIndex, and Together.ai on defining AI agents, building frameworks, and what's next for agent technology.
Chatbots, Agents, Copilots, and Robots: The Future of AI Agents
Discover insights from AI leaders at Glean, Perplexity, LlamaIndex, and Together.ai on defining AI agents, building frameworks, and what's next for agent technology.
In this panel of the HubSpot for Startups annual AI Summit in San Francisco, we explored the rapidly evolving world of AI agents. From the chatbots and copilots we interact with daily to emerging autonomous agents and robots that will transform how we live and work, this session provided valuable insights into where AI technology is heading.
Moderated by JC Mao, Distinguished Fellow and Venture Partner at Fellows Fund (ex-Google and ex-Microsoft), the panel featured an impressive lineup of industry experts:
Tamar Yehoshua, President of Product and Technology at Glean
Denis Yarats, Cofounder and CTO at Perplexity
Laurie Voss, VP Developer Relations at LlamaIndex
Tri Dao, Chief Scientist at Together.ai
Defining AI Agents
The panel kicked off by addressing a fundamental question: What exactly is an AI agent? As Tamar Yehoshua pointed out, the definition has become increasingly blurry as more companies label their products as "agents."
According to a definition pulled from Perplexity itself, an agent is something that:
Takes input and gathers information through sensors
Makes decisions based on that information
Takes autonomous actions
Is decisive in its execution
The panelists shared their favorite AI agents, highlighting both consumer and enterprise applications:
Yehoshua mentioned Google Assistant as an early example of an agent that can handle multi-turn tasks
Yarats (unsurprisingly) favored Perplexity's search agent with its multi-step reasoning capabilities
Voss highlighted an accessibility agent that reads web pages aloud while summarizing content and offering tool-based interactions
Dao pointed to Devin from Cognition Labs, an agent that functions like a junior software developer
Building Successful AI Agents
Both Glean and Perplexity shared their experiences building successful AI agent applications, highlighting key lessons learned:
Perplexity's approach:
Started with the basics: providing accurate answers quickly
Invested heavily in detailed evaluation systems
Built tools to understand where the system has blind spots
Maintained a comprehensive overview of system performance
Glean's enterprise challenges:
Security and privacy are paramount when dealing with enterprise data
Built a sophisticated retrieval engine and knowledge graph
Focused on understanding what signals determine relevance
Preserved permissions and security across various tools
Had to combat the non-deterministic nature of LLMs for enterprise settings where consistency is expected
Infrastructure for AI Agent Development
LlamaIndex and Together.ai discussed how their infrastructure platforms support AI agent developers:
Together.ai's infrastructure priorities:
Fast and robust model serving for low latency and high throughput
Support for long context windows, especially for multimodal content
Memory management for agents that need to access documents or maintain a scratch pad
LlamaIndex's framework approach:
Originally focused on retrieval augmented generation (RAG)
Discovered that complex retrieval strategies become increasingly agentic
Absorbs best practices from industry research
Provides developers with a menu of agent-building strategies
Especially useful for creating research assistants in fields with large amounts of unstructured data
Future Developments and Breakthroughs
The panelists shared their expectations for future AI agent breakthroughs:
Denis Yarats: Replacing auto-regressive generation with planning and reasoning over search
Tamar Yehoshua: Improved reliability with self-reflection capabilities for multi-turn agents
Laurie Voss: LLMs that can flawlessly write SQL queries to unlock structured data in databases
Tri Dao: Enhanced long context capabilities, with models handling millions of tokens
The panel noted that Google has already announced support for a 2-million token context window, with internal testing reaching 10 million tokens. This expanded context capability will enable developers to create agents that can process entire codebases, lengthy documents, and multimedia content.
Competing in a Crowded AI Marketplace
As foundation models continue to improve, the panelists discussed strategies for startups to compete with tech giants:
Do something against the grain - Glean focuses on horizontal integration across all platforms, something big companies are not incentivized to do
Move with greater velocity - Startups can pivot quickly and maintain tight feedback loops with customers
Focus on mission over competition - Build for the future while taking advantage of inefficiencies in larger companies
Embrace open-source approaches - LlamaIndex integrates with new technologies as they emerge
Lower barriers to entry - Together.ai emphasized how easy it is now to build AI demos without deep technical expertise
Mitigating Risks and Building Safeguards
The panel acknowledged that with great capabilities come significant risks:
Perplexity focuses on providing accurate, trustworthy information by investing heavily in verification and evaluation
Glean emphasizes permission-aware security to respect enterprise data hierarchies
Together.ai recommends using separate guardrail models (like Meta's Llama Guard) to check outputs
Final Thoughts on the Future
The panel concluded with excitement about AI agents' potential:
Automation will remove friction and drudgery from daily tasks
The industry is just getting started - today's models will seem primitive in a few years
Even modest improvements in foundation models will make agents truly reliable and capable
The barrier to entry continues to lower, making AI development accessible to more creators
AI Disclaimer: The insights shared in this video or audio were initially distilled through advanced AI summarization technologies, with subsequent refinements made by the writer and our editorial team to ensure clarity and veracity.
Full AI Summit Library
Would you like full access to the complete AI Summit video library, featuring over ten hours of educational content and insights? Click below.