Artificial Intelligence

The Shift from Prompts to Autonomous Systems

For years, organizations have focused on mastering “prompt engineering”, the art of writing precise instructions to extract useful outputs from Large Language Models (LLMs). While highly effective for simple, singular tasks, the prompt-based approach has inherent limitations when faced with complex, multi-step business problems.

The next paradigm shift in enterprise AI is the move toward Agentic Workflows.

An “Agent” is more than just an LLM. It is an autonomous or semi-autonomous system that combines reasoning capability (the LLM) with access to tools, memory, and the ability to act on its environment. Instead of answering a question, an agent performs a role, acting as an analyst, a software engineer, or a project manager, handling sequential professional tasks until a goal is achieved.

Why Agents Require Orchestration

The premise of agentic workflows is powerful, but deployment is difficult. In a complex scenario, you may need a system to:

1. Analyze a business request.
2. Search a database.
3. Process results.
4. Consult a second specialized agent (e.g., a “Coder Agent”).
5. Revise the plan based on output and finally provide a summary.

Without proper coordination, this series of steps breaks down. The model might hallucinate a tool execution, forget crucial data from step one by step four, or enter an endless loop of unhelpful actions.

Orchestration is the framework that manages this complexity. It is the conductor of the agentic orchestra, defining how different agents, tools, and memory systems interact, ensuring reliability, traceability, and successful execution of the business objective.

Anatomy of an Agentic Stack

To build a reliable orchestrator for autonomous systems, your architecture must unite three fundamental components:

• Intelligence Layer (The Brain): The reasoning core, usually an LLM, capable of taking input, breaking it into smaller tasks, and evaluating progress.
• Action Layer (The Tools): A library of external integrations, such as databases, web scrapers, computational engines, and business APIs, that the agent can use to gather real-world data or execute actions.
• Coordination Layer (The Orchestrator): The logic that manages state, standardizes how agents exchange data, handles errors, and ensures loops are terminated when goals are met.

Tools of the Trade: Navigating the Lang Ecosystem

As organizations move from proof-of-concept to production, the ecosystem of framework tools is rapidly evolving. The “Lang” suite has emerged as a particularly dominant force in defining how agents are built and orchestrated. During our workshop, we will explore several critical tools within this stack:

LangChain

While often used for simple prompt channelling, LangChain’s core contribution to agentic architecture is standardizing integration and chain creation. It provides the interface to connect the LLM to dozens of external systems. Crucially, it allows us to define custom “tools” for the agent. These are specialized, user-created functions that give the agent specific capabilities, such as querying a proprietary data warehouse or executing an internal Python script. By wrapping these functions in LangChain’s tool abstraction, the agent can autonomously decide when and how to invoke them to solve complex problems.

LangGraph

Managing complex agentic workflows required a different mental model: graphs. LangGraph extends LangChain by allowing developers to model agentic flows as stateful graphs (DAGs, or Directed Acyclic Graphs). This is crucial for systems that require robust loops, cyclical processes, and complex state management, ensuring that “Agent A” always knows what state “Agent B” left the system in.

Langfuse

Orchestrating agents is messy, and you need visibility. While not officially developed by the creators of LangChain, Langfuse is an essential open-source operational companion that integrates seamlessly with the ecosystem. It provides a robust platform for debugging, testing, and monitoring agentic systems without vendor lock-in. Langfuse allows teams to “trace” the entire multi-step process, viewing every prompt, tool call, and internal decision, making it possible to identify bottlenecks, reduce costs, and debug failures in production.

Complementary Orchestration Tools

While the Lang ecosystem excels at managing LLM logic, a true enterprise solution often requires integration with generalized orchestration and automation tools (like Zapier or n8n). These tools excel at managing event triggers, parallel processes, and standard API interactions that do not require LLM reasoning, complementing the Lang stack in a complete enterprise architecture.

Final Thoughts

Moving from single prompts to coordinated, agentic systems is a necessary evolutionary step for organizations aiming to unlock true operational efficiency with AI. Mastery of these systems requires shifting your perspective from “engineering a prompt” to “engineering a system.”

Want to see how this works in practice?

This article provides a conceptual blueprint of agentic workflows and the essential role of orchestration. To gain hands-on experience in building these systems, we invite you to join our upcoming webinar on the Orchestration of Agentic Workflows. During the session, we will demonstrate how to build multi-step autonomous systems by integrating these platforms into a single architecture, providing a practical guide for moving from simple prompts to coordinated AI systems that handle professional tasks.

Register for free

– Hernan Revale (Scalefree)

From BI to AI: Operationalizing Predictive Analytics where your Data already lives

Traditional Business Intelligence and reporting are incredibly good at telling what happened yesterday. How much revenue was generated last quarter? How many users logged in this week? But while understanding the past is important, today’s businesses need to know what will happen tomorrow.

This is where Predictive Analytics comes in. At its core, predictive analytics simply uses historical data to forecast future outcomes. Instead of asking how many customers canceled last month, predictive analytics asks:

“Which specific customers are most likely to cancel next week?”

Many organizations understand this value and eagerly hire data scientists to build these models. Yet, time and time again, these predictive initiatives fail to make it out of the PoC phase and into daily business operations because of how teams and data architectures are fundamentally structured.

The Problem: The “Two Silos” of Data

In many companies, Data Engineering and Data Science exist in two entirely different worlds.

Data Engineers and Data Warehouse Developers spend their days building the Modern Data Platform. They carefully extract, clean, conform, and govern data from dozens of sources to create a “Single Source of Truth.” When a business analyst looks at a revenue dashboard, they know they can trust the numbers because the data platform enforces strict business logic.

Data Scientists, on the other hand, often work in isolated environments like standalone Jupyter notebooks. Because they need massive amounts of data to train their machine learning models, they often bypass the data platform entirely, pulling raw, unstructured data directly from a data lake.

This disconnect creates some challenges:

• Duplicated Effort: Data Scientists waste up to 80% of their time cleaning and prepping raw data, work the Data Engineering team has already done in the platform
• Inconsistent Metrics: Because models are built on raw data, a model’s definition of “Active Customer” might completely contradict the official definition used by the in the data platform
• The “Wall of Production”: A model might look perfectly accurate on a data scientist’s laptop, but because it relies on disconnected, ungoverned data pipelines, integrating its predictions back into the daily workflows of sales or support teams becomes an IT nightmare
• Exclusivity: Data analysts are often limited to classic descriptive analytics which slows time-to-insight. The optimal solution is to democratize data science, empowering analysts to implement predictive use cases directly.

The Solution: Bring the Machine Learning to the Data

The fix to this problem requires a fundamental shift in how we think about machine learning architecture. Instead of moving data out of governed systems to feed external ML models, we need to bring the ML workflows more closer to the data.

By positioning the Modern Data Platform as the foundation for predictive analytics, you ensure that every prediction is built on the same trusted, cleansed, and governed business data used for your daily reporting. The Data Platform becomes the Feature Store, a centralized hub where data is prepared once and used everywhere, whether for a BI dashboard or training a predictive model.

When the data platform serves as the single source of truth for both analysts and algorithms, magic happens. Data science teams stop wrestling with raw data pipelines, data engineering teams maintain governance, and the business gets predictions they can actually trust and operationalize.

Two Architecture Approaches

So, how do we actually bring the machine learning to the data? There isn’t a one-size-fits-all answer. Depending on the team’s skillset and the complexity of the models, there are typically one of two foundational patterns adopted:

Pattern 1: In-Warehouse Machine Learning (Democratizing ML)

Modern cloud data platforms have evolved beyond just storing and querying data, many now have machine learning engines built directly into them.

• How it works: Using standard SQL, Data Analysts and Analytics Engineers can train, evaluate, and deploy models entirely inside the data platform (for example BigQuery ML, Snowflake Cortex or Databricks)
• The Benefit: This radically democratizes predictive analytics. You don’t need to know Python or manage complex infrastructure to build a model. If you know SQL, you can generate predictions using the exact same tables you use for your BI dashboards
• The Trade-off: While perfect for standard tasks like regression or classification, you are limited to the specific algorithms supported by the data platform

Pattern 2: The Data Platform as a Feature Store (The Hybrid Approach)

For organizations with dedicated Data Science teams building highly complex or custom models, the data platform takes on a different role: the Feature Store.

• How it works: Data Scientists continue to work in their preferred external ML platforms (like Vertex AI, Databricks, or other). However, instead of pulling messy data from a data lake, they connect directly to the data platform to pull curated, business-approved data (“features”) for training
• The Benefit: Data Scientists retain maximum flexibility to use advanced Python libraries and deep learning frameworks, while ensuring the models are trained on governed, accurate data
• The Trade-off: It requires a bit more orchestration to manage the pipeline between the data platform and the ML platform, and ensuring predictions are accurately written back to the data platform

Example: Predicting Customer Churn

To understand the benefits of these approaches, let’s look at a classic business challenge: Customer Churn Prevention.

Imagine a SaaS company trying to figure out which customers are likely to cancel their subscriptions. In a siloed environment, predicting this is can be a messy, manual science project. But on a modern data platform, it becomes an automated operational workflow:

1. The Foundation (Data): Because of the Data Engineering team’s work, the data platform already contains all necessary historical information of the customer. CRM data (company size), financial records (billing history), product logs (login frequency), and Zendesk tickets (recent complaints) are all cleaned, joined, and sitting in governed tables including a full history of changes
2. The Prediction (Modeling): An analyst uses In-Warehouse ML (Pattern 1) to run a classification model against this historical data. The model identifies the hidden patterns of a churning customer and generates a “Churn Risk Score” between 0 and 100 for every active user
3. The Operationalization (Action): This is the crucial step. The predictions aren’t left in a notebook. The risk scores are written directly back into a new table in the data platform. Through reverse-ETL, these scores can be automatically synced to the CRM as well as dashboards and reports can easily be built on top of the results.

Conclusion

Predictive analytics shouldn’t be an isolated science experiment. It should be a living, breathing part of your operational reality. By treating your modern data platform as the foundation for your machine learning workflows, you eliminate data silos, empower your analysts, and ensure your predictions are built on the trusted business data that matters most.

It is time to operationalize predictive insights where your business data already lives.

Want to see how this works in practice?

Join our upcoming webinar: Predictive Analytics on the Modern Data Platform. We will explore how to build and run predictive analytics directly on top of your data platform using trusted, governed business data as the foundation. You’ll learn practical patterns for turning warehouse models into features, training and deploying predictions, and integrating results back into reporting and operational workflows. Join me on March 17th.

Register for free

– Ole Bause (Scalefree)

From BI to AI: Operationalizing Predictive Analytics where your Data already lives

Traditional Business Intelligence and reporting are incredibly good at telling what happened yesterday. How much revenue was generated last quarter? How many users logged in this week? But while understanding the past is important, today’s businesses need to know what will happen tomorrow.

This is where Predictive Analytics comes in. At its core, predictive analytics simply uses historical data to forecast future outcomes. Instead of asking how many customers canceled last month, predictive analytics asks:

“Which specific customers are most likely to cancel next week?”

Many organizations understand this value and eagerly hire data scientists to build these models. Yet, time and time again, these predictive initiatives fail to make it out of the PoC phase and into daily business operations because of how teams and data architectures are fundamentally structured.

The Problem: The “Two Silos” of Data

In many companies, Data Engineering and Data Science exist in two entirely different worlds.

Data Engineers and Data Warehouse Developers spend their days building the Modern Data Platform. They carefully extract, clean, conform, and govern data from dozens of sources to create a “Single Source of Truth.” When a business analyst looks at a revenue dashboard, they know they can trust the numbers because the data platform enforces strict business logic.

Data Scientists, on the other hand, often work in isolated environments like standalone Jupyter notebooks. Because they need massive amounts of data to train their machine learning models, they often bypass the data platform entirely, pulling raw, unstructured data directly from a data lake.

This disconnect creates some challenges:

• Duplicated Effort: Data Scientists waste up to 80% of their time cleaning and prepping raw data, work the Data Engineering team has already done in the platform
• Inconsistent Metrics: Because models are built on raw data, a model’s definition of “Active Customer” might completely contradict the official definition used by the in the data platform
• The “Wall of Production”: A model might look perfectly accurate on a data scientist’s laptop, but because it relies on disconnected, ungoverned data pipelines, integrating its predictions back into the daily workflows of sales or support teams becomes an IT nightmare
• Exclusivity: Data analysts are often limited to classic descriptive analytics which slows time-to-insight. The optimal solution is to democratize data science, empowering analysts to implement predictive use cases directly.

The Solution: Bring the Machine Learning to the Data

The fix to this problem requires a fundamental shift in how we think about machine learning architecture. Instead of moving data out of governed systems to feed external ML models, we need to bring the ML workflows more closer to the data.

By positioning the Modern Data Platform as the foundation for predictive analytics, you ensure that every prediction is built on the same trusted, cleansed, and governed business data used for your daily reporting. The Data Platform becomes the Feature Store, a centralized hub where data is prepared once and used everywhere, whether for a BI dashboard or training a predictive model.

When the data platform serves as the single source of truth for both analysts and algorithms, magic happens. Data science teams stop wrestling with raw data pipelines, data engineering teams maintain governance, and the business gets predictions they can actually trust and operationalize.

Two Architecture Approaches

So, how do we actually bring the machine learning to the data? There isn’t a one-size-fits-all answer. Depending on the team’s skillset and the complexity of the models, there are typically one of two foundational patterns adopted:

Pattern 1: In-Warehouse Machine Learning (Democratizing ML)

Modern cloud data platforms have evolved beyond just storing and querying data, many now have machine learning engines built directly into them.

• How it works: Using standard SQL, Data Analysts and Analytics Engineers can train, evaluate, and deploy models entirely inside the data platform (for example BigQuery ML, Snowflake Cortex or Databricks)
• The Benefit: This radically democratizes predictive analytics. You don’t need to know Python or manage complex infrastructure to build a model. If you know SQL, you can generate predictions using the exact same tables you use for your BI dashboards
• The Trade-off: While perfect for standard tasks like regression or classification, you are limited to the specific algorithms supported by the data platform

Pattern 2: The Data Platform as a Feature Store (The Hybrid Approach)

For organizations with dedicated Data Science teams building highly complex or custom models, the data platform takes on a different role: the Feature Store.

• How it works: Data Scientists continue to work in their preferred external ML platforms (like Vertex AI, Databricks, or other). However, instead of pulling messy data from a data lake, they connect directly to the data platform to pull curated, business-approved data (“features”) for training
• The Benefit: Data Scientists retain maximum flexibility to use advanced Python libraries and deep learning frameworks, while ensuring the models are trained on governed, accurate data
• The Trade-off: It requires a bit more orchestration to manage the pipeline between the data platform and the ML platform, and ensuring predictions are accurately written back to the data platform

Example: Predicting Customer Churn

To understand the benefits of these approaches, let’s look at a classic business challenge: Customer Churn Prevention.

Imagine a SaaS company trying to figure out which customers are likely to cancel their subscriptions. In a siloed environment, predicting this is can be a messy, manual science project. But on a modern data platform, it becomes an automated operational workflow:

1. The Foundation (Data): Because of the Data Engineering team’s work, the data platform already contains all necessary historical information of the customer. CRM data (company size), financial records (billing history), product logs (login frequency), and Zendesk tickets (recent complaints) are all cleaned, joined, and sitting in governed tables including a full history of changes
2. The Prediction (Modeling): An analyst uses In-Warehouse ML (Pattern 1) to run a classification model against this historical data. The model identifies the hidden patterns of a churning customer and generates a “Churn Risk Score” between 0 and 100 for every active user
3. The Operationalization (Action): This is the crucial step. The predictions aren’t left in a notebook. The risk scores are written directly back into a new table in the data platform. Through reverse-ETL, these scores can be automatically synced to the CRM as well as dashboards and reports can easily be built on top of the results.

Conclusion

Predictive analytics shouldn’t be an isolated science experiment. It should be a living, breathing part of your operational reality. By treating your modern data platform as the foundation for your machine learning workflows, you eliminate data silos, empower your analysts, and ensure your predictions are built on the trusted business data that matters most.

It is time to operationalize predictive insights where your business data already lives.

Want to see how this works in practice?

Join our upcoming webinar: Predictive Analytics on the Modern Data Platform. We will explore how to build and run predictive analytics directly on top of your data platform using trusted, governed business data as the foundation. You’ll learn practical patterns for turning warehouse models into features, training and deploying predictions, and integrating results back into reporting and operational workflows. Join me on March 17th.

Register for free

– Ole Bause (Scalefree)

Not long ago, simple large language models were the pinnacle of AI. Today, they can feel almost rudimentary, as the domain of artificial intelligence is rapidly evolves. Lately, we are seeing a push trying to move beyond one-off prompts and towards AI agents. 

It only makes sense that businesses are eager to incorporate AI agents into their workflows, and one domain particularly primed for such transformation is the data team. AI agents can automate repetitive tasks, streamline operations, and enhance data analysis and allow data professionals to focus more on the business side.

AI Agents: A Brief Introduction

AI agents are autonomous software systems that perceive their environment, reason over data, and take actions to achieve specified goals. They leverage large language models, tool‑use frameworks, and API integrations to connect with external services from CRM platforms and cloud storage to data platforms and real‑time event streams. Unlike static models, agents can maintain memory across sessions, chain multiple model calls, and adapt their workflows based on real‑time feedback from connected systems.

The Anatomy of AI Agents

Figure 1: A conversation agent built in low-code automation tool n8n.

An AI agent is typically centered around a large language model that serves as its core reasoning engine, interpreting user inputs, generating plans, and orchestrating decision-making through chain-of-thought or self-prompting techniques​. Surrounding this core is a memory structure that can span across immediate working memory, episodic logs, and semantic knowledge stores that persistently captures and condenses interaction histories​. To provide durable, structured storage and enable symbolic multi-hop reasoning, agents integrate databases (e.g., SQL, graph, or vector stores) as their internal memory substrate, issuing queries to organize, link, and evolve knowledge beyond the context window of the LLM​. Finally, AI agents orchestrate a suite of external tools ranging from RESTful APIs and code execution environments to web scrapers and domain-specific plugins to act upon the world, extend their cognitive reach, and execute actions in both digital and physical domains​.

A key limitation of relying on custom APIs as connectors in an AI agent framework is scalability: as you add more agents, tools, and integrations, maintaining a separate API connection for every tool and action soon becomes unmanageable. That’s where MCPs come in.

Figure 2: A diagram showcasing Model Context Protocols (MCP)

Developed by Anthropic and open-sourced in November 2024, the Model Context Protocol (MCP) functions as a standardized integration layer that enables the reasoning engine to interface with external resources​. It accomplishes this by defining a uniform client–server protocol whereby MCP clients (the AI agents) discover available services via a registry, authenticate, and invoke capabilities such as database queries, function calls, or file retrieval through RESTful endpoints​. By decoupling the LLM from tool-specific protocols, MCP fosters a modular ecosystem in which new services can be plugged in dynamically, making AI agent development much more scalable.

Build a Solid Data Foundation for Agentic AI

Enterprises that aim to integrate AI agents into their data workloads must first build a solid data foundation. According to a cybersecurity report, 72% of professionals state that IT and security data are siloed within their organizations, creating corporate misalignment and increased security risks. Likewise, an industry study found that in three out of four companies, data silos hinder internal collaboration, and more than 40% report a growing number of such silos. 

When data remains in isolated, non-integrated environments, AI agents cannot establish a holistic overview of the data landscape of an enterprise, hence severely limiting its abilities in making meaningful impact.

Figure 3: An Enterprise Data Platform diagram, with an EDW

To overcome this, it is best to unify data sources into an enterprise data warehouse (EDW). The EDW must provide both current and historical data in a single data platform. By functioning as a true EDW, the data platform provides a single source of facts for all agents and analytics engines. This means that the AI agents across the enterprise are empowered to create what is needed with the increased availability of data. At Scalefree, we believe that a robust and well-designed data model is foundational to building a scalable and resilient EDW, that supports both operational efficiency and long-term analytical agility.

Ensure Data Quality and Metadata Management

Data quality is already a key issue in data warehousing. Poor data quality can lead to inaccurate insights, flawed decision-making, and ultimately compromise business success. The effectiveness of AI agents is also directly influenced by the quality of the data they consume. Issues such as duplicate records, missing values, and inconsistent schemas can result in erroneous behavior or reduced performance. These issues can be addressed through systematic data cleaning processes and the implementation of data quality tests across ingestion and transformation pipelines. Ongoing monitoring should be in place to detect anomalies and trigger remediation actions where necessary. 

Metadata management also plays a role in agent effectiveness. Shared taxonomies and ontologies provide agents with a consistent framework for understanding data definitions across domains. Without standardized metadata, agents may cause errors in reasoning or communication due to misinterpreted values. Establishing a well-maintained data catalog and promoting organization-wide metadata standards supports both data discoverability and semantic consistency, which are essential in multi-agent environments.

Prepare for Real-Time Processing and Efficient Retrieval

AI agents do not strictly require real-time data, but having access to it can significantly enhance their performance and decision-making capabilities. Real-time data allows AI agents to be informed in quickly changing conditions and provide more accurate and relevant responses. To support this, data platforms can be set up to process streaming data or near-real-time updates.

Additionally, indexing strategies must accommodate both structured and unstructured data. When needed, structured data can continue to rely on traditional indexing methods such as inverted indexes. For unstructured content, embedding-based vector search provides agents with the means to identify semantically similar data points.

Large data objects should also be broken into manageable segments through chunking. This practice enables agents to retrieve and reason over smaller, contextually meaningful portions of data, which improves both performance and interpretability. Determining appropriate chunk sizes may require tuning to balance context with precision.

Implement Orchestration and Observability for AI Workflows

The introduction of AI agents into business processes necessitates a layer of orchestration that governs how agents collaborate, pass information, and handle dependencies. A multi-agent orchestration system should trigger the right agents for a given task, coordinate their outputs, and manage error handling or fallback logic. Orchestrators also need to support asynchronous communication where agents operate independently but contribute to a shared goal.

Monitoring and testing these workflows is essential. Agents can fail, drift from intended behavior, or interact in unintended ways. Logging, alerting, and automated feedback loops can be integrated into orchestration frameworks to surface and correct such deviations. Performance metrics such as response time, accuracy, and success rates should be tracked to ensure continued alignment with business objectives.

AI Agents as Identity-Bearing Entities

AI agents should be treated as identity-bearing entities within the enterprise architecture. This means granting them access only to the data and systems necessary for their assigned roles. To that end, just as any other employee, AI agents should abide by the principle of least privilege. Role-Based Access Control ensures that each agent’s data permissions are explicitly defined and enforceable. For example, an AI agent responsible for financial forecasting should not have access to sensitive HR data.

Integrating AI agents into existing identity and access management (IAM) systems can help enforce compliance and support auditability. Just as human users have roles and access policies, agents should be provisioned, monitored, and offboarded in a controlled and traceable manner.

Embrace a Data Mesh for Scalable Multi-Agent Workflows

Organizations expecting to deploy multiple AI agents concurrently should consider transitioning from a centralized end-to-end model to a data mesh. A data mesh distributes data ownership across domain teams and treats data as a product, aligning well with the modular nature of AI agents. This architecture allows agents to scale horizontally across business functions while maintaining domain-specific ownership of data pipelines and logic. Each agent can operate on a defined domain without depending on a centralized data engineering team, reducing bottlenecks and increasing agility. In environments with high agent interaction, domain-driven decentralization ensures that systems remain responsive and maintainable as usage grows.

Design for Modularity and Scalability

To scale the use of AI agents across business processes, data pipelines should be decomposed into independently deployable and maintainable components. This approach allows new agents or features to be added without having to duplicate or fork existing systems. Event-driven architectures, in which agents react to messages or state changes, support this level of decoupling and flexibility.

Agent-to-agent communication should be standardized using standardized protocols and contracts to allow agent-to-agent interaction predictably. By designing systems with modular interfaces and reusable components, AI agent ecosystems can grow in an agile, iterative fashion.

Not long ago, simple large language models were the pinnacle of AI. Today, they can feel almost rudimentary, as the domain of artificial intelligence is rapidly evolves. Lately, we are seeing a push trying to move beyond one-off prompts and towards AI agents. 

It only makes sense that businesses are eager to incorporate AI agents into their workflows, and one domain particularly primed for such transformation is the data team. AI agents can automate repetitive tasks, streamline operations, and enhance data analysis and allow data professionals to focus more on the business side.

AI Agents: A Brief Introduction

AI agents are autonomous software systems that perceive their environment, reason over data, and take actions to achieve specified goals. They leverage large language models, tool‑use frameworks, and API integrations to connect with external services from CRM platforms and cloud storage to data platforms and real‑time event streams. Unlike static models, agents can maintain memory across sessions, chain multiple model calls, and adapt their workflows based on real‑time feedback from connected systems.

The Anatomy of AI Agents

Figure 1: A conversation agent built in low-code automation tool n8n.

An AI agent is typically centered around a large language model that serves as its core reasoning engine, interpreting user inputs, generating plans, and orchestrating decision-making through chain-of-thought or self-prompting techniques​. Surrounding this core is a memory structure that can span across immediate working memory, episodic logs, and semantic knowledge stores that persistently captures and condenses interaction histories​. To provide durable, structured storage and enable symbolic multi-hop reasoning, agents integrate databases (e.g., SQL, graph, or vector stores) as their internal memory substrate, issuing queries to organize, link, and evolve knowledge beyond the context window of the LLM​. Finally, AI agents orchestrate a suite of external tools ranging from RESTful APIs and code execution environments to web scrapers and domain-specific plugins to act upon the world, extend their cognitive reach, and execute actions in both digital and physical domains​.

A key limitation of relying on custom APIs as connectors in an AI agent framework is scalability: as you add more agents, tools, and integrations, maintaining a separate API connection for every tool and action soon becomes unmanageable. That’s where MCPs come in.

Figure 2: A diagram showcasing Model Context Protocols (MCP)

Developed by Anthropic and open-sourced in November 2024, the Model Context Protocol (MCP) functions as a standardized integration layer that enables the reasoning engine to interface with external resources​. It accomplishes this by defining a uniform client–server protocol whereby MCP clients (the AI agents) discover available services via a registry, authenticate, and invoke capabilities such as database queries, function calls, or file retrieval through RESTful endpoints​. By decoupling the LLM from tool-specific protocols, MCP fosters a modular ecosystem in which new services can be plugged in dynamically, making AI agent development much more scalable.

Build a Solid Data Foundation for Agentic AI

Enterprises that aim to integrate AI agents into their data workloads must first build a solid data foundation. According to a cybersecurity report, 72% of professionals state that IT and security data are siloed within their organizations, creating corporate misalignment and increased security risks. Likewise, an industry study found that in three out of four companies, data silos hinder internal collaboration, and more than 40% report a growing number of such silos. 

When data remains in isolated, non-integrated environments, AI agents cannot establish a holistic overview of the data landscape of an enterprise, hence severely limiting its abilities in making meaningful impact.

Figure 3: An Enterprise Data Platform diagram, with an EDW

To overcome this, it is best to unify data sources into an enterprise data warehouse (EDW). The EDW must provide both current and historical data in a single data platform. By functioning as a true EDW, the data platform provides a single source of facts for all agents and analytics engines. This means that the AI agents across the enterprise are empowered to create what is needed with the increased availability of data. At Scalefree, we believe that a robust and well-designed data model is foundational to building a scalable and resilient EDW, that supports both operational efficiency and long-term analytical agility.

Ensure Data Quality and Metadata Management

Data quality is already a key issue in data warehousing. Poor data quality can lead to inaccurate insights, flawed decision-making, and ultimately compromise business success. The effectiveness of AI agents is also directly influenced by the quality of the data they consume. Issues such as duplicate records, missing values, and inconsistent schemas can result in erroneous behavior or reduced performance. These issues can be addressed through systematic data cleaning processes and the implementation of data quality tests across ingestion and transformation pipelines. Ongoing monitoring should be in place to detect anomalies and trigger remediation actions where necessary. 

Metadata management also plays a role in agent effectiveness. Shared taxonomies and ontologies provide agents with a consistent framework for understanding data definitions across domains. Without standardized metadata, agents may cause errors in reasoning or communication due to misinterpreted values. Establishing a well-maintained data catalog and promoting organization-wide metadata standards supports both data discoverability and semantic consistency, which are essential in multi-agent environments.

Prepare for Real-Time Processing and Efficient Retrieval

AI agents do not strictly require real-time data, but having access to it can significantly enhance their performance and decision-making capabilities. Real-time data allows AI agents to be informed in quickly changing conditions and provide more accurate and relevant responses. To support this, data platforms can be set up to process streaming data or near-real-time updates.

Additionally, indexing strategies must accommodate both structured and unstructured data. When needed, structured data can continue to rely on traditional indexing methods such as inverted indexes. For unstructured content, embedding-based vector search provides agents with the means to identify semantically similar data points.

Large data objects should also be broken into manageable segments through chunking. This practice enables agents to retrieve and reason over smaller, contextually meaningful portions of data, which improves both performance and interpretability. Determining appropriate chunk sizes may require tuning to balance context with precision.

Implement Orchestration and Observability for AI Workflows

The introduction of AI agents into business processes necessitates a layer of orchestration that governs how agents collaborate, pass information, and handle dependencies. A multi-agent orchestration system should trigger the right agents for a given task, coordinate their outputs, and manage error handling or fallback logic. Orchestrators also need to support asynchronous communication where agents operate independently but contribute to a shared goal.

Monitoring and testing these workflows is essential. Agents can fail, drift from intended behavior, or interact in unintended ways. Logging, alerting, and automated feedback loops can be integrated into orchestration frameworks to surface and correct such deviations. Performance metrics such as response time, accuracy, and success rates should be tracked to ensure continued alignment with business objectives.

AI Agents as Identity-Bearing Entities

AI agents should be treated as identity-bearing entities within the enterprise architecture. This means granting them access only to the data and systems necessary for their assigned roles. To that end, just as any other employee, AI agents should abide by the principle of least privilege. Role-Based Access Control ensures that each agent’s data permissions are explicitly defined and enforceable. For example, an AI agent responsible for financial forecasting should not have access to sensitive HR data.

Integrating AI agents into existing identity and access management (IAM) systems can help enforce compliance and support auditability. Just as human users have roles and access policies, agents should be provisioned, monitored, and offboarded in a controlled and traceable manner.

Embrace a Data Mesh for Scalable Multi-Agent Workflows

Organizations expecting to deploy multiple AI agents concurrently should consider transitioning from a centralized end-to-end model to a data mesh. A data mesh distributes data ownership across domain teams and treats data as a product, aligning well with the modular nature of AI agents. This architecture allows agents to scale horizontally across business functions while maintaining domain-specific ownership of data pipelines and logic. Each agent can operate on a defined domain without depending on a centralized data engineering team, reducing bottlenecks and increasing agility. In environments with high agent interaction, domain-driven decentralization ensures that systems remain responsive and maintainable as usage grows.

Design for Modularity and Scalability

To scale the use of AI agents across business processes, data pipelines should be decomposed into independently deployable and maintainable components. This approach allows new agents or features to be added without having to duplicate or fork existing systems. Event-driven architectures, in which agents react to messages or state changes, support this level of decoupling and flexibility.

Agent-to-agent communication should be standardized using standardized protocols and contracts to allow agent-to-agent interaction predictably. By designing systems with modular interfaces and reusable components, AI agent ecosystems can grow in an agile, iterative fashion.