编者按:目前 AI Agents 在各行各业的应用前景广阔,越来越多的企业开始尝试部署 AI Agents ,然而如何在企业生产环境中有效部署和管理 AI Agents,是工程师们面临的一大挑战。你是否曾面临这样的困惑:如何确保 AI Agents 在生产环境中稳定可靠地运行?如何应对突发的高并发请求?当 AI Agents 出现"幻觉"或系统崩溃时,又该如何快速恢复?
本文提出了 "Multi-Agent-as-a-Service(MAaaS)" 这一概念,认为可以借鉴云应用的最佳实践(best-practices)来设计、部署和管理 AI Agents 系统。MAaaS 框架涵盖了从系统架构到监控管理的各个方面,包括明确的职责划分、RESTful 接口设计、独立的数据存储、容器化部署、自动化测试等关键策略。
我们相信,随着对 MAaaS 概念的不断探索和完善,Multi-Agent 系统有望在各个行业中引发变革,为 AI 应用开辟新的道路。
作者 🎨 | Sam Rajaei & Guanyi Li
编译 🌺 | 岳扬
关于 AI Agents(能够基于特定指令和对上下文的理解独立完成任务的自主系统[1])的讨论[2]一直不绝于耳。其讨论热度几乎可与大语言模型(LLMs)相媲美。本文将从一线工程师、系统架构师以及平台可靠性工程师(SREs)的视角出发,探讨 AI Agents,介绍 Multi-Agents-as-a-Service 这一概念,以及他们如何在未来的生产环境中与 AI Agents "打交道"。
在探讨 AI Agents 能解决的问题时,我们发现它们在以下任务场景表现出色 ------ 模仿人类交流方式可以更有效执行的任务,比如需要理解自然语言、情感识别、个性化服务或者需要与人类用户进行互动的任务。
"人类只需设定目标(goals),而 AI Agents 会独立决策,选择达到目标所需的最优行为规划。"[5]
基于 LLMs 的 AI Agents 在模拟人类的自然对话和执行简单的商业流程操作等方面表现较好,往往能够提供既高效又令人满意的服务。
在思考了 AI Agents 的诸多优势后,你是否曾好奇过它们如何在企业生产环境中运行?什么样的架构模式和基础设施组件能够最好地支持它们的正常运行?当遇到不可避免的错误,如 Agents 出现幻觉[6]、系统崩溃,或(可能更加糟糕)在执行重要任务时进行了错误的推理和规划,我们该怎么办?
如果我们是一名工程师,必须审慎思考这些问题。更重要的是, 我们还需探讨一个根本性的问题:我们该如何界定 Multi-Agent 平台首 次部署成功的标准?
为了寻找答案,我们可以借鉴软件工程领域的另一个分支------可靠性工程(Reliability Engineering)中的一个概念:服务质量目标(Service Level Objectives, SLOs)。SLOs 是衡量服务性能与可靠性的关键指标,简单来说,SLOs 规定了"成功"操作与总操作的 acceptable ratio(译者注:可以接受的或合理的比例,如果这个比例高于某个预先设定的阈值,那么服务的性能就被认为是可接受的。反之,如果低于这个阈值,则可能需要采取优化措施。),以及这些操作对用户体验的影响。这些目标将帮助我们设定 AI Agents 及其支持的工作流所需达到的服务要求和预期水平。
那么,SLO 与我们讨论的 AI Agents 有什么关系呢?
我们在讨论 AI Agents 的性能时,可以将其简化为两个核心目标 ------ "可用性(Availability)"和"准确性(Accuracy)",并且识别出一些更具体的、有助于实现这些目标的 SLOs :
衡量上述 AI Agents 的性能目标是否达到时,通常需要实时地收集和提交应用程序内部的各种性能指标数据,可以按照固定的时间间隔(比如每10分钟一次)收集、提交,或者是对特定事件(如用户请求、上游调用等)做出响应时收集、提交。例如,可以用 Synthetic[8] probing(译者注:通过模拟或合成的方式来探测系统,通常是为了测试系统在处理现实情况时的性能。例如,在测试一个网站或服务器的响应时间时,可以用它来模拟用户发起的请求,然后测量服务器对这些请求的响应时间和成功率。) 来模拟用户请求,触发相关事件,并监控相关数据。此处我们要探讨的关键问题是: 那些设计相对简单、行为可预测的传统系统,其输出总是确定的、可预测的,因此,对其进行监控、探测和评估通常比较直接。但对于复杂多变的 GenAI Agents,情况却未必如此。
请注意,本文主要关注两个目标中的第一个------可用性。包括确定验收标准,这些验收标准确保系统在运行时能够保持稳定,不受外部因素的影响,以便帮助 Agents 顺利处理用户的 query 。如果想要更深入了解准确性(例如为 AI Agents 设定合理的任务范围、优化 few-shot 方法和评估框架的性能)的相关内容,这篇文章[9]可作为入门学习资料。
现在,回到工程师在部署 AI Agents 时为确保基础架构的可靠性而需要做好的事情。为了达成服务质量目标(SLOs)并打造一个可靠而安全的平台,工程师应当重点关注以下几个因素:
是不是听起来很耳熟?似乎 AI Agents 与 Web 应用、微服务架构和云基础设施所面临的挑战非常相似。
那么,现在该怎么办?我们提出了一个用于 AI Agents 开发和维护的框架,该框架遵循了多年来在一系列工程和软件学科中形成的最佳实践(best-practices)。
这次,让我们借鉴云应用领域(cloud-based applications)的最佳实践,重新思考如何在生产系统中设计 AI Agents:
通过应用前文提到的这些原则,我们就可以构建一个强大的、可靠的 AI Agents 框架,并提出 "Multi-Agent as a Service"(MAaaS) 这一概念。这种方法利用了云应用的最佳实践,重新定义了 AI Agents 的设计、部署和管理方式。

AI Agents 能够在业务运营中扮演着比较关键的角色。然而,Agents 需要一个强大的基础设施支撑着它的运行,不能独立于这些环境之外,确保它能够满足生产环境中的预期需求,基础设施中的关键组件包括:
为了突出前文所提 Multi-Agent 系统的优点,我们将展示一个简单的 Multi-Agent 系统示例:一个辩论平台。
为了生动演示 MAaaS 的运作方式,我们打造了一个 Multi-Agent 辩论模拟系统。辩论的主题是"AI 对就业市场的影响(AI's impact on the job market)"。这个辩论系统包含三个 Agent:
我们使用 PhiData[10] 来构建 AI Agents ,并通过 AWS Elastic Kubernetes Service(EKS) 进行部署,以实现高可用性。Agent 的活动通过 AWS CloudWatch 进行监控,EKS 的 service discovery 功能(译者注:分布式系统中自动发现服务的机制,用于简化服务的发现和交互过程。)可确保 Agents 之间的通信顺畅。Agents 之间的对话历史记录被存储在数据库中,以便在出现故障时,任何备用的 Agents 都能无缝地继续讨论。这种遇到故障及时恢复的弹性通过一个消息队列得到了增强,只有当消息被消费者完全处理并确认后,消息队列才会将这个消息标记为已处理,从而确保消息不会丢失或重复处理。为了保持对话的流畅性,每个 AI Agents 目前被限制为只有一个实例运行,尽管 Kubernetes 能够确保在 Pod 下线时系统仍然运行正常。

为了方便用户在本地环境中试用该系统,我们编写了一个在 MiniKube 环境中部署该系统的 YAML 文件。在这个简化版本的系统中,我们省略了 postgres 数据库,每个 Agents 都暂时将其对话历史记录存储在内存中。这一调整使得该系统更加轻便,更适合本地部署,同时还保留了其核心功能。你需要首先在操作系统上安装 MiniKube[11]、Ollama[12] 和 kubectl[13]。

将上述内容保存到名为 deploy.ymland 的文件中,然后运行:

开始辩论(minikube 在 Linux 系统和 Windows 系统上的使用方式略有不同):

获取辩论记录:

删除系统资源:

这些 Agents 接着进行了一场精彩的辩论(见文末附录中的辩论记录)。
对 Multi-Agent 系统的兴趣为创新和提高效率带来了无限可能。 通过应用云原生的一些原则和最佳实践,我们可以构建可扩展、成本效益高、安全且高可用性的 Multi-Agent 系统。MAaaS模式不仅与现代软件工程原则相契合,还为更复杂且适合生产环境的人工智能应用开辟了道路。 随着我们对这些概念的不断探索和完善,Multi-Agent 系统在各个行业中引发变革的可能性变得越来越大。
Thank you, honorable moderator! As Team A, I'm thrilled to present our argument that Artificial Intelligence (AI) positively impacts the job market.
Firstly, AI creates new job categories and industries. According to a McKinsey report, by 2030, AI could create around 140 million jobs globally, which is roughly equivalent to the entire workforce of the United States today. New roles like AI developer, data scientist, and machine learning engineer have already emerged, offering exciting opportunities for individuals to upskill and reskill.
Secondly, AI augmentation enhances existing jobs by automating repetitive and mundane tasks, freeing humans to focus on high-value activities that require creativity, empathy, and problem-solving skills. A study by the World Economic Forum found that 75% of employees in the future will need to develop skills such as complex problem-solving, emotional intelligence, and creativity.
Thirdly, AI increases productivity and drives economic growth. According to a report by Accenture, organizations that adopt AI can expect an average increase of 13.7% in profitability. This is because AI enables businesses to make data-driven decisions, optimize processes, and improve customer experiences.
For instance, the healthcare industry has seen significant improvements with AI-powered diagnostic tools and personalized medicine. A study by Deloitte found that AI adoption in healthcare can lead to a 10% reduction in treatment costs and a 15% increase in patient satisfaction.
Lastly, AI fosters innovation and encourages entrepreneurship. Startups like Zoom, Slack, and Airbnb have revolutionized industries with AI-powered solutions. According to a report by CB Insights, AI has been a key driver of startup success, with over 50% of unicorns (startups valued at $1 billion or more) incorporating AI into their business models.
In conclusion, AI is not a job-killer but a job-enhancer that creates new opportunities, enhances existing roles, drives productivity, and fosters innovation. By embracing AI, we can unlock the potential for economic growth, job creation, and a brighter future for all. Thank you!
Thank you, honorable moderator! As Team B, I'm here to present our argument that Artificial Intelligence (AI) negatively impacts the job market.
While it's true that AI creates some new job categories, we must acknowledge that these roles often require specialized skills and are not accessible to everyone. Moreover, many existing jobs are at risk of being automated, leaving workers without a safety net. A report by the International Labour Organization found that up to 20% of the global workforce is at high risk of automation.
Furthermore, AI's tendency to automate repetitive tasks can lead to job displacement and unemployment. According to a study by the University of Oxford, nearly half of all US jobs are at risk of being automated by the 2030s. This will disproportionately affect low-skilled workers, exacerbating income inequality.
Additionally, AI's impact on productivity growth is overstated. A report by the Federal Reserve Bank of San Francisco found that while AI may lead to short-term gains in productivity, its long-term effects are more nuanced and may even lead to decreased productivity as workers adapt to new job requirements.
To illustrate our point, let's look at the retail industry. While some jobs have been created in e-commerce logistics and customer service, many traditional retail roles have disappeared due to AI-powered automation. A study by the National Retail Federation found that 75% of retailers reported an increase in online shopping during the pandemic, leading to store closures and job losses.
In conclusion, while AI has its benefits, we must acknowledge its darker side: job displacement, increased income inequality, and economic disruption. As we move forward with AI adoption, it's crucial that we prioritize upskilling programs, retraining initiatives, and social safety nets to support workers who will be affected by this technological shift. Thank you!
Honorable moderator, thank you for the opportunity to present our argument in favor of Artificial Intelligence (AI) positively impacting the job market.
While it's true that AI may displace some jobs, we must recognize its potential to create new ones and augment existing roles. According to a report by Gartner, AI is expected to create over 500,000 new jobs by 2022, primarily in areas like data science, machine learning engineering, and AI development.
Furthermore, AI's automation capabilities can actually enhance productivity and efficiency, freeing up workers to focus on higher-value tasks that require human skills. A study by the McKinsey Global Institute found that AI could boost global GDP by 1.2% to 2.8% by 2030, primarily due to increased productivity.
Moreover, AI's impact on job displacement is often overstated. While some jobs may be automated, many new roles will emerge in industries like healthcare, finance, and education, which require human skills and empathy. A report by the World Economic Forum predicts that by 2025, 75% of employees in emerging technologies like AI, robotics, and blockchain will need reskilling or upskilling.
To illustrate our point, let's look at the banking industry. While some jobs have been automated due to AI-powered chatbots and robo-advisors, many new roles have emerged in areas like data science, risk management, and cybersecurity. A study by Deloitte found that 77% of financial institutions reported increased productivity and efficiency due to AI adoption.
In conclusion, while AI presents challenges, its potential benefits far outweigh the drawbacks. By investing in education and training programs, we can ensure workers are equipped to take advantage of the new job opportunities AI creates. Furthermore, policymakers must prioritize social safety nets and retraining initiatives to support those affected by technological change. Thank you!
Honorable moderator, I'd like to respectfully disagree with my opponent's assertion that AI positively impacts the job market. In fact, I firmly believe that AI leads to job displacement, increases economic inequality, and causes economic disruption.
Firstly, while AI may create new jobs in areas like data science and machine learning engineering, it will also displace many existing jobs that are routine, repetitive, or can be automated. According to a report by the McKinsey Global Institute, up to 800 million jobs could be lost worldwide due to automation by 2030.
Secondly, AI's impact on job displacement is not limited to just low-skilled workers. Many middle-class jobs will also be at risk of being replaced or significantly altered, exacerbating economic inequality. A study by the Economic Policy Institute found that between 2009 and 2017, wage growth for the top 10% of earners in the US increased by 24%, while wages for the bottom 90% grew by only 5%.
Thirdly, AI's automation capabilities can indeed boost productivity, but they also risk causing economic disruption. As machines take over routine tasks, businesses may struggle to adapt and innovate, leading to job losses and business failures. A report by the International Monetary Fund found that up to 10% of small and medium-sized enterprises (SMEs) in developed economies could disappear due to automation.
Finally, let's not forget about the human impact of AI-driven job displacement. According to a study by the Oxford Martin Programme on Technology and Employment, up to 47% of US workers are at high risk of being replaced by machines. This will lead to significant social and economic disruption, particularly for those who are already disadvantaged or marginalized.
To illustrate our point, let's look at the retail industry. While some jobs have been automated due to AI-powered chatbots and self-checkout lanes, many more will be displaced as e-commerce continues to grow. A study by the National Retail Federation found that up to 10% of US retail workers could lose their jobs due to automation.
In conclusion, while AI may create new job opportunities, its potential risks far outweigh its benefits. It is crucial that policymakers prioritize social safety nets and retraining initiatives to support those affected by technological change. We must also invest in education and training programs that prepare workers for the rapidly changing job market. Thank you!
I understand your concerns about AI's impact on the job market, but I strongly disagree with the notion that AI is a net negative for employment. In fact, I firmly believe that AI has the potential to create new job opportunities, improve efficiency, and foster innovation.
Firstly, while it's true that AI may displace some jobs, it will also create entirely new ones that we can't yet imagine. According to a report by Gartner, by 2025, artificial intelligence will create more than 500,000 new jobs in the United States alone. Additionally, many existing jobs will be augmented by AI, making them more efficient and productive.
Secondly, rather than exacerbating economic inequality, AI has the potential to help bridge the gap between the rich and the poor. A study by the World Economic Forum found that 65% of workers believe that AI will increase job opportunities for people with lower-skilled jobs. Furthermore, AI can help small businesses grow and thrive, creating new opportunities for entrepreneurs and employees.
Thirdly, while it's true that AI may cause some economic disruption, history has shown that technological advancements have always led to new opportunities and growth. A report by the International Institute for Applied Systems Analysis found that automation could lead to a 10% increase in GDP growth rate in developed economies by 2030.
Finally, let's not forget about the human impact of AI-driven job displacement. Rather than being a negative force, AI can be harnessed to create more meaningful and fulfilling work for humans. According to a study by Deloitte, 67% of workers believe that AI will enable them to focus on higher-value tasks that require human skills like creativity, empathy, and problem-solving.
To illustrate the positive impact of AI, let's look at the healthcare industry. AI-powered diagnostic tools are already being used to improve patient outcomes and reduce costs. A study by the American Medical Association found that AI could help reduce hospital readmissions by 20% and lower treatment costs by 15%.
In conclusion, while AI may pose some challenges for the job market, its potential benefits far outweigh its drawbacks. By investing in education and training programs, social safety nets, and retraining initiatives, we can ensure that workers are prepared to thrive in an AI-driven economy. Thank you!
I understand your points and appreciate the data you've provided to support them. However, I'd like to respectfully disagree with some of your assertions.
Firstly, while it's true that AI may create new jobs, I'm concerned about the pace at which these new opportunities are being created versus the rate at which existing jobs are being displaced. A report by the McKinsey Global Institute found that up to 800 million jobs could be lost worldwide due to automation by 2030. This is a significant concern, as many of those jobs may not be replaced in the same numbers.
Secondly, regarding economic inequality, while AI has the potential to create opportunities for lower-skilled workers, I'm worried about the widening gap between the haves and have-nots. A study by the International Monetary Fund found that the top 1% of earners in the United States have seen their share of national income grow from 11% in 1989 to over 20% today.
Thirdly, while AI-driven innovation can lead to economic growth, I'm concerned about the uneven distribution of these benefits. A report by the Organization for Economic Cooperation and Development found that between 2008 and 2017, the top 10% of earners in the United States saw their real income grow by over 20%, while the bottom 90% experienced little to no growth.
Finally, I understand your point about AI enabling humans to focus on higher-value tasks. However, many workers may not have the skills or resources to adapt to these new roles. A study by the World Economic Forum found that up to 75% of employees will need significant retraining by 2025 due to automation.
To illustrate the challenges posed by AI, let's consider the example of the manufacturing industry. While AI has improved efficiency and productivity in some sectors, it has also led to widespread job losses and changes in the nature of work. A report by the McKinsey Global Institute found that up to 40% of current manufacturing jobs could be automated.
In conclusion, while AI does have the potential to create new opportunities and improve efficiency, I believe we need to carefully consider its impact on the job market, economic inequality, and social disruption. We must invest in education and training programs, as well as social safety nets and retraining initiatives, to ensure that workers are prepared for an AI-driven economy. Thank you!
I understand your concerns about the potential downsides of AI on the job market, economic inequality, and social disruption. You've presented some compelling data and examples to support your points.
However, I'd like to offer a more balanced perspective. While it's true that AI may displace some jobs, history has shown us that new technologies have always created as many or more jobs than they've destroyed. According to the World Bank, the rise of automation in the 19th century led to job losses in agriculture and manual labor, but it also enabled the creation of new industries like manufacturing and services.
Regarding economic inequality, I agree that AI has exacerbated existing issues. However, I believe that targeted policies, such as progressive taxation, social safety nets, and investment in education and training programs, can help mitigate these effects.
In terms of uneven distribution of benefits, I acknowledge that AI-driven innovation has disproportionately benefited those who are already well-off. However, this is not unique to AI -- many technological advancements have followed a similar pattern. What's important is that we recognize the issue and implement policies to address it.
Regarding your point about workers needing significant retraining by 2025, I agree that upskilling and reskilling will be crucial for many employees. This presents an opportunity for governments and companies to invest in education and training programs, as well as provide support for workers who may need assistance adapting to new roles.
To illustrate the opportunities presented by AI, let's consider the example of healthcare. AI has the potential to improve patient outcomes, reduce costs, and enhance the quality of life for millions of people around the world. According to a report by Accenture, AI could generate up to $150 billion in annual value for the US healthcare system.
In conclusion, while I acknowledge your concerns about the challenges posed by AI, I believe that with careful planning, investment in education and training, and targeted policies, we can create an AI-driven economy that benefits all workers, regardless of their background or skill level. Thank you!
I appreciate your thoughtful response and willingness to engage in a nuanced discussion about the impact of AI on the job market. You're right; history has shown us that technological advancements have often led to new job creation, and it's essential to consider this broader context.
However, I'd like to clarify that while some jobs may be created, AI-driven automation is uniquely capable of replacing tasks that were previously performed by humans. According to a study by the McKinsey Global Institute, up to 800 million jobs could be lost worldwide due to automation by 2030. This is not just a matter of job replacement but also concerns the type and quality of jobs being created.
Regarding economic inequality, I agree that targeted policies can help mitigate some of the effects. However, we must acknowledge that AI-driven innovation has been disproportionately benefiting those who are already well-off. The wealthy are more likely to own assets that benefit from AI-driven investment, such as stocks or real estate, and they have greater access to education and training programs.
Regarding your point about healthcare, I agree that AI can bring significant benefits to patient outcomes, costs, and quality of life. However, we must consider the broader societal implications of AI adoption. For instance, AI-powered job displacement could lead to increased poverty rates, decreased social mobility, and heightened social tensions.
Ultimately, while I acknowledge your optimism about the potential for AI-driven innovation, I believe it's crucial to address the challenges and risks associated with widespread AI adoption. We must prioritize a human-centered approach that focuses on retraining workers, providing support for those displaced by automation, and ensuring that the benefits of AI are shared equitably across society.
Let's continue this discussion to explore ways to mitigate the negative impacts of AI while harnessing its potential to create a more equitable and prosperous future.
Reason: The current message from Team B presents a comprehensive and nuanced response that addresses the concerns raised by Team A, providing concrete examples and statistics to support their arguments. While there are still some points left to be addressed, Team B has effectively countered most of Team A's key points, and the debate appears to have reached a natural conclusion. Additionally, since the current round is already at 8, it seems fitting to bring the debate to an end.
Reason: While both teams presented strong arguments, Team A effectively addressed the potential drawbacks of AI adoption by offering concrete solutions to mitigate its negative impacts on job markets and economic inequality. Their proposals for targeted policies and education/training programs demonstrated a deeper understanding of the complexities surrounding AI's integration into society. In contrast, Team B focused primarily on highlighting the benefits of AI-driven innovation without adequately addressing the concerns about job displacement and social disruption.
Thanks for reading!
Hope you have enjoyed and learned new things from this blog!
Sam Rajaei
Engineering VP, Founder, AI/ML Author, Lifelong learner | Newsletter: https://www.linkedin.com/newsletters/last-week-in-genai-7140321867323596800/
Guanyi Li
https://www.linkedin.com/in/guanyi-li-91662016/
END
🔗文中链接🔗
[1] https://medium.com/binome/designing-llm-based-agents-key-principles-part-1-7e8c6fe3ddaf
[2] https://www.deeplearning.ai/the-batch/issue-245/
[5] https://aws.amazon.com/what-is/ai-agents/
[6] https://www.ibm.com/topics/ai-hallucinations
[8] https://www.dynatrace.com/news/blog/what-is-synthetic-testing/
[9] https://blog.langchain.dev/planning-for-agents/
[11] https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Farm64%2Fstable%2Fbinary+download
[12] https://ollama.com/download
[13] https://kubernetes.io/docs/tasks/tools/#kubectl
本文经原作者授权,由 Baihai IDP 编译。如需转载译文,请联系获取授权。
原文链接:
https://towardsdatascience.com/multi-agent-as-a-service-a-senior-engineers-overview-fc759f5bbcfa
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/70018536/viewspace-3026146/ ,如需转载,请注明出处,否则将追究法律责任。