Collaborative AI Agents and Critics for Fault Detection and Cause Analysis in Network Telemetry

Introduction Multimodal foundation models are several times more complex than unimodal models.

Developing multimodal foundation models that support multiple modalities increases complexity several times more than uni-modal models because of processing different modalities' data.

Introduction Multi-agent collaboration with specialized models increases task accuracy.

Instead of developing foundation models that work for many modalities, we can work with specialized models that do specific tasks very well and then integrate these foundation models of desired modalities; this can be done through multi-agent collaboration. Each agent can have access to one or more foundation models; we call such agents AI agents . Recently, systems utilizing foundation model-based single autonomous agents are progressing towards multiple collaborative agents. Every agent of the foundation models-based multi-agent system can have a unique capability, and they collaboratively c

Introduction The paper develops stochastic algorithms for AI agents and critics in a federated multi-agent system.

We propose stochastic algorithms to control the number of AI agents and critics in a federated multi-agent system. In the proposed model, multiple agents and critics collaborate with a central server to complete multimodal tasks and aim to minimize the overall cost to the system.

Introduction Application includes fault detection, severity, and cause analysis in network telemetry.

We present an example of fault detection, severity, and cause analysis in network telemetry and thorough evaluation to check the algorithm's efficacy.

Problem Agents and critics minimize overall cost without inter-agent or inter-critic communication.

Collaboratively, they minimize the overall cost to the system with no inter-agent or inter-critic communication.

Problem Cost functions and derivatives are kept private.

AI agents and critics keep their cost functions or derivatives of cost functions private.

Problem The paper provides convergence guarantees using multi-time scale stochastic approximation techniques.

Using multi-time scale stochastic approximation techniques, we provide convergence guarantees on the time-average active states of AI agents and critics.

Problem Communication overhead is independent of the number of agents and critics.

The communication overhead is a little on the system, of the order of , for modalities and is independent of the number of AI agents and critics.

Method Agents calculate participation probability using feedback signals, time-average states, and cost derivatives.

After receiving the feedback signal from the central server at time step, agent calculates probability to make a decision to be active or not to participate in completing the task of modality at the next time step.

Method The central server updates feedback signals by subtracting gain times aggregate error.

The central server updates the feedback signal for agents of modality as follows: and.

Method Critics evaluate agent responses and send feedback; agents revise if they agree.

The critic then evaluates the agent's response and sends feedback to the agent. The agent considers the critic's feedback and improves its response if it agrees with the critic's evaluation.

Results Time-average active states converge almost surely to optimal values.

For any modality, and fixed constant initial states and, the sequence converges, almost surely.

Results XG Boosting agents outperform LLM-enabled agents on all fault detection metrics.

We observe that agents enabled with XG Boosting provide faster results and higher accuracy, F1-score, precision, and recall than AI agents and critics enabled with LLMs.

Results The algorithm's cost ratio evolves close to the CVX solver's optimal benchmark.

The evolution of the ratio of total cost obtained by the developed algorithm and the total optimal cost obtained by the CVX solver.

Results LLM agents classify faults as Critical or Non-critical with cause summaries using ChromaDB log files.

The AI agents first classify the faults as 'Critical' or 'Non-critical' with access to the network telemetry log files (stored in ChromaDB) and provide a brief cause of the fault.

Conclusion The algorithms provide convergence guarantees for time-average active states.

Using multi-time scale stochastic approximation techniques, we provide convergence guarantees on the time-average active states of agents and critics.

Conclusion Communication overhead is independent of the number of agents and critics.

The communication overhead is a little on the system, of the order of, for modalities and is independent of the number of AI agents and critics.

Conclusion XG Boosting agents outperform LLM agents on fault detection metrics.

For fault detection, the agents enabled with the classical machine learning approach have better accuracy, F1 score, precision, and recall than the large language model-enabled agents.

Conclusion Future work includes real-time deployment and extension to smart energy and industrial domains.

It would be exciting to implement the algorithms on real network devices and detect the faults in real-time and analyze the causes. Moreover, the algorithm can be implemented in several application domains, such as smart energy systems, fault detection, and cause analysis in industrial applications.