Mastering Fuzzy Logic in Multi-Agent LLM Analysis System

Integrating fuzzy logic into a multi-agent system (MAS) enhances the agents’ ability to handle uncertainty and make decisions based on imprecise information. Below is an example of how to implement a simple MAS where agents use fuzzy logic for decision-making.

Photo by Aswin Karuvally on Unsplash

Fuzzy Logic in LLMs

Fuzzy logic allows systems to process varying degrees of truth rather than binary true/false evaluations. Incorporating fuzzy reasoning into LLMs enables them to better interpret and generate nuanced language, reflecting the complexities of human communication. However, research indicates that current LLMs face challenges in effectively performing fuzzy reasoning tasks. For instance, the FRoG benchmark evaluates LLMs’ abilities to handle generalized quantifiers in mathematical word problems, revealing significant difficulties in this area.

Multi-Agent Systems and Fuzzy Logic

In multi-agent systems, agents must often collaborate and make decisions under uncertainty. Applying fuzzy logic within these systems allows agents to reason with partial truths and handle ambiguous information more effectively. For example, the MAgIC framework assesses LLM-powered agents on various cognitive and collaborative metrics, highlighting the importance of advanced reasoning capabilities in multi-agent interactions.

Enhancing LLMs with Fuzzy Theory

Recent studies propose augmenting LLMs with fuzzy theory to improve their commonsense reasoning. By treating model output probabilities as fuzzy variables and applying predefined fuzzy rules during inference, LLMs can achieve more nuanced understanding and generation of language. This approach not only enhances performance in reasoning tasks but also improves the interpretability of the model’s processes.

Let’s illustrate how fuzzy logic can be applied in risk assessment within a multi-agent LLM system, specifically in finance.

Scenario: Assessing the credit risk of a loan applicant.

1. Multi-Agent LLM System:

• Agent 1 (Financial Data Agent): This agent processes structured financial data like income, debt-to-income ratio, credit score, loan amount, loan term, etc. It uses LLMs to identify trends and anomalies in the applicant’s financial history. For example, it might detect inconsistencies in reported income or identify a pattern of missed payments on smaller loans.
• Agent 2 (Contextual Information Agent): This agent gathers and processes unstructured data, such as news articles about the applicant’s industry, social media sentiment about the applicant’s company (if applicable), and even textual information from the loan application itself (e.g., the applicant’s explanation for a recent job change). LLMs are used to extract relevant information and assess its potential impact on creditworthiness. For example, negative news about the applicant’s industry could be a risk factor.
• Agent 3 (Expert Knowledge Agent): This agent contains a knowledge base of financial regulations, best practices, and expert opinions on credit risk. It uses LLMs to provide context and interpret the information provided by the other agents. For example, it can weigh the importance of different financial ratios based on the loan type and the current economic climate.

2. Fuzzy Logic Integration:

Each agent outputs a “fuzzy value” for different risk factors. These values represent the degree to which a particular factor contributes to the overall risk. Instead of just “high” or “low” risk, we have a spectrum.

Example:

Agent 1 might output:

• “Income Stability”: 0.7 (fairly stable income, some minor fluctuations)
• “Debt-to-Income Ratio”: 0.3 (low debt, good sign)
• “Credit Score”: 0.9 (excellent credit history)

Agent 2 might output:

• “Industry Outlook”: 0.5 (moderate risk, some uncertainty in the industry)
• “Financial Behavior Sentiment”: 0.2 (positive sentiment, no red flags)

3. Fuzzy Inference System (FIS)

A FIS takes these fuzzy inputs and combines them using a set of fuzzy rules to produce an overall credit risk score.

Example Rules:

• IF “Income Stability” is HIGH AND “Debt-to-Income Ratio” is LOW AND “Credit Score” is HIGH THEN “Credit Risk” is LOW.
• IF “Industry Outlook” is MEDIUM AND “Financial Behavior Sentiment” is NEUTRAL THEN “Credit Risk” is MEDIUM.
• IF “Income Stability” is LOW OR “Debt-to-Income Ratio” is HIGH OR “Credit Score” is LOW THEN “Credit Risk” is HIGH.

The FIS uses membership functions to define the “fuzziness” of the inputs (e.g., what constitutes “HIGH” income stability) and then applies these rules to generate a fuzzy output for “Credit Risk.” This output is then defuzzified to produce a crisp numerical risk score (e.g., a number between 0 and 1, where 0 is low risk and 1 is high risk).

4. Decision Making:

The final risk score is used by the system to make a decision about the loan application. This could involve:

• Loan Approval/Rejection: Based on a pre-defined threshold.
• Interest Rate Setting: Higher risk applicants are offered loans with higher interest rates.
• Loan Amount Adjustment: The loan amount offered might be adjusted based on the risk assessment.

Benefits of Fuzzy Logic in this Example:

• Handles Uncertainty: Fuzzy logic can handle the inherent uncertainty in assessing credit risk. Factors like “industry outlook” are not easily quantifiable, and fuzzy logic allows the system to work with imprecise information.
• Combines Diverse Information: It provides a framework for combining information from different sources (structured and unstructured data, expert knowledge) in a meaningful way.
• More Nuanced Assessment: Instead of just “approve” or “reject,” the system can provide a more nuanced assessment of risk, allowing for more flexible decision-making.

This is a simplified example, but it illustrates the basic principles of using fuzzy logic in a multi-agent LLM system for risk assessment. In a real-world application, the system would be much more complex, with more agents, more risk factors, and a more sophisticated FIS.

import numpy as np
import skfuzzy as fuzz
from skfuzzy import control as ctrl

# 1. Define Input Variables (Fuzzy Sets)

income_stability = ctrl.Antecedent(np.arange(0, 1.1, 0.1), 'income_stability')  # 0 to 1, step 0.1
debt_ratio = ctrl.Antecedent(np.arange(0, 1.1, 0.1), 'debt_ratio')  # 0 to 1, step 0.1
credit_score = ctrl.Antecedent(np.arange(0, 1.1, 0.1), 'credit_score') # 0 to 1, step 0.1
industry_outlook = ctrl.Antecedent(np.arange(0, 1.1, 0.1), 'industry_outlook') # 0 to 1, step 0.1
financial_sentiment = ctrl.Antecedent(np.arange(0, 1.1, 0.1), 'financial_sentiment') # 0 to 1, step 0.1


# Define membership functions for each input variable
income_stability['low'] = fuzz.trimf(income_stability.universe, [0, 0, 0.5])
income_stability['medium'] = fuzz.trimf(income_stability.universe, [0.25, 0.5, 0.75])
income_stability['high'] = fuzz.trimf(income_stability.universe, [0.5, 1, 1])

debt_ratio['low'] = fuzz.trimf(debt_ratio.universe, [0, 0, 0.5])
debt_ratio['medium'] = fuzz.trimf(debt_ratio.universe, [0.25, 0.5, 0.75])
debt_ratio['high'] = fuzz.trimf(debt_ratio.universe, [0.5, 1, 1])

credit_score['low'] = fuzz.trimf(credit_score.universe, [0, 0, 0.5])
credit_score['medium'] = fuzz.trimf(credit_score.universe, [0.25, 0.5, 0.75])
credit_score['high'] = fuzz.trimf(credit_score.universe, [0.5, 1, 1])

industry_outlook['low'] = fuzz.trimf(industry_outlook.universe, [0, 0, 0.5])
industry_outlook['medium'] = fuzz.trimf(industry_outlook.universe, [0.25, 0.5, 0.75])
industry_outlook['high'] = fuzz.trimf(industry_outlook.universe, [0.5, 1, 1])

financial_sentiment['negative'] = fuzz.trimf(financial_sentiment.universe, [0, 0, 0.5])
financial_sentiment['neutral'] = fuzz.trimf(financial_sentiment.universe, [0.25, 0.5, 0.75])
financial_sentiment['positive'] = fuzz.trimf(financial_sentiment.universe, [0.5, 1, 1])


# 2. Define Output Variable (Fuzzy Set)

risk = ctrl.Consequent(np.arange(0, 1.1, 0.1), 'risk')  # 0 to 1, step 0.1

# Define membership functions for the output variable
risk['low'] = fuzz.trimf(risk.universe, [0, 0, 0.5])
risk['medium'] = fuzz.trimf(risk.universe, [0.25, 0.5, 0.75])
risk['high'] = fuzz.trimf(risk.universe, [0.5, 1, 1])


# 3. Define Fuzzy Rules

rule1 = ctrl.Rule(income_stability['high'] & debt_ratio['low'] & credit_score['high'] & industry_outlook['medium'] & financial_sentiment['positive'], risk['low'])
rule2 = ctrl.Rule(income_stability['medium'] & debt_ratio['medium'] & credit_score['medium'] & industry_outlook['medium'] & financial_sentiment['neutral'], risk['medium'])
rule3 = ctrl.Rule(income_stability['low'] | debt_ratio['high'] | credit_score['low'] | industry_outlook['low'] | financial_sentiment['negative'], risk['high']) #OR condition example

# More rules can be added for other combinations...

# 4. Create Control System

risk_ctrl = ctrl.ControlSystem([rule1, rule2, rule3]) # Add all rules here

# 5. Create Control System Simulation

risk_simulation = ctrl.ControlSystemSimulation(risk_ctrl)

# 6. Provide Input Values (from your LLM agents)

risk_simulation.input['income_stability'] = 0.8  # Example values from LLM agents
risk_simulation.input['debt_ratio'] = 0.2
risk_simulation.input['credit_score'] = 0.9
risk_simulation.input['industry_outlook'] = 0.6
risk_simulation.input['financial_sentiment'] = 0.7


# 7. Compute the Result

risk_simulation.compute()

# 8. Print the Result

print(risk_simulation.output['risk'])
risk.view(sim=risk_simulation) # This will show the fuzzy membership graphs and the final output.

fuzzy membership graphs

Key improvements and explanations

• scikit-fuzzy (skfuzzy): This code uses the skfuzzy library, which is the recommended way to work with fuzzy logic in Python. Install it using: pip install scikit-fuzzy
• Clearer Variable Names: More descriptive variable names (e.g., income_stability, debt_ratio) improve readability.
• Membership Functions: The code now defines triangular membership functions (fuzz.trimf) for each input and output variable. You can adjust the ranges and shapes of these functions to better reflect your specific problem. Other membership function types are available (trapezoidal, Gaussian, etc.).
• Fuzzy Rules: The fuzzy rules are defined using the control.Rule class, making them more structured and readable. The example includes an OR condition in rule3 to show how to implement it. You'll need to add more rules to cover a wider range of scenarios.
• Control System and Simulation: The code creates a control.ControlSystem and control.ControlSystemSimulation to manage the fuzzy inference process.
• Input Values from LLMs: The example shows how to provide input values to the simulation. These values would come from your LLM agents in a real application.
• Output and Visualization: The code prints the final risk score and uses risk.view(sim=risk_simulation) to visualize the fuzzy membership functions and the output. This visualization is very helpful for understanding how the fuzzy system is working.
• Range of Values: Input values are now between 0 and 1, representing a normalized value (e.g., from the LLM agent). You can adjust the universe of discourse (the range of possible values) for the fuzzy sets if needed.

How to Use with Your LLMs:

1. LLM Agent Output: Your LLM agents should be designed to output values between 0 and 1 for each of the input variables (income stability, debt ratio, etc.). For example, an agent might analyze text and determine that the “industry outlook” is moderately positive, assigning a value of 0.6.
2. Integration: Take the output values from your LLM agents and feed them into the risk_simulation.input dictionary, as shown in the example.
3. Fuzzy Inference: Run risk_simulation.compute() to get the fuzzy risk score.
4. Decision Making: Use the risk score to make decisions in your application (loan approval, interest rate setting, etc.).

Remember to customize the membership functions, rules, and input variables to fit the specifics of your risk assessment scenario. This code provides a solid foundation for integrating fuzzy logic with your multi-agent LLM system.

Mastering Fuzzy Logic in Python

System Design: Multi-Agent LLM Legal Analysis System (Coding)

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in Large Language Models

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability…

Enhancing Large Language Models by Fuzzy Theory for Commonsense Reasoning

Large Language Model Evaluation Criteria Framework in Healthcare: Fuzzy MCDM Approach - SN Computer…

If you found this article insightful and want to explore how these technologies can benefit your specific case, don’t hesitate to seek expert advice. Whether you need consultation or hands-on solutions, taking the right approach can make all the difference. You can support the author by clapping below 👏🏻 Thanks for reading!

Oleh Dubetsky|Linkedin