Neurosymbolic AI combats LLM Hallucinations

The Problem with Large Language Models

The primary issue with big tech's AI initiatives isn't the potential for AI to dominate humanity. Instead, it's that large language models (LLMs) such as OpenAI's ChatGPT, Google's Gemini, and Meta's Llama, are still inaccurate, and this problem is proving difficult to solve.

These inaccuracies are known as hallucinations. One notable instance involved US law professor Jonathan Turley, who was falsely accused by ChatGPT. OpenAI's response was to essentially remove Turley by instructing ChatGPT not to answer questions about him, which is not a suitable solution. Addressing hallucinations on a case-by-case basis isn't effective.

LLMs also tend to amplify stereotypes and produce Western-centric responses. Furthermore, there's a lack of accountability for this widespread misinformation, as it's hard to determine how the LLM arrived at its conclusions.

These issues sparked debate following the release of GPT-4, though the discussion has since subsided. The EU passed its AI Act to oversee the field, but the act relies on AI companies to self-regulate and doesn't directly tackle the core problems. Tech companies continue to release LLMs globally without proper oversight.

Recent tests indicate that even advanced LLMs remain unreliable, yet leading AI companies avoid taking responsibility for errors.

LLMs' tendency to misinform and reproduce bias cannot be fixed with gradual improvements. The increasing use of agentic AI, where LLMs are assigned tasks like booking travel or managing bill payments, could amplify these problems.

Neurosymbolic AI as a Solution

The emerging field of neurosymbolic AI has the potential to resolve these issues and decrease the vast amounts of data needed to train LLMs. Neurosymbolic AI combines neural networks' predictive learning with formal rules that enable more reliable reasoning. These rules include logic rules, mathematical rules, and agreed-upon meanings of words and symbols. Some rules are directly input, while others are deduced by the AI through knowledge extraction from training data.

This should result in an AI that doesn't hallucinate and learns more efficiently by organizing knowledge into reusable components. For example, if the AI knows that things get wet outside when it rains, it doesn't need to store every instance of wet objects—the rule applies to new objects as well.

During development, neurosymbolic AI uses a neurosymbolic cycle, integrating learning and formal reasoning. A partially trained AI extracts rules from data and incorporates this knowledge back into the network before further training.

Benefits of Neurosymbolic AI

This approach is more energy-efficient because less data needs to be stored. The AI is also more accountable because users can control how conclusions are reached and how the system improves. It can also be made fairer by ensuring its decisions don't depend on factors like race or gender.

Symbolic AI, the first wave of AI in the 1980s, involved teaching computers formal rules. Deep learning was the second wave in the 2010s, and neurosymbolic AI is considered the third.

Neurosymbolic AI is easiest to implement in specific areas where rules are well-defined. It has emerged in Google's AlphaFold and AlphaGeometry. DeepSeek uses a learning technique called distillation. More research is needed to make neurosymbolic AI fully viable for general models and to enhance their ability to extract knowledge.

It's unclear if LLM creators are pursuing this approach, but they seem committed to scaling up with more data.

For AI to advance, systems must adapt to new situations with limited examples, verify their understanding, multitask, reuse knowledge, and reason reliably. Well-designed technology could offer an alternative to regulation, with built-in checks and balances. While there is progress to be made, a path forward exists.