A Call for Reasoning Empowered Foundation Models

• Motivation
• Hierarchical Construction of Reasoning-Empowered Foundation Models
• Conclusion
• Citation

In recent months, we have conducted a thorough analysis of major challenges in the AI for Science domain and believe that A Call for Reasoning Empowered Foundation Models is crucial. Here I present the key perspectives, though research details are omitted due to certain constraints. If you are interested, please feel free to reach out (https://yataobian.com/) for further discussions.

Motivation

Artificial Intelligence for Science (AI4Sci) is rapidly transforming how we approach discovery in fundamental fields like chemistry, materials science, and biology. We’ve seen remarkable progress, with AI models such as Graph Neural Networks (GNNs) and Transformers demonstrating powerful capabilities in pattern recognition and prediction across various atomic data representations (e.g., 1D sequences, 2D topological graphs, 3D coordinates) – a form of “fast thinking”. These models can efficiently process vast datasets, identifying correlations that might elude human researchers.

However, to unlock the next level of scientific breakthroughs, we need to move beyond current paradigms. While existing models excel at this “fast thinking,” they often function as “black boxes,” struggle with generalization to novel, out-of-distribution (OOD) scientific data (a common scenario in scientific exploration), and critically, lack the deep scientific reasoning capabilities – the “deep thinking” – that characterizes human scientific inquiry. They can tell us what (e.g., predict a property) but often not why (e.g., explain the underlying chemical intuition or causal mechanism) or how to achieve a novel outcome through multi-step processes. This gap limits their utility as genuine engines of scientific discovery, confining them mostly to roles of data analysis. Furthermore, the reliability and safety of generative models remain significant concerns, as unconstrained AI could propose unstable, toxic, or otherwise hazardous structures.

{#figure-1} Figure 1: Reasoning Capabilities to be developed, from bottom to top: Target 1) Basic reasoning (Joint generation-prediction): enabling bi-directional reasoning between structure & property; Target 2) Deep reasoning (Abductive & Multi-step reasoning): equip Foundation Models with a deeper understanding of the underlying principles governing atomic systems, enabling to perform tasks beyond prediction, such as explanation, hypothesis generation, and multi-step planning; Target 3) Safety reasoning (Safety alignment with human values & ethics): generative models could be misused by adversaries for designing novel biochemical weapons, toxins, or hazardous materials, also making “mirror life” molecules potentially leading to catastrophic interactions with biological systems. Safety & ethical alignment are not afterthoughts but integral design principles woven into the Reasoning Empowered Foundation Models’ architectures & reasoning processes.

The “big problem” we face is transforming AI4Sci from data-driven pattern predictors into genuine scientific assistants capable of prediction, understanding, causal reasoning, planning, and discovery. We envision a new generation of AI: Reasoning-Empowered Foundation Models (RE-FMs). This framework, as conceptualized in our research vision (see Figure 1, aims to integrate the “fast thinking” System I capabilities (pattern recognition, intuitive prediction) with explicit modules for “deep thinking” System II capabilities (deliberate, multi-step reasoning).

Hierarchical Construction of Reasoning-Empowered Foundation Models

Developing RE-FMs involves a hierarchical approach to infusing reasoning:

1. Laying the Foundation with Basic Reasoning: The journey begins by constructing unified, cross-domain atomic foundation models. These models would be pre-trained on diverse atomic graph data (molecules, proteins, materials, etc.). A key capability here is Basic Reasoning, which involves enabling joint prediction (e.g., $P(\text{property} | \text{structure})$) and generation (e.g., $P( \text{structure} | \text{property})$). Mastering this bi-directional reasoning is fundamental for tasks like inverse design and enhances data efficiency and OOD generalization, providing a robust base for higher-level reasoning. This involves leveraging advanced self-supervised learning tasks, expressive geometric architectures, and cross-domain alignment techniques (e.g., based on Schrödinger Bridge).

2. Ascending to Deep Reasoning: Building upon this foundation, the next crucial step is to embed explicit, interpretable scientific reasoning capabilities, moving towards deeper scientific understanding. This encompasses:

   • Abductive Reasoning: Empowering models to infer the best explanation for observations or predictions and generate testable scientific hypotheses. This isn’t just about prediction, but about asking “why.” It involves a principled fusion of:
     • Interpretability: Identifying critical substructures (e.g., functional groups) responsible for properties, using methods like energy-based Shapley values or intrinsically interpretable models.
     • Causal Discovery: Distinguishing correlation from genuine causation within atomic systems.
     • Counterfactual Explanation: Understanding how minimal, chemically valid changes to a structure can alter its properties, answering “what if” questions.
   • Multi-step Reasoning & Planning: Enabling models to execute sequences of inferential steps for complex problems. This is vital for tasks like retrosynthesis pathway planning, reaction mechanism elucidation, and multi-step molecular optimization or generation, potentially adapting techniques from search algorithms like those in AlphaZero or LLM reasoning paradigms.

3. The Essential Umbrella: Safety Reasoning: As these AI models become increasingly powerful, especially in generative tasks, ensuring their reliability, safety, and ethical alignment is not an afterthought but an integral design principle. This Safety Reasoning layer involves:

   • Verifying model robustness against perturbations.
   • Developing techniques for safe and constrained generation and planning – ensuring, for example, that generated molecules are non-toxic, stable, and synthesizable.
   • Critically, adapting methods like knowledge editing techniques to align RE-FMs with safety protocols and ethical considerations, preventing the generation of hazardous molecules/materials, biochemical weapons, or problematic “mirror life” molecules that could have catastrophic interactions with biological systems.

Conclusion

The journey towards RE-FMs is a call to the scientific and AI communities. It requires a concerted effort to build models that don’t just predict, but actively participate in the scientific discovery process by reasoning about the underlying principles. By imbuing AI with this hierarchy of reasoning capabilities, we can hope to significantly accelerate discovery cycles, enhance our fundamental understanding through hypothesis generation and explanation, tackle complex scientific tasks requiring multi-step logic, and ultimately, develop a truly collaborative, transparent, and trustworthy AI platform for science. This is not just an incremental improvement; it’s a necessary paradigm shift for the future of AI-driven scientific exploration.

Citation

If you found this useful, please cite this as:

@article{bluewhalelab@nus2025a-call-for,
  title   = {A Call for Reasoning Empowered Foundation Models},
  author  = {Yatao Bian},
  year    = {2025},
  month   = {Jun},
  url     = {http://127.0.0.1:4000/blog/2025/A-Call-for-Reasoning-Empowered-Foundation-Models/}
}