ZEBRA-Prop: A Zero-Shot Embedding-Based Rapid and Accessible Regression Model for Materials Properties
2026.11.11
Abstract
Large language models (LLMs) exhibit substantial potential across diverse scientific disciplines, including materials science. A property prediction framework, ZEBRA-Prop (Zero-Shot Embedding-Based Rapid and Accessible Regression Model for Materials Properties), is presented here as an extension of LLM-Prop. In contrast to LLM-Prop, which requires task-specific fine-tuning of the LLM, ZEBRA-Prop keeps the text encoder frozen and eliminates task-specific fine-tuning of the LLM, thereby reducing computational cost and enabling rapid model training. The framework employs MatBERT, a language model pretrained on materials science literature, to enhance predictive capability. Multiple textual embeddings are incorporated through a learnable weighting mechanism, which alleviates the context-length constraints inherent in LLM-Prop and facilitates effective integration of diverse textual representations. Evaluation is conducted using two datasets: the TextEdge dataset (~140,000 entries) and an in-house dataset (~2000 entries) derived from the Materials Project database, with physical properties obtained from first-principles calculations. The predictive performance of ZEBRA-Prop is close to that of LLM-Prop, while the training time is reduced by approximately 95%. The performance improvements are attributable to three principal factors: domain-specific LLM utilization, diversified textual descriptions, and systematic text preprocessing. ZEBRA-Prop is a scalable, computationally efficient framework that supports accelerated materials discovery, particularly under limited computational resources.
Large language models (LLMs) exhibit substantial potential across diverse scientific disciplines, including materials science. A property prediction framework, ZEBRA-Prop (Zero-Shot Embedding-Based Rapid and Accessible Regression Model for Materials Properties), is presented here as an extension of LLM-Prop. In contrast to LLM-Prop, which requires task-specific fine-tuning of the LLM, ZEBRA-Prop keeps the text encoder frozen and eliminates task-specific fine-tuning of the LLM, thereby reducing computational cost and enabling rapid model training. The framework employs MatBERT, a language model pretrained on materials science literature, to enhance predictive capability. Multiple textual embeddings are incorporated through a learnable weighting mechanism, which alleviates the context-length constraints inherent in LLM-Prop and facilitates effective integration of diverse textual representations. Evaluation is conducted using two datasets: the TextEdge dataset (~140,000 entries) and an in-house dataset (~2000 entries) derived from the Materials Project database, with physical properties obtained from first-principles calculations. The predictive performance of ZEBRA-Prop is close to that of LLM-Prop, while the training time is reduced by approximately 95%. The performance improvements are attributable to three principal factors: domain-specific LLM utilization, diversified textual descriptions, and systematic text preprocessing. ZEBRA-Prop is a scalable, computationally efficient framework that supports accelerated materials discovery, particularly under limited computational resources.
For inquiries regarding this article
Associate Professor
Department of Medical Life Science, 皇家华人, Yokohama, Japan