One-sentence Summary

MIT's Concrete Sustainability Hub develops a data-driven screening framework that uses large language models to extract chemical compositions from 88,000 papers and a multi-headed neural network to predict three cementitious reactivity metrics, identifying construction and demolition wastes, municipal solid waste incineration ash, and natural rocks including ignimbrite and pumice as viable clinker substitutes capable of reducing global greenhouse gas emissions by 3%.

Key Contributions

• Large language models extract chemical compositions and material types for 14,000 secondary and natural cementitious precursors from 88,000 academic papers, expanding candidate materials beyond coal fly ash and ground granulated blast furnace slag.

• A multi-headed neural network predicts heat release, calcium hydroxide consumption, and bound water from chemical composition, median particle size, specific gravity, and amorphous or crystalline phase content, providing a unified assessment of cementitious reactivity and pozzolanicity.

• Nearly 5 to 25 percent of 20 rock types, including ignimbrite, silicic tuff, pumice, shale, and rhyolite, are identified as reactive with heat release above 200 J/g, while supply-constrained scenarios indicate that construction and demolition wastes and municipal solid waste incineration ash could substitute half of global cement production and reduce global greenhouse gas emissions by 3 percent.

Introduction

The study sits at the intersection of low-carbon cement production and materials screening. It matters because cement production contributes more than 2.5 billion metric tons of greenhouse gas emissions each year, and replacing part of clinker with reactive by-products or natural minerals could sharply reduce that footprint.

Prior work has shown that some industrial by-products and natural rocks can react with lime to form cementitious phases, but reliable adoption is hard because reactivity varies widely with chemistry, availability, activation method, and long-term performance. Natural materials, in particular, lack broad and standardized reactivity maps, while real replacement materials must also meet workability, strength, durability, and leaching requirements.

The authors leverage data-driven reactivity mapping to identify a wider pool of potential clinker substitutes, including biomass ashes, municipal waste ashes, recycled construction materials, and natural rock types. Their main contribution is a reactivity-based screening framework that highlights mechanically activated natural precursors and by-products with plausible pozzolanic or hydraulic behavior, while also estimating large-scale emissions reductions if such materials replace half of global cement.

Dataset

• The authors build a literature-mined dataset of cementitious materials by mining journal papers on cement and concrete.
• The starting corpus contains approximately 5.7 million journal publications.
• Keyword filtering first narrows the corpus to about 88,000 relevant cement-and-concrete papers.
• From these, 4,312 journal papers are mined to extract 14,434 materials.
• The extracted records include material names, chemical compositions, material types, and subtypes.
• Chemical compositions are extracted from XML tables using a fine-tuned LLM.
• A second fine-tuned LLM classifies materials into 19 predefined types and subtypes, including fly ash, slag, natural pozzolans, biomass ashes, mine tailings, cements, and lime species.
• The dataset expands earlier work by classifying 12,898 materials into 19 types, compared with 7,490 materials in 11 types previously.
• The current dataset includes 2,028 fly ash samples and 1,346 slag samples.
• For retrieval, the authors create two vector databases: one for sentences and one for tables.
• The sentence database stores embeddings for approximately 3 million sentences.
• The table database stores embeddings for approximately 104,000 XML tables.
• Embedding-based retrieval is performed using transformer encoders, including all-mpnet-base-v2, all-MiniLM-L6-v2, and MatSciBERT.
• The authors select all-MiniLM-L6-v2 for table retrieval after comparing models.
• For chemical-composition extraction, the authors test pretrained and fine-tuned language models, including GPT-3.5, GPT-4, Mistral 7b, and sequential LLM agents.
• They create a manually labeled ground-truth subset from 200 randomly selected tables to fine-tune and evaluate extraction models.
• GPT-3.5 fine-tuned is selected as the final extraction model because it performs well on table detection, material detection, and numerical chemical-composition extraction.
• The final dataset is used to study compositional variation among potential cementitious precursors.
• The authors filter materials with combined CaO, Al2O3, and SiO2 content greater than 80 wt% for ternary-diagram analysis.
• Among the analyzed materials, 56% contain 15 to 70 wt% CaO, 73% contain 15 to 70 wt% SiO2, and 70.51% contain less than 15 wt% Al2O3.
• Nearly all samples, 94.52%, contain 0 to 15 wt% Fe2O3, and over 95% contain less than 10 wt% MgO.
• The dataset does not appear to rely on image cropping; processing focuses on retrieval, table extraction, and metadata construction from journal tables.
• The paper uses the mined data primarily as a structured materials database with composition and type labels, rather than describing a conventional train-test split for a predictive model.

Method

The overall workflow is designed to convert unstructured literature into a machine-readable materials dataset and then use that dataset to predict cementitious reactivity. The authors leverage a retrieval-augmented generation pipeline. First, journal papers are filtered by keywords to select papers related to cement and concrete. The resulting corpus is processed into two vector stores. One contains sentence-level embeddings, and the other contains table-level embeddings. These vector representations encode semantic information and allow relevant tables and sentences to be retrieved for downstream language model extraction.

For chemical composition extraction, table embeddings are generated with transformer encoders. A compact sentence transformer model is selected for efficient table retrieval, while domain-specific embeddings are also evaluated. The retrieval step narrows candidate tables, and LLM agents perform structured extraction from the retrieved tables. A GPT-3.5 model is fine-tuned on manually annotated tables so that it can detect relevant materials and extract numerical oxide contents in a consistent format. This module converts heterogeneous XML tables into aligned chemical compositions and material names.

Because extracted material names can be ambiguous, abbreviations, or non-descriptive, the authors add a hierarchical classification module. First, the sentence vector database is queried for context sentences associated with each material. A general-purpose open-source LLM summarizes these sentences to capture the identity and context of the material. A fine-tuned GPT-3.5 model then assigns the material to one of 19 predefined types, such as cements, clinkers, fly ashes, slags, silica fume, clays, calcined clays, mine tailings, biomass ashes, and construction wastes. The model also generates subtypes, which enables finer analysis by family and origin.

The reactivity prediction model then operates on a feature set constructed from chemical composition and physical properties. The input descriptors include major oxide contents, median particle size, amorphous content, specific gravity, and variables that describe the test paste and curing conditions. Test paste ingredients such as $\text{Ca(OH)}_2$Ca(OH)2​, $\text{CaCO}_3$CaCO3​, water, $\text{K}_2\text{SO}_4$K2​SO4​, and $\text{KOH}$KOH are included, along with curing temperature and sample age. The targets are three related $R^3$R3 test metrics. Heat release measures the extent of hydration reactions under isothermal conditions. $\text{Ca(OH)}_2$Ca(OH)2​ consumption estimates pozzolanicity through the amount of portlandite consumed during reaction. Bound water evaluates the amount of water incorporated into hydration products or adsorbed on solid phases and can serve as a proxy for heat release.

The training data are limited and unevenly distributed across the three targets. The authors therefore design an imputation-aware multi-task neural network rather than training separate models for each metric. The shared hidden layers learn common representations of descriptors such as chemistry, amorphous fraction, particle size, and paste conditions, while separate heads predict heat release, $\text{Ca(OH)}_2$Ca(OH)2​ consumption, and bound water. This structure enables cross-task transfer learning, so patterns learned from one metric can support prediction of the others.

Missing values are handled at two levels. For output targets, the loss function is computed only over non-missing labels, so a sample contributes to a task only when that target is observed. For input features, missing descriptors are imputed with an iterative process based on multiple imputation by chained equations. A LightGBM model is trained for each descriptor with missing entries, using the other descriptors as predictors. The resulting imputed values are paired with binary masks that indicate whether each input was originally observed or estimated. The network receives the concatenated descriptor vector and mask vector, which allows it to adjust predictions according to whether the value was measured or inferred.

The neural network itself is relatively compact. The input layer passes concatenated descriptors and masks into a sequence of four dense layers with ReLU activations. Dropout layers are interleaved to reduce overfitting, and batch normalization layers stabilize training. Output heads are trained with a customized weighted loss. The weights assigned to the three targets are inversely related to the amount of available data for each target, which prevents the larger dataset, such as heat release, from overwhelming the smaller targets, such as $\text{Ca(OH)}_2$Ca(OH)2​ consumption.

Training is organized as a hyperparameter-optimized regression task. The optimizer, learning rate, number of layers, and layer widths are tuned using an automated search procedure. Early stopping monitors validation loss and restores the best model weights, which is important given the limited number of experimental data points and the imputation-induced uncertainty. During inference, the trained network processes available descriptors and outputs the three reactivity estimates.

Interpretability is incorporated as part of the modeling workflow. Permutation importance is used to rank the descriptors that most strongly affect prediction error. The resulting ranking identifies major oxides, amorphous content, and specific gravity as central predictors. SHAP analysis then provides local explanations for individual predictions. The SHAP results indicate that higher $\text{Al}_2\text{O}_3$Al2​O3​ generally increases reactivity metrics, consistent with the acceleration of aluminate hydration and formation of calcium aluminate hydrates. Higher amorphous content also increases reactivity because the disordered structure provides more reactive silica and alumina. Sample age increases predicted heat release and bound water because hydration proceeds over time. $\text{CaCO}_3$CaCO3​ can increase heat release and bound water through formation of carboaluminate and ettringite phases and through conversion of anhydrous carbonate into hydrated products. In contrast, higher $\text{CaO}$CaO can reduce $\text{Ca(OH)}_2$Ca(OH)2​ consumption because the added calcium source supplies calcium ions directly, reducing the need to dissolve additional portlandite. Lower specific gravity can increase pozzolanic consumption because porous materials offer more nucleation and growth sites during early hydration.

Experiment

The experiments evaluate a pipeline that uses large language models to mine cement and concrete literature into a typed materials database, then applies machine learning to predict cementitious reactivity from chemical and physical descriptors. Interpretability analyses and comparisons with established material behavior show that composition, amorphous content, age, and particle properties are key drivers of predicted reactivity. Applying the model to secondary and waste materials identifies pozzolanic and hydraulic candidates, including construction, demolition, and municipal wastes, while screening a global rock database reveals naturally occurring precursors concentrated in volcanic and tectonic regions, supported by literature and historical evidence.