The ML benchmarks that dominate the field use ImageNet, GPT pre-training corpora, or protein databases with millions of entries. Materials datasets look nothing like this. A carefully curated experimental dataset for a specific composition-property relationship in a mid-complexity materials system typically runs 80 to 400 measurements. The teams who assume they need "more data" before starting predictive modelling are often wrong. The teams who assume standard deep learning architectures will work are almost always wrong.
This post is about what actually transfers from large-scale ML to the small-dataset materials context, and what does not. We have worked with datasets ranging from 52 experimental points on a ceramic oxide system to around 380 measurements on a polyurethane formulation family. The lessons are different from what you read in most ML-for-materials tutorials.
Why Deep Learning Fails Below ~500 Points
Deep learning models with 10,000 to 10 million parameters learn by generalizing over many examples. With 150 experimental points and a 3-layer neural network, you have more parameters than data. The model will memorise the training set. Validation error will be high. You will get overconfident predictions in sparsely sampled regions.
The common fix suggested online is dropout, L2 regularization, or data augmentation. These help at the margins but do not solve the underlying problem. You cannot regularize your way to a well-specified posterior when the likelihood function is underdetermined by the data you have.
The practical implication: below 500 experimental observations, Gaussian process regressors and gradient-boosted trees (XGBoost, LightGBM) consistently outperform deep neural networks on materials property prediction tasks, especially on held-out test sets drawn from different compositional regions than the training data. The reason is inductive bias. GPs assume smooth Gaussian correlation structure; gradient boosted trees partition the feature space into locally linear regions. Both of these assumptions are more appropriate for composition-property relationships in most materials systems than the generic function approximation a deep network provides.
Feature Engineering Is Not Optional
A model is only as good as its input representation. For materials composition, raw molar fractions as inputs are a weak representation. They carry no information about atomic size, electronegativity, valence electron count, or oxidation state preferences, all of which are physically relevant to almost any property you care about.
The standard move is to compute composition-weighted elemental descriptors: weighted averages and standard deviations of periodic table features (Pettifor scale, Goldschmidt radius, Allen electronegativity, etc.) across all elements present in proportion to their molar fraction. The MAGPIE descriptor set, the Meredig feature set, and related approaches systematize this. We typically use a custom subset of 32 to 45 features selected based on physicochemical relevance to the property under study, combined with compositional statistics (mean, range, standard deviation).
For polymer systems, SMILES-derived fingerprints (Morgan fingerprints at radius 2 or 3) or graph-based representations of repeat unit structure outperform pure compositional descriptors, because the connectivity and sequence of monomer units matters for bulk properties like Tg and tensile modulus in ways that molar fraction alone cannot capture.
We are not saying feature engineering is a permanent solution. Graph neural networks trained on large multi-task datasets can learn representations that outperform handcrafted features. But in the small-dataset regime, the cost of good feature engineering is an afternoon, and the benefit is 15 to 30 percent better generalization on held-out data in most tasks we have tried.
Transfer Learning from DFT: Where It Works
One of the most useful tools for expanding an effectively small experimental dataset is pre-training on computed data. DFT databases like the Materials Project contain tens of thousands of computed formation energies, band gaps, and elastic constants. Transfer learning from these computed properties to experimental measurements can substantially reduce the experimental data requirement for achieving a given prediction accuracy.
The caveat is that the transfer is property-specific and phase-specific. Pre-training on DFT formation energies transfers well to experimental hardness models for intermetallic systems because the underlying physics (bonding strength) is related. It transfers poorly to electrochemical properties like ionic conductivity because DFT formation energy and conductivity are only loosely related at the thermodynamic level.
A practical workflow: train a model on all available DFT data for the relevant element space, then fine-tune on experimental measurements using a much lower learning rate and a frozen representation layer. With 60 to 80 experimental fine-tuning points, this approach can achieve accuracy comparable to a dataset 3 to 5 times larger trained from scratch. The proviso: the DFT pre-training data must cover the same compositional space as your experimental targets. Pre-training on oxides and fine-tuning on sulfides rarely helps because the electronic structure regimes are too different.
Cross-Validation Choices Matter Enormously
With small datasets, cross-validation is doing double duty: it is both your model selection criterion and your estimate of generalisation. How you construct the folds directly shapes what you conclude about model quality.
The common mistake is random k-fold CV. For materials composition datasets, random splits almost always put compositionally similar samples in training and test folds. If your dataset has 30 samples near one composition cluster and 15 samples near another, random 5-fold CV will have representatives of each cluster in both train and test. The measured CV error will be optimistic for interpolation tasks, and it will give you no information about extrapolation, which is usually the question that matters in a discovery campaign.
We use compositional clustering to group samples by similarity (hierarchical Ward clustering on the same descriptor set used for modelling), then hold out entire clusters. This cluster-based CV is more pessimistic and the error estimates are more honest. When we compare random CV to cluster CV on the same dataset, cluster CV error rates are typically 40 to 80 percent higher. The models that look best under cluster CV are the ones worth putting into production.
Uncertainty Quantification Is Part of the Prediction
A point prediction without an uncertainty estimate is half a prediction. For a discovery campaign, knowing whether the model is confident or uncertain at a candidate composition is as important as knowing what value it predicts.
Gaussian processes provide posterior predictive variance natively. For gradient boosted trees and other non-probabilistic models, conformal prediction intervals and ensemble-based approaches provide calibrated uncertainty. We use quantile regression forests as a secondary model specifically for uncertainty estimation when the primary model is a gradient boosted tree, because they are fast to train and reliably calibrated at the 90th percentile prediction interval on held-out cluster CV folds.
The practical use of this: any composition where the prediction uncertainty spans more than 30 percent of the property range is a poor candidate for synthesis planning decisions. It should either be queued for active learning (the model needs data there), or deprioritized if synthesis cost is high. Candidates with narrow uncertainty intervals in the high-performance region are where you want to concentrate synthesis effort.
How Many Points Do You Actually Need
The honest answer is: it depends on the dimensionality of the composition space, the smoothness of the property landscape, and the ratio of measurement noise to property range. As a practical heuristic based on what we have observed across several campaigns, 80 to 120 carefully placed experimental points in a 3 to 5 component system is usually sufficient to build a surrogate accurate enough to rank candidates for a focused synthesis campaign. Below 50 points, we would be cautious about using the model for anything beyond rough screening. Above 300 to 400 points, diminishing returns set in and the incremental cost of additional measurements rarely justifies the improvement in model accuracy.
The more important question is where the points are. 120 points drawn randomly from an 8-component composition space give you thin coverage and a poor surrogate. 80 points placed by a space-filling Latin hypercube design give you a far better model. Dataset size and dataset placement are both inputs to model quality, and placement is often the more controllable one.
Starting with an intentional initial design, using GP surrogates with calibrated uncertainty, and running a small number of active learning batch rounds will get you further than throwing more budget at random synthesis. That is the core principle behind how we think about data efficiency at Matforgelab.