A prediction model that does not update when new measurements come in is not a model for discovery. It is a lookup table with confidence intervals. The whole point of coupling a surrogate model to an experimental campaign is that each measurement should change what you do next, and specifically, it should change the model's beliefs about regions of the composition space it has not yet sampled.
Loop closure is the mechanic that makes this happen. When a synthesis run returns characterization data, those measurements become new observations that update the posterior distribution of the surrogate. The updated posterior changes which candidates rank highest in the next acquisition function evaluation. The loop: predict, select, synthesize, characterize, update, predict again.
In practice, running this loop well requires decisions at every step that are not obvious from the textbook description. This post covers the ones we encounter most often.
What a Posterior Update Actually Does
In a Gaussian process surrogate, adding new observations does two things to the posterior. First, it reduces variance at and near the observed composition. The model becomes more confident about predictions in that neighbourhood. Second, it shifts the mean prediction in regions that are correlated with the new observation. If you observe a higher-than-predicted value at composition A, the mean prediction will rise slightly at compositions similar to A, with the magnitude of the shift scaling with the kernel correlation coefficient between A and its neighbors.
The second effect is the more important one for discovery. It means a single well-placed measurement carries information about many nearby compositions that you did not directly measure. In a smooth property landscape, a measurement at one point in a sparse region can update the mean prediction for 50 to 100 neighbor candidates simultaneously. This is why active learning in a GP framework can be more efficient than random sampling: each query is placed to maximally compress the total posterior uncertainty, not just to fill a grid.
For tree-based models (gradient boosted trees, random forests), there is no exact Bayesian posterior update. You retrain the model from scratch on the full dataset including the new observations. This is computationally cheap for datasets below a few thousand points and it works well in practice, but it lacks the propagation-of-information property that makes GPs so useful in small-data settings. If you are running a tight budget campaign with 30 to 60 total experiments, the GP's information propagation is a meaningful advantage.
Measurement Noise and Its Effect on Update Quality
GP posterior updates assume you know the noise variance of your measurements. In practice this parameter is fit from the data using marginal likelihood maximization, but the quality of the fit depends on having at least a few replicate measurements at some compositions so the noise level can be estimated separately from the signal.
If your experimental protocol has no replicates and heteroscedastic noise (noise that varies across compositions, for instance because some synthesis conditions are harder to control than others), the GP noise model will be misspecified. In our experience, the practical consequence is posterior overconfidence: the model thinks it knows more than it does after an update. Predictions in regions near observed compositions will have artificially narrow intervals.
The fix is to include 10 to 15 percent duplicate measurements in your initial experimental design, not as wasted budget but as noise calibration points. This gives you an empirical noise estimate that you can use to regularize the GP likelihood, preventing overconfident posteriors after early-campaign updates when data is still sparse.
How Many Cycles Before Confidence Stabilizes
The question we get most often from materials teams starting their first active learning campaign is: how many update rounds before the model is good enough to trust for synthesis decisions?
The honest answer is that it depends on three factors: the smoothness of the property landscape, the initial dataset size, and the measurement noise level. For a 4 to 6 component system with smooth property variation, an initial design of 20 to 30 points, and moderate measurement noise (coefficient of variation around 5 to 8 percent), we typically see the following pattern across update cycles.
After round 1 (8 to 12 new measurements): mean absolute prediction error on held-out validation points drops by 20 to 30 percent. Posterior variance in previously unsampled regions drops substantially. The top-ranked candidates shift as new information reshapes the acquisition landscape.
After round 2 (another 8 to 12 measurements): error continues to drop, typically another 15 to 20 percent. More importantly, the ranking of candidates in the high-performance region stabilizes: the same 5 to 8 compositions appear near the top of the acquisition function regardless of which iteration of the model you use. This stability is the signal that the model has converged on the shape of the performance landscape.
After round 3: incremental improvement is typically small unless the round-2 experiments revealed a new high-performance region that was previously unexplored. If the Pareto front did not move substantially in round 2, round 3 will not move it much either, and it is usually time to shift from active learning to a focused synthesis of the top candidates.
For a rougher landscape with multiple local optima, or for a higher-dimensional composition space, convergence takes longer. We would not declare a model converged until the top-5 ranked candidates are stable across two consecutive update rounds.
When New Data Contradicts the Prior
The most disruptive event in an active learning campaign is a measurement that strongly contradicts the model's prediction. You predict 620 MPa yield strength; you measure 410 MPa. The residual is large relative to the posterior standard deviation. What does this mean and what do you do?
First, check the measurement. Large residuals are often measurement outliers: synthesis batch contamination, incorrect specimen geometry, or characterization instrument error. Before updating the model with a high-residual observation, verify it with a replicate if possible. Running a replicate on a 5 to 8 sigma outlier is almost always worth the cost because the alternative is updating your posterior with bad data, which can derail the campaign for 2 to 3 rounds.
Second, if the measurement is verified, treat it as a signal that the model's prior assumptions are wrong in that region. A Matern 5/2 kernel assumes a specific spatial correlation scale. If the property drops sharply near a phase boundary that the model did not know was there, the GP will be surprised by the measurement because the kernel assumes smoother variation than exists at that boundary. The update will widen the posterior in the surrounding region, which is the correct response: the model should become less confident where it was wrong, not more. But you may need to query more densely near the outlier region in the next round to determine whether the sharp feature is a phase boundary or an isolated defect.
Data Provenance and the Version Problem
A closed experimental loop over 3 to 4 rounds generates 50 to 80 new data points alongside the initial dataset. By the end of the campaign, you have a growing database of characterization results, each linked to a specific synthesis batch, processing record, and characterization protocol. If any of these attributes changes mid-campaign (a different furnace, a technician change that introduces handling differences, a revised DSC protocol), you have a covariate shift in the data that will confuse the model.
We maintain strict versioning on synthesis and characterization protocols across a campaign, and we tag every measurement with the protocol version. If a protocol changes, new measurements are treated as a separate data source and the model is checked for consistency between old and new protocol measurements before mixing. This overhead is small: tagging protocol version takes 30 seconds per measurement. The alternative is a corrupted dataset that takes days to diagnose when predictions go wrong in round 3.
The experimental loop only closes cleanly if the data coming back into it is trustworthy and well-characterized. Model updating mechanics are the easy part. Data provenance is where most real campaigns run into trouble, and it is the part that no amount of acquisition function design will fix once the data is already mixed.