Assessing spatial transferability of a random forest metamodel for predicting drainage fraction
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Assessing spatial transferability of a random forest metamodel for predicting drainage fraction. / Bjerre, Elisa; Fienen, Michael N.; Schneider, Raphael; Koch, Julian; Højberg, Anker L.
In: Journal of Hydrology, Vol. 612, 128177, 2022.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Assessing spatial transferability of a random forest metamodel for predicting drainage fraction
AU - Bjerre, Elisa
AU - Fienen, Michael N.
AU - Schneider, Raphael
AU - Koch, Julian
AU - Højberg, Anker L.
N1 - Publisher Copyright: © 2022 The Author(s)
PY - 2022
Y1 - 2022
N2 - Fully distributed hydrological models are widely used in groundwater management, but model speed and data requirements impede their use for decision support purposes. Metamodels provide a simpler and faster model which emulates the underlying complex model using machine learning techniques. However, metamodel predictions beyond the ranges, in space and/or time, of training data are highly uncertain, and thus it is important to assess the predictive model performance to ranges outside the training data, i.e., model transferability. We present a novel methodology for evaluating model transferability to areas not contained in the training data set, based on various metrics that quantify the differences in covariate distributions between training and testing data. The transferability method can be employed as a screening tool to assess the suitability of a metamodel for spatial prediction beyond its training domain. We evaluated this transferability approach on a Random Forest metamodel of a 1000 km2 fully distributed coupled groundwater model for predicting drainage fraction, the partitioning of infiltrating water between drains and groundwater. We conducted spatial cross-validation on 9 holdout sub-basins to assess metamodel transferability beyond sampling locations and compared this estimate with a random split-sample validation test. Using mappable covariates only, the metamodel showed high performance (R2 = 0.79) tested on a 20% randomly sampled holdout. Conversely, metamodel performance significantly decreased for the 9 spatial holdouts (R2 ranging from 0.13 to 0.61). We document that the proposed transferability metric correlates with metamodel predictive performance, and demonstrate its use to assess model transferability to datasets outside the training data spatial domain.
AB - Fully distributed hydrological models are widely used in groundwater management, but model speed and data requirements impede their use for decision support purposes. Metamodels provide a simpler and faster model which emulates the underlying complex model using machine learning techniques. However, metamodel predictions beyond the ranges, in space and/or time, of training data are highly uncertain, and thus it is important to assess the predictive model performance to ranges outside the training data, i.e., model transferability. We present a novel methodology for evaluating model transferability to areas not contained in the training data set, based on various metrics that quantify the differences in covariate distributions between training and testing data. The transferability method can be employed as a screening tool to assess the suitability of a metamodel for spatial prediction beyond its training domain. We evaluated this transferability approach on a Random Forest metamodel of a 1000 km2 fully distributed coupled groundwater model for predicting drainage fraction, the partitioning of infiltrating water between drains and groundwater. We conducted spatial cross-validation on 9 holdout sub-basins to assess metamodel transferability beyond sampling locations and compared this estimate with a random split-sample validation test. Using mappable covariates only, the metamodel showed high performance (R2 = 0.79) tested on a 20% randomly sampled holdout. Conversely, metamodel performance significantly decreased for the 9 spatial holdouts (R2 ranging from 0.13 to 0.61). We document that the proposed transferability metric correlates with metamodel predictive performance, and demonstrate its use to assess model transferability to datasets outside the training data spatial domain.
KW - Drain partitioning
KW - Histogram distance
KW - Machine learning
KW - Model generalization
KW - Model portability
KW - Predictive modelling
U2 - 10.1016/j.jhydrol.2022.128177
DO - 10.1016/j.jhydrol.2022.128177
M3 - Journal article
AN - SCOPUS:85134575492
VL - 612
JO - Journal of Hydrology
JF - Journal of Hydrology
SN - 0022-1694
M1 - 128177
ER -
ID: 318196377