Conditional Bias-Penalized Kalman Filter for Improved Estimation and Prediction of Extremes

Abstract

Kalman filter (KF) and its variants are widely used for real-time updating of model states and prediction in environmental sciences and engineering. Whereas in many applications the most important performance criteria may be the fraction of the times when the filter performs satisfactorily under different conditions, in many other applications the performance for estimation and prediction of extremes, such as floods, droughts, algal blooms, etc., may be of primary importance. Because KF is essentially a least squares solution, it is subject to conditional biases (CB) which arise from the error-in-variable, attenuation, effects when the model dynamics are highly uncertain, the observations have large errors and/or the system is not very predictable. In this work, conditional bias-penalized Kalman filter is developed based on CB-penalized linear estimation which minimizes a weighted sum of error covariance and expectation of Type-II CB squared, and comparatively evaluate with KF through a set of synthetic experiments for one-dimensional state estimation under the idealized conditions of normality and linearity. The results show that CBPKF reduces root mean square error (RMSE) over KF by 10 to 20% or more over the tails of the distribution of the true state. For nonstationary cases, CBPKF performs comparably to KF in the unconditional sense in that CBPKF increased RMSE over all ranges of the true state only by 3% or less. With the ability to reduce CB explicitly, CBPKF provides a significant addition to the existing suite of filtering techniques toward improving analysis and prediction of extreme states of uncertain environmental systems.