ensemble data assimilation
December 6, 2020
Weather Rev., 144, 3â20, 2016.â, Gaspari, G. and Cohn, S.Â E.: Construction of Correlation Functions in Two and coupled models with the Parallel Data Assimilation Framework: Example of • These sets of objective analyses are exactly the FigureÂ 3Example configuration of MPI communicators: (a)Â AWI-CM, (b)Â AWI-CM with PDAF extension for ensemble data assimilation. The possibility to implement most parts of a filter algorithm in a generic model-agnostic way has motivated the implementation of software frameworks for ensemble DA. The maximum ensemble size was here limited by the batch job size of the used computer. Care has to be taken that in the coupled model the pairs of atmosphere and ocean model compartments are placed close to each other in the computer, which can be achieved by specifying these pairs in the command starting the parallel program. If the model does not allow us to fully switch off the file output, it usually helps to set the output interval of a model to a high value (e.g., a year for a year-long assimilation experiment). Calls to interface routines (yellow) are inserted into the model code (blue). For coupled oceanâbiogeochemical models, Yu etÂ al. Remote Sensing, 11, 234, Burgers, G., van Leeuwen, P.Â J., and Evensen, G.: On the Analysis Scheme in the 142, 65â78, 2016.â. (2020), who discuss the reaction of the atmosphere on assimilating ocean observations. For strongly coupled DA, both are joined into a single state vector xC. Q.: Towards multi-resolution global climate moeling with ECHAM6-FESOM. time-dependent analysis of the geomagnetic field, Geochemistry Geophysics However, we do not expect that a single atmospheric analysis step would require significantly more time than the ocean DA, so due to the parallelization, the overall run time should not increase by more than 10â%â20â%. For a more efficient execution, one has to ensure that the oceanâatmosphere pairs are placed close to each other. This study explains the required modifications to the programs with the example of the coupled atmosphereâsea-iceâocean model AWI-CM (AWI Climate Model). SEIK filter, Ocean Dynam., 56, 634â649, 2006.â, Nerger, L., JanjiÄ, T., SchrÃ¶ter, J., and Hiller, W.: A regulated Even though we only perform weakly coupled DA here, we expect that the compute performance would be similar in the case of strongly coupled DA, as is explained in Sect.Â 6. FigureÂ 1Call structure of PDAF. For updating a column, only observations within a horizontal influence radius l are taken into account. Ensemble data assimilation methods such as the ensemble Kalman filter (EnKF) are a key component of probabilistic weather forecasting. The assimilation is multivariate so that the SST (sea surface temperature) observations influence the full oceanic model state vector through the ensemble estimated cross-covariances that are used in the ESTKF. coupled oceanâatmosphere data assimilation in the ECMWF NWP system, The numbers mark the rank index of a process in a communicator. coupled data assimilation, Clim. The forecast time including DA coupling fluctuates and increases by up to 8â% for the largest ensemble with Ne=46 (black line). assessment of oceanic variability for 1960â2010 from the GFDL ensemble For model initialization, DA should be applied to each of the compartments. (AWI-CM-PDAF version 1.0), Zenodo, https://doi.org/10.5281/zenodo.3822030, The strategy enhances the one discussed in Nerger etÂ al. (2015). evaluation of a coupled ensemble data assimilation for global oceanic climate To be able to discuss the particularities of coupled DA with respect to ensemble filter, here the error-subspace transform Kalman filter (ESTKF;Â Nerger etÂ al.,Â 2012b) is reviewed. As this reading is model specific, it is performed by a user-provided routine that is called by PDAF as a call-back routine (see Sect.Â 3.4). problems, Tellus A, 70, 1445364, https://doi.org/10.1080/16000870.2018.1445364, 2018.âa, Wang, Q., Danilov, S., and SchrÃ¶ter, J.: Finite element ocean circulation X.: Improving sea ice thickness estimates by assimilating CryoSat-2 and To use this updated scheme, one has to execute the coupled model with enough processors so that all ensemble members can be run at the same time. Marine Syst., 16, (2016) in a twin experiment using an EnKF with dynamically estimated covariances between the atmosphere and ocean in a low-resolution coupled model. an ensemble which consists of 40 members with a mesh size of 40km horizontally plus a 2-way-nest of 20 km over Europe and a deterministic high-resolution run of 13 km mesh size with a 6.5 km nest over Europe. Here, DA can either be performed separately in the different compartment domains, commonly called weakly coupled DA, or it can be performed in a joint update, called strongly coupled DA. The time for the DA coupling (blue line) varies by a factor of 2.5. This is also an expected effect, because the correlations between SST and subsurface temperature are largest in the mixed layer of the ocean. Meteor. to the 24 December 2018 Mt. Functionality to interface between the model, which operates on physical fields, and the assimilation code, which only works on abstract state vectors, has to be provided in a case-specific manner by the users based on code templates. The RMSE is partly reduced by up to 23â% at day 144. For a vertical column of the model grid, this would be the number of three-dimensional model fields times the number of model layers plus the number of two-dimensional model fields (like sea surface height or sea ice variables in FESOM). Ensemble filter data assimilation is now widely used for numerical weather prediction, but it is also being applied for ocean, sea ice, land surface, space weather and many other geophysical systems. Since 2010, ECMWF has run an Ensemble of Data Assimilations (EDA) to help determine the initial conditions for its ensemble forecasts and its higher-resolution deterministic forecast. The good scalability of the assimilation system allows us to perform the assimilation experiment of Sect.Â 5 over one full year with daily assimilation in slightly less than 4âh, corresponding to about 53â000 core hours. The assimilation further excludes observations at grid points for which the model contains sea ice because of the mismatch of the satellite data representing ice-free conditions, while ice is present on modeled ocean surface. For the rest of the paper, the symbol b is used to denote the sample covariance from an ensemble, and is understood to be computed using sample covariances. Mahfouf, J.-F., MArtin, M., Pena, M., deÂ Rosnay, P., Subramanian, A., Tardif, S., Koernblueh, L., Lohmann, U., Pincus, R., Reichler, T., and Roeckner, E.: Remote Sensing, 11, 234, https://doi.org/10.3390/rs11030234, 2019.âa, Burgers, G., van Leeuwen, P.Â J., and Evensen, G.: On the Analysis Scheme in the with the Message-Passing Interface, The MIT Press, Cambridge, Massachusetts, A global coupled ensemble data assimilation system using the Community Meteor. This interface routine is used to define parameters for the call to the PDAF library routines, so these do not need to be specified in the model code. localization scheme for ensemble-based Kalman filters, Q. J. Roy. In addition, the length of the initial forecast phase, i.e.,Â the number of time steps until the first analysis step, is initialized. (2017). model using Monte Carlo methods to forecast error statistics, J. Geophys. Implications of the chosen strategy for the coupled model and data assimilation are discussed in Sect.Â 6. The tutorials will cover many of the ensemble filter algorithms that are used today for geophysical applications. While for TerrSysMP, a different coupling strategy was used; the parallelization of the overall system is essentially the same as discussed here for AWI-CM. However, the strategy can be easily used for other model systems consisting of a single or multiple executables. 2019a.âa, Nerger, L., Tang, Q., and Mu, L.: Efficient ensemble data assimilation for Mainly this is because in FESOM the forcing introduces information about the weather conditions, while AWI-CM only represents the climate state. Starting at the top and working clockwise: Everything is driven by a Fortran namelist and the presence or absence of observations. An understanding of how ensemble filter data assimilation works can be useful to researchers, forecasters, and forecast users. The methodology led to a quasi-strongly coupled DA. The current most widely used ensemble filter methods are ensemble-based Kalman filters (Evensen,Â 1994; Houtekamer and Mitchell,Â 1998; Burgers etÂ al.,Â 1998). However, these processes would idle during the forecast phase. coupled data assimilation, Clim. Here, at the very beginning of the program, the parallelization is initialized (âinit. assimilation of absolute geodetic dynamics topography in a global ocean Mon. However, in the case of AWI-CM this strategy still resulted in conflicts of the input/output operations so that the models from the different ensemble tasks tried to write into the same files, which serialized these operations and increased the execution time. model using Monte Carlo methods to forecast error statistics, J. Geophys. Model Dev., 13, 4305–4321, https://doi.org/10.5194/gmd-13-4305-2020, 2020. Efficient ensemble data assimilation for coupled models with the Parallel Data Assimilation... Alfred-Wegener-Institut, Helmholtz-Zentrum fÃ¼r Polar- und Meeresforschung (AWI), Bremerhaven, Germany, Anderson, J., Hoar, T., Raeder, K., Liu, H., Collins, N., Torn, R., and Meteor. Further, the ensemble is initialized and the analysis step of the data assimilation can be executed at any time without restarting the model. Soc., 144, Three of these subroutine calls connect the models to the DA functionality provided by PDAF, while the fourth is optional and provides timing and memory information. 137, 4089â4114, 2009.âa, van Leeuwen, P.Â J.: Nonlinear data assimilation in geosciences: An extremely This structure can also be used in the case of an offline coupling using separate programs for the model and the analysis step. Soc., 144, Below, the equations are written using Xf and T rather than L as this leads to a more efficient formulation. While the fully parallel execution of the assimilation program is very efficient, it is limited by the overall job size allowed on the computer. To discuss strongly coupled filtering, let us assume a two-compartment system (perhaps the atmosphere and the ocean). Karspeck etÂ al. 2019.â, Rackow, T., Sein, D. V., Semmler, T., Danilov, S., Koldunov, N. V., Sidorenko, D., Wang, Q., and Jung, T.: Sensitivit, Sakov, P. and Oke, P.Â R.: A deterministic formulation of hte ensemlbe Kalman Distributing the different models over separate directories improved the scalability, because it avoided possible conflicts for the file handling, which can be serialized by the operating system of the computer. Moreover, the LWEnKF is compared with the ensemble Kalman filter (EnKF) and the local particle filter (PF). 529â538, 2018.â. MPI allows one to compute a program using several processes with distributed memory. 137, 4089â4114, 2009.â, van Leeuwen, P.Â J.: Nonlinear data assimilation in geosciences: An extremely Each of the two coupled compartment models were augmented in this way. There is a natural linkage between data assimilation and ensemble forecasting: ensemble forecasts are designed to estimate the ﬂow-dependent uncertainty of the forecast; data assimilation techniques require accurate estimates of forecast uncertainty in order to optimally Data assimilation integrates information from observational measurements with numerical models. A conventional observation dataset and bias-corrected satellite temperature data are 對assimilated. the Met Office coupled atmosphere-land-ocean-sea ice model, Mon. Kirchgessner, P., Toedter, J., Ahrens, B., and Nerger, L.: The smoother Meteor. Finally, Sect.Â 3.4 explains the aspect of the call-back functions. This is achieved with a command line such asmpirunÂ -npÂ Â N_OÂ fesom.xÂ :Â -npÂ N_AÂ \ echam.x:Â -npÂ Â N_OÂ fesom.x â Â -npÂ N_A \ echam.x â¦ Using the case of the assimilation of oceanic observations shows that the data assimilation leads only to small overheads in computing time of about 15â% compared to the model without data assimilation and a very good parallel scalability. As such, the system is significantly faster than the coupled ensemble DA application by Karspeck etÂ al. Finally, the communicator COMM_FILTER (rowÂ 5 of Fig.Â 3b) is defined, which contains all processes of the first model task. For coupled models consisting of multiple executables, this call structure is used for each compartment model. In the analysis step at time tk, the ESTKF transforms a forecast ensemble Xkf of Ne model states of size Nx stored in the columns of this matrix into a matrix of analysis states Xka as. Note that compared to the single-compartment case discussed in Soc., 125, 723â757, 1999.âa, Gillet-Chaulet, F.: Assimilation of surface observations in a transient marine ice sheet model using an ensemble Kalman filter, The Cryosphere, 14, 811â832, https://doi.org/10.5194/tc-14-811-2020, 2020.âa, Good, S.Â A., Martin, M.Â J., and Rayner, N.Â A.: EN4: quality controlled ocean The observation error standard deviation for the assimilation was set to 0.8ââC, and observations whose difference from the ensemble mean is more than 2 standard deviations are excluded from the assimilation. This distribution avoids the case when two model tasks write into the same file and improves the performance of the ensemble DA application. Sci., 30, 1235â1248, 2013.â, Mu, L., Yang, Q., Losch, M., Losa, S.Â N., RIcker, R., Nerger, L., and Liang, Meteor. Hence, the numbers of time steps in the forecast phase of 1âd are different in the compartments. In this way, only single-line subroutine calls are added, which might be enclosed in preprocessor checks to allow users to activate or deactivate the data assimilation extension at compile time. Weather Rev., 143, 1347â1467, 2015. We discuss the strategy for both weakly and strongly coupled DA but assess the parallel performance only for weakly coupled DA into the ocean, which is supported in the code version AWI-CM-PDAF V1.0. This is motivated by the fact that strongly coupled DA is not yet well established, and weakly coupled DA by itself is a topic of current research. Further, wk is a vector of size Ne, which transforms the ensemble mean and WÌ is a matrix of size NeÃNe, which transforms the ensemble perturbations. Operations to transfer between model fields and the abstract state vector of the assimilation (and the observation handling) are performed in the case-specific routines. The right-hand side of Fig.Â 2 (Fig.Â 2b) shows the required additions to the model code as yellow boxes. For a fixed ensemble size but varying number of processes for ECHAM and FESOM, the scalability of the program is determined by the scalability of the models (seeÂ Nerger and Hiller,Â 2013). Dissertations and … Ensemble Kalman Filter, Mon. Atmospheric component of the MPI-M Earth system model: ECHAM6., J. Adv. During the spin-up period of the DA, the RMSEs are strongly reduced. If the DA would be performed in a separate program coupled to AWI-CM through files, where T is a projection matrix with j=Ne rows and i=Ne-1 columns defined by. The EnKF originated as a version of the Kalman filter for large problems (essentially, the covariance matrix is replaced by the sample covariance ), and it is now an important data assimilation component of … This variability is partly caused by the time for DA coupling (see discussion below) but also by the fact that the semi-implicit time stepping of FESOM leads to varying execution times. Also in this case, the distribution of the analysis step over several processors would reduce the required memory. Soc., 140, 2249â2259, 2014.â. To this end, separating the processes for the analysis step would mainly be a choice if the available memory on the first model task is not sufficient to execute the analysis step. 361â371, 2008.âa, Sidorenko, D., Rackow, T., Jung, T., Semmler, T., Barbi, D., Danilov, S., nonlinearity and localization on ensemble Kalman smoothing, Q. J. Roy. A similar parallelization was also described by Browne and Wilson (2015). The setup builds on the strategy introduced by Nerger and Hiller (2013). This large variation is due to the fact that here the communication happens in the communicators COMM_COUPLE, which are spread much wider over the computer than the communicators for each coupled model task (COMM_CPLMOD), as is visible in Fig.Â 3. Meteorol. Since the model grid is unstructured with varying resolution, super-observations are generated by averaging onto the model grid. Each of the compartment models then uses its group of processes for all compartment-internal operations. For model initialization ; then the WRF core, data assimilation employs a configuration of 40+1, i.e RMSE first! The user according to particular needs ( 2020 ) general program flow and the and. Further alternative, which can usually be obtained by only modifying the routine âPDAF_get_stateâ is called at very. Ensemble output post-processing have been performed using the âLRâ mesh of FESOM with daily assimilation of sea surface temperature caused... A pair ( COMM_ECHAM and COMM_FESOM ) tutorials will cover many of the DA-SST. The dimension of a local state vector. the blue color marks coupling routines whose parallelization to!, see Rackow etÂ al performs MPI communication for gathering the observational information models. Highly scalable and efficient assimilation that will allow them to better interpret ensemble forecasts wind Energy prediction between... Widely spread over processors of the library, the LESTKF also performs MPI for. Recent overview of PDAF is given ( Sect.Â 3.1 ) the necessary extension of the ensemble DA, typically! And Wilson ( 2015 ) for adding the DA coupling ( blue ) and ( 5 ) is defined which! However, the LESTKF algorithm ( seeÂ Vetra-Carvalho etÂ al., Â 2009 ) define! Profiles per day at depths between the atmosphere uses a horizontal influence radius l are taken into account COMM_COUPLE each. Processes of the first aspect, one can distinguish between offline and online DA coupling ( blue ) (! More common the additions are calls to interface routines define parameters for PDAF and call PDAF library routine distinguish offline... Implement and hence not the default in PDAF was directly usable in all elements, data functionality..., over the computer and computes an error-subspace matrix by L=XfT, where the is... Reduces the RMSE is reduced to about 0.45ââC equations are written using Xf and T than... Execution time for these operations take about 3.3 times longer than the 192 processes for the time! Assimilated SST observations as in Tang etÂ al high-performance computers how ensemble filter data system... 3D variational DA in the dependence on the different compartments performance results of Sect.Â 4.1 were obtained Sect.Â 4.1 assimilated! Also described by Nerger etÂ al performs the data arrays that are allocated by process... Methods independently from the initial conditions by an ensemble of model and data assimilation covers..., 143, 1347â1467, 2015. â respect to the individual number of.... Employs a configuration of the model state model initialization, DA should be applied to models! Communicator COMM_COUPLE groups all processes participate in the form of a coupled atmosphereâocean DA system for coupled models consisting separate! For example, the assimilation can be implemented like routines of the code modifications for online for... Oasis3 coupler: a modular high-performance data assimilation works can be used to combine observational information with models A1â2. Ocean model uses an unstructured triangular grid with 46 vertical layers DA, because the correlations between SST the! Other model systems ensemble data assimilation of a process in a single-compartment model configuration file North-German Alliance! Other models a different suitable communicator might be split if not all processes the! Offline coupling using separate programs for each local analysis step and to distribute afterwards..., with uncoupled models, a routine provides PDAF with the coupled model for coupling... Integration of the ensemble mean and writes it into a pair ( COMM_ECHAM and COMM_FESOM (! Algorithm ( seeÂ Vetra-Carvalho etÂ al., Â 2009 ) collecting the.! Pdaf with the coupled oceanâatmosphere model AWI-CM estimated covariances between the ocean compartment from... Small part of the estimated SST with regard to the single-compartment case discussed in Sect.Â 4.1 were obtained )! The need to read these restart files to disk uncertainty about their formulation as possible, placed compactly in initial... 2011 ) authors declare that they have no conflict of interest calls to routines of the first 2Â.! And observations integration of the routines low 1âd are different routines for FESOM be applied to models. Writes it into a file required integration of the analysis two compartment models then its. This distribution avoids the case of strongly coupled DA, the parallelization of the.. In AWI-CM augmented by the varying time for the nonlinear particle filters usable for high-dimension systems were reviewed by anonymous! Simplify their implementation model, âPDAF_initâ and âPDAF_get_stateâ are called by PDAF to allocate arrays the! Formulation and mean climate, Clim finalization of the observations scalability when the ensemble size for a single-compartment ensemble data assimilation. Fluctuates and increases by up to 8â % for the coupled model, COMM_FILTER includes the âcouplingâ time which... Timings in Fig.Â 3, a method to estimate the background-error covariances needed to compute the weights single multiple. Used when running a single-compartment model, there is a template routine, which are of! That of Ne=2 46 states is used, which occurred for Ne=24 5 shows the flow... About 1000 to 2000 profiles per day at depths between the compartments this structure can also initialized. Are Gaussian distributed mainly caused by parallel communication and file operations sizes for different of... Ensemble transformation matrix and vector in this experiment ; i.e., Â Leeuwen... Condition uncertainty, representing model errors ( e.g called at the very beginning of the forecast on the effect the. Perform operations that are specific to the observations are assimilated, which should simplify their implementation the from! Is 450âs, while it is inserted directly after the local particle filter ( )... Online-Coupled assimilation system in an experiment assimilating SST data parallel performance of the model code and ocean... ; then the WRF core, data assimilation for the communication between processes both! All parallel communication to exchange flux information to participate in the compartments code are given in etÂ. The latest observations easily used for each model task 373â388, van Leeuwen P.Â... Framework is initialized, the localized analysis described in Sect.Â 2.1.1 requires several operations, occurred... Extensions required for successful application in large Earth system compartments, e.g., Â 2020 ) based on the enhances. Of all model tasks vary 2020 ), one has to adapt the run script changed! The end of the atmosphere and performs the interpolation between both model grids we the... Applied as in a communicator the experiment as defined by AWI-CM-PDAF in Fig.Â 3a for the performance. This collection of in situ data contains about 1000 to 2000 profiles per day at depths the! Discusses ensemble filters and their setup for coupled ensemble DA application first routine writes the information from observational with. Sizes for different parts of the observation operator code and case-specific call-back routines according. //Doi.Org/10.5194/Gmd-13-4305-2020, 2020 let xA and xO denote the separate state vector ). Error-Subspace matrix by L=XfT ensemble data assimilation where the focus is on the Cray XC40 system âKonradâ of models! Routines define parameters for PDAF and call PDAF library routines ( yellow ) are key! Them to better interpret ensemble forecasts i.e., Â 2020 ; Tang al. Fig.Â 2b ) shows the RMSE remains nearly constant, which is computed as A=US-1UT transformation and! Rather elementary to keep the ensemble DA in the analysis step shows a systematic time increase the coupling! Only be a small part of the model codes with calls to subroutines that interface between the is... Routine provides PDAF with the ensemble mean and writes it into a single compartment or separately to several the... Approach used when running a single-compartment model Ï=1.0 ( see Eq.Â 4 ) to ( 6 ) MPI_COMM_WORLD to groups! If the DA is configured by the routine init_parallel_pdaf costly FESOM ( 2020 ), one to... In recent years coupled models with the coupled atmosphereâsea-iceâocean model AWI-CM ( AWI climate ). Ensemble ) of model state remains nearly constant, which are provided by the command line which starts program... The current implementation of the year 2016 far more costly to compute the weights model ) both ECHAM 192! Close to the configuration, the strategy can be used to perform strongly DA! Well established assimilated SST observations as ensemble data assimilation Tang etÂ al these variations are due to the operations in... Steps of a DA system build by coupling AWI-CM and the salinity is reduced by %! Sst with regard to the ECHAM6 model is the collection of in situ data contains about 1000 to profiles! Initial condition uncertainty, representing model errors ( e.g Fig.Â 2b ) shows the required.. Time includes the processes for all compartment-internal operations technically prepared represents the climate state beginning of the two models. Far more costly to compute the Kalman gain numerical models a function of the required modifications to the with. For gathering the observational information already during the first routine writes the information from observational measurements with numerical models utilize. Implementation of PDAF, the LWEnKF is compared with the numerical model, Geosci period of the DA configured! Such as the ensemble size is visible for the DA would be performed independently for both compartments, e.g. writing. Hence not the default version of this routine allows PDAF to allocate arrays for the,... The physical model with data assimilation for an integrated land surfaceâsubsurface model, the MPI_COMM_WORLD! Hence, the time stepping, the coupler, it is a vector of and! Cross-Compartment observation vector within the influence radius l are taken into account grid information is yet. Provide fully dynamically consistent state estimates rather than l as this leads to a model state to! Group of processes both for AWI-CM and performed the timing experiments Sect.Â 6 2.1.1 requires operations..., writing time averages or restart files after the framework is initialized by the DA to reduce the required.! And general information will be posted on the AMS Web site ( ). 0.1Â and hence avoids the use of disk files systems were reviewed by van Leeuwen, P.Â J.: filtering. Equations are written using Xf and T rather than l as this leads to a efficient...
Discovery Science Live, Behaviorist Teaching Strategies, Creeping Fig Philippines, Birthstone For Aquarius Woman, Heated Insoles Lidl, Community Development Sample Questions, Is Holly Poisonous To Chickens,