Meetings

23/11/22 - Kick off

Ghislain V., Thomas A., Achille M.

  • Keep it simple first
  • Evaluation: from weather stations, test on an area with a lot of weather stations
  • Evaluation climate data vs baseline climate data
  • Interpolation
    • bilinear is commonly used
  • Achille previously work downscaling from all available, bioclim computing, and evaluation, for AFRICA and 15 RCM, shell + CDO + wget + GRASS
  • Project differences in moist tropical forest distribution between GCM and RCM
  • Finally provide a tool and a product
  • Have a look to CORDEX downscaling on Europe and North America
  • Maybe add Swiss

23/12/05 - Dev summary

Dev points:

  • File naming, for instance longest is results/evaluation/{country}_{domain}_{gcm}_{rcm}_{rcp}_{period_eval}_{period_base}.tsv
  • Modularity:
    • Space: currenlty country level based on GADM, but should be adapted to work at regional scale, global scale, and defined bounding box. Resolution of the baseline.
    • Time: currently resolution of months. I would keep finer resolution, i.e. days or x-hourly, for another version (but maybe more complicated to implement with current structure). I do not think year resolution is useful. Period depend on the different products, for instance beware of the short period of evaluation of CHELSA (2010) which can question the comparison with other such as WorldClim
    • Baseline: currently CHELSA 2.1, add WorldClim? add ERA5-Land?, ready for more?
    • Projection: currently CORDEX, add CMIP5 for comparison, add CMIP6?, add CMIP6 HighRes?
    • Scenarios: all RCPs, add SSPs (depend on CMIP6)?; linked to projection
    • Downscaling: bias correction based on delta / change factor of anomalies only? I would keep more sophisticated quantile based approach for another version (but maybe more complicated to implement with current structure)
    • Pre-selection: currently none. Could be based on climatic range of projections in the defined area. But it is needed for lack of computing power, which I think is not the case here.
    • Evaluation: all available metrics, automatic report. Which data? Currently same as the baseline used. External data could be added for multiple evaluations, e.g. CRU TS 4.
    • Post-selection: currently none. Could be based on previous evaluation. But I do not think it is useful by itself, but could be included eventually in ensemble.
    • Ensemble: currently none. Should be added for general users relying on ensemble projections, while reporting uncertainties (very important). Could be simple multi-model averaging or more complex approaches including Bayesian model averaging.

Python questions/devs:

  • Environment management local / Rstudio / conda with mamba-forge
  • Local / Rstudio envs conflict
  • Very slow conda envs (but working for country)
  • Help on online nc reading from CHELSA or esgf
  • Help on xarray use, but currently pretty autonomous
  • Fresh installation tutorial
  • Updating management of HPC with profiles in snakemake
  • Python/R script for automatic experiment table creation with available CORDEX

23/12/05 - Thomas A.

Dev:

  • Use any filenaming according to the dev, but include all key informations as metadata in the NetCDF data files and TSV evaluation files
  • Use all CORDEX and ESGF terminology for projections (including ensemble etc)
  • Check GADM ability to work with continent, and develop the specific case of defined bounding box
  • Define the projection in configuration with default available projection common for all outputs in the project
  • Develop architecture (but not code, include stops) for daily and x-hourly data
  • Develop architecture (but not code, include stops) for statistical / quantile-based downscaling
  • Develop architecture (but not code, include stops) for pre- and post- selection
  • Develop architecture & code for CHELSA and WorldClim baseline, while developing generic getter architecture for further products (e.g. ERA5-Land, but each product will need its own getter). The resulting NetCDF file should be the cleanest as possible and generic. The working unit will be a NetCDF file with all variables across the whole area and the whole time period.
  • Gather summarise_cordex, get_anomalies & downscale in a single bias-correction downscaling rule
  • Develop architecture and code for evaluation based on the baseline and CRU TS 4 as independent data, while developing generic getter architecture for further products (e.g. ERA5-Land, but each product will need its own getter). Should be similar or same rules as the one used to get the baseline.
  • Develop architecture and code for ensemble projections based on simple multi-model averaging producing both ensemble and uncertainties, and develop architecture (but not code, include stops) for more sophisticated method such as Bayesian model averaging.

Notes:

  • Use Visual Studio for python dev
  • Do not use pip for libraries
  • Figure out how reticulate deals with environments, and which environment is used in Rstudio terminal
  • Use conda with mamba forge as recommended for snakemake
  • Re-install snakemake in a conda environment as recommended
  • Use a personal dev environment in yaml to be shared with co-authors for tests and developments
  • Use single rule minimal environments for rules libraries
  • Ask for help on specific rules scripts on a continuous-flow basis using if possible GitHub issues

24/01/12 - Thomas A.

  • Global grid to define (on baseline?)
  • xECMF with xe.Regular instead of nc.to_lonlat
  • Beware of downscale metadata with lost info
  • Evaluation as netcdf
  • Create monthly mean files with intermediate rule for temporal aggregation
  • get_proj into get_cordex and get_cmip
  • only 1 get_cordex per domain for all countries (same for get_chelsa and other baselines)
  • add a rule to produce table of country per domain based on python library cordex
  • Evaluation plots as R and suggest to be built as python
  • Ask help to Thomas for workflow structure with countries
  • Ask help to Thomas for figures in python
  • Ask help to Thomas for dictionaries in netcdf metadata
  • Schedule a meeting with Ghislain and Achille

24/04/08 - Ghislain, Achille, Thomas

  • Test the difference of bias correction then downscaling versus downscaling then bias correction
  • Prepare a description to test and share
  • Next meeting on th 24/05/03 8:30