Benchmark

We quickly reported benchmarks of the tool for other users. Most rules takes less than a minute except the one retrieving data. CORDEX and CMIP6 retrieval takes around 7 minutes and almost always less than 10 minutes. CHELSA 2 retrieval takes more than 20 minutes but is only done once. Similarly most rules are using less than 1 GB of memory, except CMIP6 retrieval that approach 10 GB, CORDEX retrieval above 20 GB, and CHELSA 2 retrieval above 30 GB. Finally most rules write less than 1 GB, CORDEX and CMIP6 around 5 GB, and CHELSA more than 20 GB, but a lot are only temporary files.

Code

bc <- lapply(
  list.files("results/benchmarks/", full.names = TRUE),
  readr::read_tsv
) %>%
  bind_rows() %>%
  mutate(file = list.files("results/benchmarks/"))
bc <- bc %>%
  separate(file, c("rule1", "rule2")) %>%
  mutate(rule = paste(rule1, rule2)) %>%
  mutate(rule = recode(rule,
    "downscale bc" = "downscale",
    "eval C" = "eval",
    "eval French" = "eval",
    "eval New" = "eval",
    "eval Vanuatu" = "eval",
    "bias C" = "bias",
    "bias French" = "bias",
    "bias New" = "bias",
    "get area" = "area",
    "get chelsa2" = "CHELSA",
    "get chirps" = "CHIRPS",
    "get gshtd" = "GSHTD",
    "get cmip6" = "CMIP6",
    "get cordex" = "CORDEX",
    "hist base" = "hist",
    "hist ds" = "hist",
    "merge eval" = "merge",
    "merge hist" = "merge",
    "tmf C" = "tmf",
    "tmf French" = "tmf",
    "tmf New" = "tmf"
  ))
write_tsv(bc, file = "outputs/bencmark.tsv")

Code

read_tsv("outputs/bencmark.tsv") %>%
  ggplot(aes(s / 60, rule, label = `h:m:s`)) +
  geom_boxplot() +
  geom_point() +
  theme_bw() +
  scale_x_sqrt() +
  xlab("Minutes") +
  ylab("")

Code

read_tsv("outputs/bencmark.tsv") %>%
  ggplot(aes(max_vms / 10^3, rule)) +
  geom_boxplot() +
  geom_point() +
  theme_bw() +
  scale_x_sqrt() +
  xlab("Maximum Virtual Memory Size (GB)") +
  ylab("")

Rules Maximum “Virtual Memory Size”, this is the total amount of virtual memory used by the process.

Code

read_tsv("outputs/bencmark.tsv") %>%
  ggplot(aes(io_out / 10^3, rule)) +
  geom_boxplot() +
  geom_point() +
  theme_bw() +
  scale_x_sqrt() +
  xlab("GB Written") +
  ylab("")