Name	Name	Last commit message	Last commit date
Latest commit History 2,987 Commits
20210603	20210603
20210604	20210604
20210605	20210605
20210610	20210610
20210611	20210611
20210618	20210618
20210624	20210624
20210625	20210625
20210703	20210703
20210704	20210704
20210709	20210709
20210714	20210714
20210715	20210715
20210723	20210723
20210726	20210726
20210727	20210727
20210730	20210730
20210806	20210806
20210813	20210813
20210819	20210819
20210824	20210824
20210826	20210826
ADAPTATIONS_EXAMPLES	ADAPTATIONS_EXAMPLES
RESULTS Alberta	RESULTS Alberta
RESULTS British Columbia	RESULTS British Columbia
RESULTS CANADA	RESULTS CANADA
RESULTS Manitoba	RESULTS Manitoba
RESULTS Nova Scotia	RESULTS Nova Scotia
RESULTS Ontario	RESULTS Ontario
RESULTS Quebec	RESULTS Quebec
RESULTS Saskatchewan	RESULTS Saskatchewan
journal articles, this work	journal articles, this work
longitudinal	longitudinal
sandbox	sandbox
situation reports	situation reports
LICENSE	LICENSE
README.md	README.md
readme.md	readme.md

CovidVisualizedCountry

Combine and visualize international periodically updated estimates of COVID-19 pandemic

at the country level, countries with subnational level estimates

🇨🇦 Canada: national level and provinces

Project: Combine and visualize international periodically updated estimates of COVID-19 pandemic at the country level (CovidVisualizedCountry) Countries with subnational estimates: Canada
Person: Farshad Pourmalek (pourmalek_farshad at yahoo dot com) || || PubMed

|| UBC SPPH || UW IHME || YouCheck
Time (initial): 2021-04-14

Pre-print for this project:

Farshad Pourmalek. CovidVisualized: Visualized compilation of international updating models’ estimates of COVID-19 pandemic at global and country levels, 02 August 2021, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-768714/v1]

PDF

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Related repositories:

THREE “CovidVisualized” repositories, use a common template and logic for visualization of the results of estimates of FIVE international and periodically updated COVID-19 pandemic models for the future of the epidemic:

CovidVisualizedGlobal for the GLOBAL level

CovidVisualizedCountry for countries with subnational estimates: CANADA

covir2 for countries without subnational estimates: IRAN

These results in these three repositories get periodically updated.

The codes in these repositories can be adapted for use for any country or region in the world.

For a sample application of “covir2” to a country without subnational estimates see Afghanistan, Pakistan

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Among the 5 available international periodically updated studies or models of COVID-19 pandemic, only TWO studies, DELP and IHME, provide subnational level estimates for some countries.

For Canada, DELP and IHME models provide subnational estimates for SEVEN provinces: Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, and Saskatchewan.

I. SELCTED GRAPHS FROM LATEST UPTAKE

LATEST UPTAKE: uptake 20210824

DELP 20210824, IHME 20210819, IMPE 20210819, LANL 20210822, SRIV 20210824

👀 SEE: Canada COVID-19 epidemic models situation report No 04 on 2021-08-24

Selected graphs from latest uptake - Canada

Selected graphs from latest uptake - Alberta

Selected graphs from latest uptake - British Columbia

Selected graphs from latest uptake - Manitoba

Selected graphs from latest uptake - Nova Scotia

Selected graphs from latest uptake - Ontario

Selected graphs from latest uptake - Quebec

Selected graphs from latest uptake - Saskatchewan

Abbreviations used in graphs:

(See Methods and Results for full details.)

DELP: DELPHI. Differential Equations Lead to Predictions of Hospitalizations and Infections. COVID-19 pandemic model named DELPHI developed by Massachusetts Institute of Technology, Cambridge

IHME: Institute for Health Metrics and Evaluation. COVID-19 pandemic model by developed Institute for Health Metrics and Evaluation, Seattle

IMPE: Imperial. COVID-19 pandemic model developed by Imperial College, London

JOHN: Johns Hopkins. Coronavirus resource center, Johns Hopkins University, Baltimore

LANL: Los Alamos National Laboratories. COVID-19 pandemic model developed by Los Alamos National Laboratories, Los Alamos

SRIV: Srivastava, Ajitesh. COVID-19 pandemic model developed by Ajitesh Srivastava, University of Southern California, Los Angeles

II. METHODS AND RESULTS OF THIS WORK

Pre-print for this project:

PDF

CovidVisualized: Visualized compilation of international updated models’ estimates of COVID-19 pandemic at global and country levels

Farshad Pourmalek, MD PhD

SUMMARY

Objectives: To identify international and periodically updated models of the COVID-19 pandemic, compile and visualize their estimation results at the global and country levels, and periodically update the compilations. When one or more models predict an increase in daily cases or infections and deaths in the next one to three months, technical advisors to the national and subnational decision-makers can consider this early alarm for assessment and suggestion of augmentation of preventive measures and interventions.

Methods and Results: Five international and periodically updated models of the COVID-19 pandemic were identified, created by: (1) Massachusetts Institute of Technology, Cambridge, (2) Institute for Health Metrics and Evaluation, Seattle, (3) Imperial College, London, (4) Los Alamos National Laboratories, Los Alamos, and (5) University of Southern California, Los Angeles. Estimates of these five identified models were gathered, combined, and graphed at global and two country levels. Canada and Iran were chosen as countries with and without subnational estimates, respectively. Compilations of results are periodically updated. Three Github repositories were created that contain the codes and results, i.e., “CovidVisualizedGlobal” for the global level, “CovidVisualizedCountry” for a country with subnational estimates – Canada, and “covir2” for a country without subnational estimates – Iran.

Keywords: COVID-19, pandemic, epidemic, models, visualization, global, Canada, Iran

BACKGROUND

Objectives and rationale: The objectives are to identify international and periodically updated models of the COVID-19 pandemic, compile and visualize their estimations’ results at the global and country levels, and periodically update the compilations. The ultimate objective is to provide an early warning system for technical advisors to the decision-makers. When the predictions of one or more models show an increase in daily cases or infections, hospitalizations, or deaths in the next one to three months, technical advisors to the national and subnational decision-makers may consider assessing the situation and suggesting augmentation of non-pharmacologic preventive interventions and vaccinations. In doing so, the strengths and weaknesses of individual models need to be considered and those of this work. Models’ estimates demonstrate the trajectory of COVID-19 deaths, cases or infections, and hospital-related outcomes in one to three months into the future.

METHODS

Eligibility criteria: The criteria for inclusion of target COVID-19 pandemic models were (1) an international model scope and (2) periodic updates. “International model” denotes a model that estimates COVID-19 cases or infections and deaths for all countries of the world, with global-level estimates that equate the sum of the national-level estimates. “Periodically updated” denotes a model with a record of periodically updated estimates since its first release, with continued updates in 2021.

Finding the eligible models: The eligible models were found within the literature search of a previous publication, “Rapid review of COVID-19 epidemic estimation studies for Iran” [1]. The results were verified by comparison with models found in a recently published study on “Predictive performance of international COVID-19 mortality forecasting models” [2]. While non-updated or one-time models can be contemporaneously usable, their results would not sustain up-to-dateness in the long run, especially with the emergence of new variants of concern and various degrees of uncertainties in the progression of vaccination coverage.

RESULTS

Results are described under the following items: (1) The five identified models / studies, (2) The CovidVisualized repositories created in this work, (3) Data management, and (4) Periodical uptakes.

(1) The five identified models / studies

Five international and periodically updated models of the COVID-19 pandemic were identified: (1) DELPHI , Massachusetts Institute of Technology, Cambridge (abbreviation used in this work: DELP) [3], (2) Institute for Health Metrics and Evaluation, Seattle (IHME) [4], (3) Imperial College, London (IMPE) [5], (4) Los Alamos National Laboratories, Los Alamos (LANL) [6], (5) University of Southern California, Los Angeles, by Srivastava, Ajitesh (SRIV) [7]. Official reports of countries to World Health Organization, curated by Johns Hopkins University Coronavirus resource center (JOHN) [8], were also used for comparison.

(1) DELP

. DELP = DELPHI: Differential Equations Lead to Predictions of Hospitalizations and Infections
. Citation: COVID Analytics. DELPHI epidemiological case predictions. Cambridge: Operations Research Center, Massachusetts Institute of Technology. https://www.covidanalytics.io/projections
. Study website: https://www.covidanalytics.io/projections
. Estimates web site: https://www.covidanalytics.io/projections, down the page, link that reads, "Download Most Recent Predictions"
. License: https://github.com/COVIDAnalytics/DELPHI/blob/master/LICENSE
. Institution: Operations Research Center, Massachusetts Institute of Technology, Cambridge
. Among articles: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7883965/ , https://www.medrxiv.org/content/10.1101/2020.06.23.20138693v1, https://www.covidanalytics.io/DELPHI_documentation_pdf
. Periodically updated: Yes
. Periodical updates accessible: Yes

(2) IHME

. IHME = Institute for Health Metrics and Evaluation
. Citation: Institute for Health Metrics and Evaluation (IHME). COVID-19 mortality, infection, testing, hospital resource use, and social distancing projections. Seattle: Institute for Health Metrics and Evaluation (IHME), University of Washington. http://www.healthdata.org/covid
. Study web site: http://www.healthdata.org/covid
. Estimates web site: http://www.healthdata.org/covid/data-downloads
. License: http://www.healthdata.org/about/terms-and-conditions
. Institution: IHME, University of Washington, Seattle
. Among articles: https://www.nature.com/articles/s41591-020-1132-9
. Periodically updated: Yes
. Periodical updates accessible: Yes

(3) IMPE

. IMPE = Imperial College
. Citation: MRC Centre for Global Infectious Disease Analysis (MRC GIDA). Future scenarios of the healthcare burden of COVID-19 in low- or middle-income countries. London: MRC Centre for Global Infectious Disease Analysis, Imperial College London. https://mrc-ide.github.io/global-lmic-reports/
. Study web site: https://mrc-ide.github.io/global-lmic-reports/
. Estimates web site: https://github.com/mrc-ide/global-lmic-reports/tree/master/data
. License: https://github.com/mrc-ide/global-lmic-reports
. Institution: Imperial College, London
. Among articles: https://science.sciencemag.org/content/369/6502/413
. Periodically updated: Yes

. Periodical updates accessible: Yes, up to 20210823, No, from 20210824

(4) LANL

. LANL = Los Alamos National Laboratories
. Citation: Los Alamos National Laboratory (LANL). COVID-19 cases and deaths forecasts. Los Alamos: Los Alamos National Laboratory (LANL). https://covid-19.bsvgateway.org
. Study web site: https://covid-19.bsvgateway.org
. Estimates web site: https://covid-19.bsvgateway.org, Model Outputs, Global
. License: https://covid-19.bsvgateway.org
. Institution: Los Alamos National Laboratories, Los Alamos
. Among documents: https://covid-19.bsvgateway.org/static/COFFEE-methodology.pdf
. Periodically updated: Yes
. Periodical updates accessible: Yes

(5) SRIV

. SRIV = Srivastava, Ajitesh
. Citation: University of Southern California (USC). COVID-19 forecast. Los Angeles: University of Southern California. https://scc-usc.github.io/ReCOVER-COVID-19
. Study web site: https://scc-usc.github.io/ReCOVER-COVID-19/
. Estimates web site: https://github.com/scc-usc/ReCOVER-COVID-19/tree/master/results/historical_forecasts
. License: https://github.com/scc-usc/ReCOVER-COVID-19/blob/master/LICENSE
. Institution: University of Southern California, Los Angeles
. Among articles: https://arxiv.org/abs/2007.05180
. Periodically updated: Yes
. Periodical updates accessible: Yes

(0) JOHN

. JOHN = Johns Hopkins University. Coronavirus resource center. https://coronavirus.jhu.edu
. Not a target study, but a benchmark for comparison.
. Citation: "COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University"
. Study web site: https://coronavirus.jhu.edu
. Estimates web site: https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_time_series , "COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University"
. License: https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_time_series
. Institution: Johns Hopkins University, Baltimore
. Among articles: Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020 May;20(5):533-534. doi: 10.1016/S1473-3099(20)30120-1. Epub 2020 Feb 19. . Periodically updated: Yes
. Periodical updates accessible: Yes

The COVID-19 pandemic model by Youyang Gu [9] and the model by University of California, Los Angeles model [10] could not be categorized as international and periodically updated models. The COVID-19 International Modelling Consortium (CoMo Consortium) model, created by researchers at the University of Oxford and Cornell University [11], and CovidSim (COVID Simulation) model, developed by researchers at Imperial College, London [12], provide templates for researchers to model the future of epidemic trajectory at national and subnational levels of their choice, through adjusting the model inputs and setting the time horizon into future for the estimations. Unlike the five international and periodically updated models mentioned above, the latter two models are not intended for periodic updates by their original creators. The CoMo Consortium has engaged some countries, including Iran, but not Canada. There is no evidence of either model being used on a periodically updated basis in Iran or Canada.

(2) The CovidVisualized repositories created in this work

GitHub (https://github.com) is used for sharing the codes and data. Global and country levels were chosen for demonstration of results. For the national level, one country with and one country without subnational estimates – Canada and Iran, respectively – were chosen based on personal bonds.

Four of the five identified models share codes and estimates updates via GitHub repositories, and the IHME estimates are released on IHME’s website [4]. Three Github repositories were created for this project: “CovidVisualizedGlobal” [13] for the global level, “CovidVisualizedCountry” [14] for Canada, and “covir2” [15] for Iran. These are referred to as CovidVisualized GitHub repositories hereon. The “covir2” repository was created as “COVID Iran Review number 2” in March 2021, as an update after a first-iteration review was published [1]. The idea and work were further developed toward applicability to any location in the world, with case application for the global level, Canada, and Iran.

GitHub repositories allow others to view and/or download, scrutinize, and verify the integrity of the codes and data. It is also possible to minimally modify the codes to recreate similar repositories for any other country that reports COVID-19 cases and deaths to World Health Organization. Such use of the codes and data in GitHub is free of charge and bound to the pertinent licenses.

The three GitHub repositories created in this project are:

. CovidVisualizedGlobal, COVID-19 pandemic estimates at the global level [13] https://github.com/pourmalek/CovidVisualizedGlobal
. CovidVisualizedCountry, COVID-19 pandemic estimates at the country level: Canada [14] https://github.com/pourmalek/CovidVisualizedCountry
. covir2, COVID-19 pandemic estimates at the country level: Iran [15] https://github.com/pourmalek/covir2

(3) Data management

Data management template: A data management template was created for assigning comparable variable names to various outcomes from different models. Comparable and common variable names consist of generic parts (positions in the variable name) denoting the following items: (1) daily or total, (2) deaths, cases or infections, or other outcomes, (3) mean estimate, or lower, or upper uncertainty limit, (4) raw or smoothed estimate, (5) each of the five individual models, and (6) scenarios within each model. This template is described in detail in “variable name structure” in the CovidVisualized GitHub repositories [16]. Stata SE 14.2 (Stata Statistical Software. StataCorp. College Station, Texas) was used to write and run the codes on macOS Big Sur, and test run on Microsoft Windows 10.

Data management: Data acquisition, management, and graphing were performed via Stata codes. Stata codes download the models’ output files from their respective websites, edit them according to the data management template, store each model’s estimates in a single file, and create graphs for all outcomes produced by each model. Then, the models’ single data files are compiled into a single final file, and graphs for all common outcomes are created for this compilation of all models. These graphs are shown on the pages of the three CovidVisualized GitHub repositories [13-15] and in periodical Situation Reports created with each uptake.

Outcome types: Besides (1) daily deaths, (2) daily incident cases or infections, (3) total deaths, and (4) total cases or infections, other outcomes estimated by one or more individual models include prevalence, active or prevalent cases, recovered cases, hospital admissions, regular beds needed, ICU (Intensive Care Unit) beds needed, ventilated cases, seroprevalence, and effective reproduction number. All outcome types were graphed.

Secondary variables: Secondary variables are created in this work using the primary variables released by the individual models. They include case fatality rate (CFR), infection fatality rate (IFR), cases per deaths, ratio of estimated to reported deaths, and ratio of estimated to reported cases. CFR and IFR have only a daily version, and the other secondary variables have both daily and total (cumulative) versions.

Uncertainty, scenarios, variants, and vaccines: For each model and for each outcome, both the point (mean) estimates and the interval estimates (95% uncertainty limits) were graphed where available. Similarly, both the “reference” scenario (aka status quo) and alternative scenarios (i.e., “better” and “worse” scenarios) were graphed for models with more than one scenario (i.e., IHME and IMPE). Assumptions about and empirical inputs from distributions of variants and vaccination coverages across space and time have been progressively included in models and scenarios of IHME and IMPE.

Subnational estimates: The DELP and IHME models provide subnational-level estimates for countries reporting national and subnational level COVID-19 outcomes. Graphs were created for national and subnational-level locations (i.e., provinces in Canada) available in DELP and IHME model outputs.

(4) Periodical uptakes

A set of conventions were created for the periodic uptake of the models’ estimates updates. The two models with the least frequency of periodic updates of estimates are IHME and IMPE, updated almost weekly and bi-weekly, respectively. With the release of each update of either of these two models, the whole set of the five included models are updated in all the three CovidVisualized GitHub repositories. The most recent update of each model is used. These updates of CovidVisualized repositories are labelled as “uptakes” to differentiate them from models’ estimates updates. These conventions for periodical uptakes are described in detail in the CovidVisualized GitHub repositories [17]. R software via RStudio 1.4 (Integrated Development for R. RStudio. PBC, Boston, Massachusetts) was used for semi-automatization of the uptakes’ execution. Estimates of the LANL model get updated about every 3-4 days, and DELP and SRIV models get updated daily. Uptaking the models’ estimates updates with every update of these latter three models is not expected to depict a much more informative profile of the epidemic’s trajectory in future, when compared against the current convention for uptakes. The IHME, IMPE, and SRIV models provide estimates for about three months into the future with each update release, the DELP model for about two months, and the LANL model for about one month.

With each uptake, a directory is created in the root of the main branch of each of the three CovidVisualized repositories and named with the uptake date (e.g., 20210730). Uptakes are also created retrospectively, compiling the results of the previous updates of the models’ results. Available uptakes for Iran and the global level have been created going back to April 2021 and for Canada to June 2021. Under each dated uptake directory, there are two directories for “code” and for “output”. Under each, there are located directories with the abbreviated name of the models. For example, “DELP” directories under “code” store the Stata code files (.do), and those under “output” contain the outputs from executing the codes: the single data file for model estimates (in. dta and .csv formats), Stata log file (.smcl), and the graphs in PDF format. The directory “master” contains the master Stata do-file which executes all the other do-files, and the directory “merge” (under “code”) contains the code for creating the single final merged file of all models. The directory “merge” (under “output”) stores the created single final merged file of all compiled models, as well as the graphs that contain all the models. Selected graphs that contain all the models are visible on the root page of each uptake directory and are also stacked in reverse chronological order on the main page of each repository. With each uptake, selected graphs of estimated outcomes are added at the start of the main page for each repository. Situation Reports are created and shared with national and subnational health authorities.

Similar work: The “covidcompare” tool [18] provides graph visualization of the latest estimates of daily and total deaths from international and periodically updated COVID-19 pandemic models for countries of the world and US states, along with historical forecasts and model performance, based on IHME’s “Predictive performance of international COVID-19 mortality forecasting models” [2].

LIMITATIONS, STRENGTHS, AND FURTHER DIRECTIONS

Limitations: Limitations of this work include the programming languages, automatization of uptakes, and choice of the website for presentation of the results.

Stata programming language constitutes about 99% of the codes. Whereas Stata is a commercial software package, using non-commercial packages such as R and/or Python can increase the accessibility and adaptability of the codes for other researchers. Further use of R and/or Python can also make the uptakes almost fully automatized. Some health researchers may not be familiar with GitHub and GIT programming. Therefore, additional use of a dedicated website that is more visible to and accessible for the target audience can increase the reach and effect of this work.

Strengths and weaknesses of individual international and periodically updated COVID-19 pandemic models are not mentioned here, but they have been discussed elsewhere [1-2].

Strengths: Strengths of this work include usability for informing technical advisors to the decision-makers, adaptability for use in other countries, and automatized data acquisition.

Tested usability for informing technical advisors to the decision-makers at the country level: Results of the GitHub repository “covir2” [15] were used to present the predictions of the five international and periodically updated models of COVID-19 pandemic about the possibility, timing, slope, height, and drivers of a potential fifth wave of the epidemic in Iran. This presentation was done using the results of the covir2 repository along with the results of an e-mail survey of more than 40 epidemiologists and public health specialists. The predictions and results were presented and described in a live online session for three Deputy Ministers of Health and six epidemiologists selected by Iran’s Ministry of Health and Medical Education (MOHME). Periodical situation reports based on each uptake are also shared with MOHME.

Adaptability of the codes for use in other countries or regions in the world: The codes available in GitHub repositories “CovidVisualizedCountry” [14] and “covir2” [15] can be slightly modified by any researcher to be used for countries with and without subnational estimates respectively (See examples for United States, Afghanistan, and Pakistan). “CovidVisualizedCountry” can be adjusted for use for any type of regionalization of the countries of the world, e.g., World Health Organization regions.

Automatized data acquisition: The Stata codes in these repositories automatically download the estimates’ data files from the five included models once executed. There is no additional need for users to locate, download, and edit the estimates’ data of individual models before running the codes. This automatic data acquisition further enhances computational reproducibility – “obtaining consistent results using the same input data; computational steps, methods, and code; and conditions of analysis” [19].

Further research: Further directions include using the “ensemble” method to statistically combine models’ estimates, and retrospective assessment of models’ predictive performance. In ensemble methods, individual models are evaluated for minimum requirements of quality and reporting. They are statistically combined using specific relative weights for each model, where the weights reflect the comparative accuracy of each model. Such ensemble methods are used by European Centre for Disease Prevention and Control [20] and US COVID-19 Forecast Hub [21]. The ensemble models have been empirically shown to be more accurate than any of the individual models used in the ensemble method [22]. Retrospective assessment of models’ predictive performance includes using statistical and graphical methods to estimate and visualize the accuracy of models’ estimations [2].

DELERATIONS

Ethics approval and consent to participate

All the used and produced data are at non-individual and aggregate level; publicly available on the Internet; and under pertinent licenses and copyrights for non-commercial use, reproduction, and distribution for scientific research, provided that the conditions mentioned in their respective licenses and copyrights are met, as provided in [23]. Therefore, no ethics approval or consent to participate were applicable.

Consent for publication

Not applicable.

Availability of data and materials

The data described in this Data note can be freely and openly accessed on (1) GitHub repository “CovidVisualizedGlobal” under (http://doi.org/10.5281/zenodo.5019030) [13], (2) GitHub repository “CovidVisualizedCountry” under (http://doi.org/10.5281/zenodo.5019482) [14], and (3) GitHub repository “covir2” under (http://doi.org/10.5281/zenodo.5020797) [15]. See references [13-15] for details and links to the data.

Competing interests

The author worked as a post-graduate research fellow in Institute for Health Metrics and Evaluation from 2009 to 2011 and continues voluntary collaboration as a Global Burden of Disease study collaborator without employment or financial relation. The author declares that he has no competing interests.

Funding

There were no sources of funding for this research.

References

Pourmalek F, Rezaei Hemami M, Janani L, Moradi-Lakeh M. Rapid review of COVID-19 epidemic estimation studies for Iran. BMC Public Health. 2021 Feb 1;21(1):257. doi: 10.1186/s12889-021-10183-3.
Friedman J, Liu P, Troeger CE, Carter A, Reiner RC Jr, et al. Predictive performance of international COVID-19 mortality forecasting models. Nat Commun. 2021 May 10;12(1):2609. doi: 10.1038/s41467-021-22457-w.
COVID Analytics. DELPHI epidemiological case predictions. Cambridge: Operations Research Center, Massachusetts Institute of Technology. https://www.covidanalytics.io/projections and https://github.com/COVIDAnalytics/website/tree/master/data/predicted
Institute for Health Metrics and Evaluation (IHME). COVID-19 mortality, infection, testing, hospital resource use, and social distancing projections. Seattle: Institute for Health Metrics and Evaluation (IHME), University of Washington. http://www.healthdata.org/covid/ and http://www.healthdata.org/covid/data-downloads
MRC Centre for Global Infectious Disease Analysis (MRC GIDA). Future scenarios of the healthcare burden of COVID-19 in low- or middle-income countries. London: MRC Centre for Global Infectious Disease Analysis, Imperial College London. https://mrc-ide.github.io/global-lmic-reports/ and https://github.com/mrc-ide/global-lmic-reports/tree/master/data
Los Alamos National Laboratory (LANL). COVID-19 cases and deaths forecasts. Los Alamos: Los Alamos National Laboratory (LANL). https://covid-19.bsvgateway.org
Srivastava, Ajitesh. University of Southern California (USC). COVID-19 forecast. Los Angeles: University of Southern California. https://scc-usc.github.io/ReCOVER-COVID-19 and https://github.com/scc-usc/ReCOVER-COVID-19/tree/master/results/historical_forecasts
Johns Hopkins University. Coronavirus resource center. https://coronavirus.jhu.edu/map.html and https://github.com/CSSEGISandData/COVID-19
Gu, Youyang. COVID-19 Projections Using Machine Learning. https://covid19-projections.com and https://github.com/youyanggu/covid19_projections
Statistical Machine Learning Lab, Computer Science Department, University of California, Los Angeles. Combating COVID-19. https://covid19.uclaml.org/info.html and https://github.com/uclaml/ucla-covid19-forecasts/tree/master/current_projection
The COVID-19 International Modelling Consortium (CoMo Consortium), University of Oxford and Cornell University. https://www.medsci.ox.ac.uk/news/como-consortium-the-covid-19-pandemic-modelling-in-context and https://github.com/ocelhay/como
MRC Centre for Global Infectious Disease Analysis, Imperial College London. CovidSim. COVID-19 Scenario Analysis Tool. https://covidsim.org/v5.20210727/?place=ca and https://covidsim.org/v5.20210727/?place=ir
Pourmalek, F. GitHub repository “CovidVisualizedGlobal”: Combine and visualize international periodically updated estimates of COVID-19 at the global level. Version 1.1, Released June 23, 2021. http://doi.org/10.5281/zenodo.5019030 https://github.com/pourmalek/CovidVisualizedGlobal
Pourmalek, F. GitHub repository “CovidVisualizedCountry”: Combine and visualize international periodically updated estimates of COVID-19 at the country level, countries with subnational level estimates: Canada, national level, provinces, and territories. Version 1.1, Released June 23, 2021. http://doi.org/10.5281/zenodo.5019482 https://github.com/pourmalek/CovidVisualizedCountry
Pourmalek, F. GitHub repository “covir2”: Combine and visualize international periodically updated estimates of COVID-19 at the country level, countries without subnational level estimates: Iran. Version 2.2, Released June 23, 2021. http://doi.org/10.5281/zenodo.5020797 https://github.com/pourmalek/covir2
Pourmalek, F. “covir2”: Combine and visualize international periodically updated estimates of COVID-19 at the country level: Iran. Version 2.2, Released June 23, 2021. Variable name structure. http://doi.org/10.5281/zenodo.5020797 https://github.com/pourmalek/covir2/blob/main/Variable%20name%20structure.md
Pourmalek, F. “covir2”: Combine and visualize international periodically updated estimates of COVID-19 at the country level: Iran. Version 2.2, Released June 23, 2021. Setup. http://doi.org/10.5281/zenodo.5020797 https://github.com/pourmalek/covir2/tree/main/setup
Friedman J, Liu P, Akre S. The covidcompare tool. https://covidcompare.io/about
National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science. Washington, DC: The National Academies Press. 2019. https://doi.org/10.17226/25303
European Centre for Disease Prevention and Control. European Covid-19 Forecast Hub. https://covid19forecasthub.eu/background.html and https://github.com/epiforecasts/covid19-forecast-hub-europe
COVID-19 Forecast Hub. https://covid19forecasthub.org/doc/ensemble and https://github.com/reichlab/covid19-forecast-hub Accessed 23 June 2021.
Cramer EY, Ray EL, Lopez VK, Bracher J, Brennen A, et al. Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US. medRxiv preprint. Posted February 05, 2021. https://www.medrxiv.org/content/10.1101/2021.02.03.21250974v1
Pourmalek, F. GitHub repository “covir2”: Combine and visualize international periodically updated estimates of COVID-19 at the country level, countries without subnational level estimates: Iran. Version 2.2, Released June 23, 2021. Licenses / Copyrights of data and/or graphs used in this repository. http://doi.org/10.5281/zenodo.5020797 https://github.com/pourmalek/covir2#licenses--copyrights-of-data-and--or-graphs-used-in-this-repository Accessed 23 June 2021.

III. INNER WORKS OF THIS REPOSITORY

The Stata codes can be executed on local machines:

Run in Stata "Users/local-user-name/Downloads/CovidVisualizedCountry/20210709/code/master/do country master.do" on your local machine after the directory /CovidVisualizedCountry/ is downloaded into and is located in the root of /Downloads/ folder of your local machine, for uptake 20210709; and other instances of 202YMMDD for other uptake dates.

Preproduced outputs are stored for each uptake, e.g., 20210709, in folder output, for each component study, i.e., DELP, IHME, IMPE, LANL, and SRIV, plus JOHN as benchmark. Combined results are stored in merge. All merged graphs for each uptake are accessible via main page of each uptake date, e.g., https://github.com/pourmalek/CovidVisualizedCountry/tree/main/20210709/ and the link is placed in front of Merged graphs of uptake 20210709 here

Variable name structure describes the template for models’ output data management used in this repository.

Rule for uptakes describes the rule for periodical uptakes used in this repository.

Troubleshooting describes possible difficulties in running the Stata codes on your computer after the repository has been downloaded to your local machine.

Troubleshooting:

On 20210824, the Imperial College COVID-19 model/study removed their "old fits" – all previously released updates from day 1 up to and including IMPE update 20210719.

This means the Imperial College COVID-19 model/study is no more a periodically updated study with ACCESSIBLE updates – short of early retirement of the study. The predictive performance of the Imperial College model is no longer assessable.

Several thousand lines of codes that used their previously released updates will not work.

The remedy for CovidVisualized repositories (https://github.com/pourmalek/covir2 for Iran, https://github.com/pourmalek/CovidVisualizedCountry for Canada, and https://github.com/pourmalek/CovidVisualizedGlobal for the global level) is to comment out the command line reading csv data from https://github.com/mrc-ide/global-lmic-reports/tree/master/data.

To -comment out-,

For https://github.com/pourmalek/CovidVisualizedCountry,

In files:

/Users/username/Downloads/CovidVisualizedCountry-main/20210806/code/master/do CovidVisualizedCountry master.do, downloaded to your local machine,

where 20210806 denotes uptake 20210806 in this repository that uses IMPE update 20210719, or earlier uptake here that use earlier updates of IMPE,

in line 124, that currently reads as:

do "$pathCovidVisualizedCountry/IMPE/do CovidVisualizedCountry IMPE.do"

double forward slashes // have been put at the beginning of the line, so that it reads as:

// do "$pathCovidVisualizedCountry/IMPE/do CovidVisualizedCountry IMPE.do"

save the do file and then run the file.

However, the previously stored results, such as the graphs stored in "CovidVisualizedCountry-main/20210401/output/merge/asterisk.pdf" and "CovidVisualizedCountry-main/20210401/output/IMPE/asterisk.pdf" preserve a graphical copy of previously accessed the Imperial College COVID-19 model/study removed updates.
Data version of previously accessed Imperial College COVID-19 model/study removed updates are not stored in CovidVisualizedCountry repository due to very large file size. Removal of Imperial College COVID-19 model/study "old fits" hampers retrospective longitudinal assessment of the predictive performance of Imperial model, as a side effect or main effect.

Scenario number within component study

Component studies (the international periodically updated models) and their scenarios are mentioned below.

[Names within brackets assigned by this project.]

A00 JOHN

S00 = [Not Applicable]

A01 DELP

S00 = [Single scenario]

A02 IHME

S01 = Reference scenario [Status Quo]
S02 = Best scenario (Universal masks) [Best]
S03 = Worse scenario (Mandates easing) [Worst]

A03 IMPE

S01 = Additional 50% Reduction [Best]
S02 = Current level of interventions [Reference]
S03 = Relax Interventions 50% [Worst]

. additional alternate scenarios:
S04 = Surged Additional 50% Reduction [Best, Surged]
S05 = Surged Maintain Status Quo [Reference, Surged]
S06 = Surged Relax Interventions 50% [Worst, Surged]

A04 LANL

S00 = [Single scenario]

A05 SRIV

S00 = [current]
Note: SRIV has 3 sets of scenrios; see

uptakes in this repository, since April 2021

bold italic fonts show the uptake was triggered by either IHME or IMPE.

(uptake number) uptake date: study update date, study update date

(20) uptake 20210824: DELP 20210824, IHME 20210819, IMPE 20210819, LANL 20210822, SRIV 20210824

(19) uptake 20210819: DELP 20210819, IHME 20210819, IMPE 20210806, LANL 20210815, SRIV 20210819

(18) uptake 20210806: DELP 20210806 , IHME 20210806, IMPE 20210719, LANL 20210801, SRIV 20210801

(17) uptake 20210730: DELP 20210730, IHME 20210730, IMPE 20210719, LANL 20210725, SRIV 20210730

(16) uptake 20210727: DELP 20210727, IHME 20210723 version 2, IMPE 20210719, LANL 20210725, SRIV 20210727

(15) uptake 20210726: DELP 20210726, IHME 20210723 version 2, IMPE 20210709, LANL 20210718, SRIV 20210726

. 20210726: IHME estimates for Canada in update 20210723 and in update 20210715 WERE identical UPON FIRST RELEASE OF update 20210723, with numerical value difference of zero. As of 20210726, update 20210723 has been replaced by IHME and is not identical with update 20210715.

(14) uptake 20210723: DELP 20210723, IHME 20210723, IMPE 20210709, LANL 20210718, SRIV 20210723

(13) uptake 20210715: DELP 20210715, IHME 20210715, IMPE 20210709, LANL 20210711, SRIV 20210715

(12) uptake 20210714: DELP 20210714, IHME 20210702, IMPE 20210709, LANL 20210711, SRIV 20210714

(11) uptake 20210709: DELP 20210708, IHME 20210702, IMPE 20210702, LANL 20210704, SRIV 20210709

(10) uptake 20210704: DELP 20210704, IHME 20210702, IMPE 20210626, LANL 20210704, SRIV 20210704

(09) uptake 20210703: DELP 20210703, IHME 20210702, IMPE 20210618, LANL 20210627, SRIV 20210703

(08) uptake 20210625: DELP 20210625, IHME 20210625, IMPE 20210618, LANL 20210613, SRIV 20210624

(07) uptake 20210624: DELP 20210624, IHME 20210618, IMPE 20210618, LANL 20210613, SRIV 20210624

(06) uptake 20210618: DELP 20210618, IHME 20210618, IMPE 20210611, LANL 20210613, SRIV 20210618

(05) uptake 20210611: DELP 20210611, IHME 20210610, IMPE 20210611, LANL 20210606, SRIV 20210611

(04) uptake 20210610: DELP 20210610, IHME 20210610, IMPE 20210604, LANL 20210606, SRIV 20210610

(03) uptake 20210605: DELP 20210605, IHME 20210604, IMPE 20210604, LANL 20210602, SRIV 20210605

(02) uptake 20210604: DELP 20210604, IHME 20210604, IMPE 20210527, LANL 20210602, SRIV 20210604

(01) uptake 20210603: DELP 20210603, IHME 20210528, IMPE 20210527, LANL 20210526, SRIV 20210603

IV. SELECTED GRAPHS FROM PREVIOUS UPTAKES

Selected graphs for Canada national level and provinces are displayed in separate web pages, links to which are provided below. This facilitates side-by-side comparisons across provinces and the national level.

Among the 5 available international periodically updated studies or models of COVID-19 pandemic, only TWO studies, DELP and IHME, provide subnational level estimates for some countries.

For Canada, DELP and IHME models provide subnational estimates for SEVEN provinces: Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, and Saskatchewan.

List of graphs

(1) Daily deaths, reference scenarios, all time

(2) Daily deaths, reference scenarios, 2021

(3) Daily deaths, 3 scenarios, 2021

(4) Daily cases or infections, reference scenarios, all time

(5) Daily cases or infections, reference scenarios, 2021

(6) Daily cases or infections, 3 scenarios, 2021

(7) Hospital-related outcomes, all time

(8) Hospital-related outcomes, 2021, without IHME Bed need and IMPE Hospital demand

(9) Daily deaths estimated to reported, reference scenarios, 2021

(10) Daily cases or infections estimated to reported cases, reference scenarios, 2021

Logical order of graphs

Location levels: National, followed by provinces for which estimations are available: Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, and Saskatchewan

. To follow prediction for the national level, and further examine if province-level increases are predicted
Uptake date: Most recent uptake, followed by less recent uptakes

. To examine changes in predictions across consecutive uptakes
Outcomes: Daily deaths, Daily cases or infections, Hospital-related outcomes, Daily deaths estimated to reported, Daily cases or infections estimated to reported cases.

. To examine different outcomes with different public health consequences and model performance implications
Calendar time of estimates coverage: All-time, followed by 2021

. To view the whole epidemic trajectory, and further focus on near future
Scenarios: Reference scenarios, followed by alternative scenarios

. To examine the main (aka. status quo) scenario, and alternative (better and worse) scenarios; scenarios mentioned here
Five models: Different models within each graph (for which model estimates update release dates are maximally synchronized), plus official reports of the country to WHO (curated by Johns Hopkins university) as under-reported benchmark for trends

. To examine how heterogeneity in methods used by different models results in heterogenous results for the same outcome (same time-place-person aggregated units)

Selected graphs - CANADA

Selected graphs - Alberta

Selected graphs - British Columbia

Selected graphs - Manitoba

Selected graphs - Nova Scotia

Selected graphs - Ontario

Selected graphs - Quebec

Selected graphs - Saskatchewan

Licenses / Copyrights of data and / or graphs used in this repository:

All the data and / or graphs used in this repository are at non-individual and aggregate level, publicly available on the Internet, and under pertinent licenses and copyrights for non-commercial use, reproduction, and distribution for scientific research, provided that the conditions mentioned in the respective licenses and copyrights are met, as referred to below.

(1) ABBREVIATED NAME IN THIS REPOSITORY: DELP

CITATION: COVID Analytics. DELPHI epidemiological case predictions. Cambridge: Operations Research Center, Massachusetts Institute of Technology. https://www.covidanalytics.io/projections and https://github.com/COVIDAnalytics/website/tree/master/data/predicted

SOURCE REPOSITORY: https://github.com/COVIDAnalytics/DELPHI

SOURCE REPOSITORY LICENCE: https://github.com/COVIDAnalytics/website/blob/master/LICENSE

(2) ABBREVIATED NAME IN THIS REPOSITORY: IHME

CITATION: Institute for Health Metrics and Evaluation (IHME). COVID-19 mortality, infection, testing, hospital resource use, and social distancing projections. Seattle: Institute for Health Metrics and Evaluation (IHME), University of Washington. http://www.healthdata.org/covid/ and http://www.healthdata.org/covid/data-downloads

SOURCE REPOSITORY: http://www.healthdata.org/covid/data-downloads

SOURCE REPOSITORY LICENCE: http://www.healthdata.org/about/terms-and-conditions

(3) ABBREVIATED NAME IN THIS REPOSITORY: IMPE

CITATION: MRC Centre for Global Infectious Disease Analysis (MRC GIDA). Future scenarios of the healthcare burden of COVID-19 in low- or middle-income countries. London: MRC Centre for Global Infectious Disease Analysis, Imperial College London. https://mrc-ide.github.io/global-lmic-reports/ and https://github.com/mrc-ide/global-lmic-reports/tree/master/data

SOURCE REPOSITORY: https://github.com/mrc-ide/global-lmic-reports/tree/master/data

SOURCE REPOSITORY LICENCE: https://mrc-ide.github.io/global-lmic-reports/

(4) ABBREVIATED NAME IN THIS REPOSITORY: LANL

CITATION: Los Alamos National Laboratory (LANL). COVID-19 cases and deaths forecasts. Los Alamos: Los Alamos National Laboratory (LANL). https://covid-19.bsvgateway.org

SOURCE REPOSITORY: https://covid-19.bsvgateway.org

SOURCE REPOSITORY LICENCE: https://covid-19.bsvgateway.org

(5) ABBREVIATED NAME IN THIS REPOSITORY: SRIV

CITATION: Srivastava, Ajitesh. University of Southern California (USC). COVID-19 forecast. Los Angeles: University of Southern California. https://scc-usc.github.io/ReCOVER-COVID-19 and https://github.com/scc-usc/ReCOVER-COVID-19/tree/master/results/historical_forecasts

SOURCE REPOSITORY: https://github.com/scc-usc/ReCOVER-COVID-19/tree/master/results/historical_forecasts

SOURCE REPOSITORY LICENCE: https://github.com/scc-usc/ReCOVER-COVID-19/blob/master/LICENSE

(6) ABBREVIATED NAME IN THIS REPOSITORY: JOHN

CITATION: "COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University" https://coronavirus. jhu.edu/map.html and https://github.com/CSSEGISandData/COVID-19

SOURCE REPOSITORY: https://github.com/CSSEGISandData/COVID-19

SOURCE REPOSITORY LICENCE: https://github.com/CSSEGISandData/COVID-19

(7) ABBREVIATED NAME IN THIS REPOSITORY: covidcompare

CITATION: Friedman J, Liu P, Akre S. The covidcompare tool. https://covidcompare.io/about

SOURCE REPOSITORY: https://covidcompare.io/

SOURCE REPOSITORY LICENCE: https://covidcompare.io/about

License, DOI, and suggested Citation of this reposirory

All codes are copyrighted by the author under Apache License 2.0.

See LICENSE

DOI

Pourmalek, F. GitHub repository “CovidVisualizedCountry”: Combine and visualize international periodically updated estimates of COVID-19 pandemic at the country level, countries with subnational level estimates: Canada, national level and provinces. Version 1.1, Released June 23, 2021. http://doi.org/10.5281/zenodo.5019482 , https://github.com/pourmalek/CovidVisualizedCountry

License

pourmalek/CovidVisualizedCountry

Folders and files

Latest commit

History

Repository files navigation

CovidVisualizedCountry

Combine and visualize international periodically updated estimates of COVID-19 pandemic

at the country level, countries with subnational level estimates

🇨🇦 Canada: national level and provinces

TOC

I. SELECTED GRAPHS FROM LATEST UPTAKE

II. METHODS AND RESULTS OF THIS WORK

III. INNER WORKS OF THIS REPOSITORY

IV. SELECTED GRAPHS FROM PREVIOUS UPTAKES

👀 SEE graphs, code, and data of periodical updates of COVID-19 pandemic models’ estimates:

by models: Delphi, IHME, Imperial college, Los Alamos, and Srivastava,

for daily deaths, daily cases or infections, hospitalizations

for Canada: National level, Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, Saskatchewan

or other countries via code adjustment

Related repositories:

Among the 5 available international periodically updated studies or models of COVID-19 pandemic, only TWO studies, DELP and IHME, provide subnational level estimates for some countries.

For Canada, DELP and IHME models provide subnational estimates for SEVEN provinces: Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, and Saskatchewan.

I. SELCTED GRAPHS FROM LATEST UPTAKE

LATEST UPTAKE: uptake 20210824

👀 SEE: Canada COVID-19 epidemic models situation report No 04 on 2021-08-24

Selected graphs from latest uptake - Canada

Selected graphs from latest uptake - Alberta

Selected graphs from latest uptake - British Columbia

Selected graphs from latest uptake - Manitoba

Selected graphs from latest uptake - Nova Scotia

Selected graphs from latest uptake - Ontario

Selected graphs from latest uptake - Quebec

Selected graphs from latest uptake - Saskatchewan

Abbreviations used in graphs:

Selected graphs - Canada

Selected graphs - Alberta

Selected graphs - British Columbia

Selected graphs - Manitoba

Selected graphs - Nova Scotia

Selected graphs - Ontario

Selected graphs - Quebec

Selected graphs - Saskatchewan

II. METHODS AND RESULTS OF THIS WORK

. Periodical updates accessible: Yes, up to 20210823, No, from 20210824

The three GitHub repositories created in this project are:

III. INNER WORKS OF THIS REPOSITORY

Scenario number within component study

A00 JOHN

A01 DELP

A02 IHME

A03 IMPE

A04 LANL

A05 SRIV

uptakes in this repository, since April 2021

IV. SELECTED GRAPHS FROM PREVIOUS UPTAKES

Selected graphs for Canada national level and provinces are displayed in separate web pages, links to which are provided below. This facilitates side-by-side comparisons across provinces and the national level.

Among the 5 available international periodically updated studies or models of COVID-19 pandemic, only TWO studies, DELP and IHME, provide subnational level estimates for some countries.

For Canada, DELP and IHME models provide subnational estimates for SEVEN provinces: Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, and Saskatchewan.

List of graphs

Logical order of graphs

Selected graphs - CANADA

Selected graphs - Alberta

Selected graphs - British Columbia

Selected graphs - Manitoba

Selected graphs - Nova Scotia

Selected graphs - Ontario

Selected graphs - Quebec

Selected graphs - Saskatchewan

Licenses / Copyrights of data and / or graphs used in this repository:

License, DOI, and suggested Citation of this reposirory

About

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Releases 2

Packages