Module 2: Epidemic propagation

We are starting a new module on modelling epidemic propagation.

Let's start off by analysing some of the data that is now available on the current COVID-19 pandemic.

Exploring COVID-19 data

In this notebook we will explore and analyse data on the COVID-19 pandemic. The aim is to use Julia's tools to analyse and visualise the data in different ways.

Here is an example of the kind of visualisation we will be able to produce:

Download and load data

We will need a couple of new packages. The data is in CSV format, i.e. Comma-Separated Values. This is a common data format in which observations, i.e. data points, are separated on different lines. Within each line the different data for that observation are separated by commas or other punctuation (possibly spaces and tabs).

We can load the data from a CSV using the File function from the CSV.jl package, and then convert it to a DataFrame:

A DataFrame is a standard way of storing heterogeneous data in Julia, i.e. a table consisting of columns with different types. As you can see from the display of the DataFrame object above, each column has an associated type, but different columns have different types, reflecting the type of the data in that column.

In our case, country names are stored as Strings, their latitude and longitude as Float64s and the (cumulative) case counts for each day as Int64s. .

Using the data

Since we need to manipulate the columns, let's rename them to something shorter. We can do this either in place, i.e. modifying the original DataFrame, or out of place, creating a new DataFrame. The convention in Julia is that functions that modify their argument have a name ending with ! (often pronounced "bang").

We can use the head function to see only the first few lines of the data.

Extracting useful information

How can we extract the list of all the countries? The country names are in the second column.

For some purposes we can think of a DataFrame.as a matrix and use similar syntax. For example, we can extract the second column:

It turns out that some countries are divided into provinces, so there are repetitions in the country column that we can eliminate with the unique function:

[Here we used string interpolation with $ to put the text into a Markdown string.]

You can also use Select to get a dropdown instead:

How can we extract the data for a particular country? First we need to know the exact name of the country. E.g. is the US written as "USA", or "United States"?

We could scroll through to find out, or filter the data to only look at a sample of it, for example those countries that begin with the letter "U".

One way to do this is with an array comprehension:

Array comprehension:

Note that this returns an array of booleans of the same length as the vector all_countries. We can now use this to index into the DataFrame:

We see that the correct spelling is "US". (And note how the different provinces of the UK are separated.)

Now we would like to extract the data for the US alone. How can we access the correct row of the table? We can again filter on the country name. A nicer way to do this is to use the filter function.

This is a higher-order function: its first argument is itself a function, which must return true or false. filter will return all the rows of the DataFrame that satisfy that predicate:

Here we have used an anonymous function with the syntax x -> ⋯. This is a function which takes the argument x and returns whatever is on the right of the arrow (->).

To extract a single row we need the index of the row (i.e. which number row it is in the DataFrame). The findfirst function finds the first row that satisfies the given predicate:

Now we can extract the data into a standard Julia Vector:

Note that we are only passing a single vector to the scatter function, so the $x$ coordinates are taken as the natural numbers $1$, $2$, etc.

Also note that the $y$-axis in this plot gives the cumulative case numbers, i.e. the total number of confirmed cases since the start of the epidemic up to the given date.

This is an example of a time series, i.e. a single quantity that changes over time.

Using dates

We would like to use actual dates instead of just the number of days since the start of the recorded data. The dates are given in the column names of the DataFrame:

Now we need to parse the date strings, i.e. convert from a string representation into an actual Julia type provided by the Dates.jl standard library package:

Since the year was not correctly represented in the original data, we need to manually fix it:

Exploratory data analysis

Working with cumulative data is often less intuitive. Let's look at the actual number of daily cases. Julia has a diff function to calculate the difference between successive entries of a vector:

Note that discrete data should always be plotted with points. The lines are just to guide the eye.

Cumulating data corresponds to taking the integral of a function and is a smoothing operation. Note that the cumulative data is indeed visually smoother than the daily data.

The oscillations in the daily data seem to be due to a lower incidence of reporting at weekends. We could try to smooth this out by taking a moving average, say over the past week:

Exponential growth

Simple models of epidemic spread often predict a period with exponential growth. Do the data corroborate this?

A visual check for this is to plot the data with a logarithmic scale on the $y$ axis (but a standard scale on the $x$ axis).

The reason for this is that if we observe a straight line on such a semi-logarithmic plot, we have

$$\log (y)\sim ax+b,$$

where we are using $\sim$ to denote approximate equality.

Hence, taking exponentials of both sides, we have

$$y\sim \exp (ax+b)=c{e}^{ax},$$

for some constant $c$.

Since the data contains some zeros, we need to replace those with NaNs ("Not a Number"), which Plots.jl interprets as a signal to break the line

Let's zoom on the part where the growth seems linear on this semi-log plot:

We see that there is a period lasting from around day 38 to around day 60 when the curve looks straight on the semi-log plot. This corresponds to the following date range:

i.e. the first 3 weeks of March. Fortunately the imposition of lockdown during the last 10 days of March (on different days in different US states) significantly reduced transmission.

We can fit a straight line using linear regression to this portion of the data.

Geographical data

Our data set contains more information: the geographical locations (latitude and longitude) of each country (or, rather, of a particular point that was chosen as being representative of that country).

Let's extract and plot the geographical information. To reduce the visual noise a bit we will only use those

If the province is missing we should use the country name instead:

Adding maps

We would also like to see the outlines of each country. For this we can use, for example, the data from Natural Earth, which comes in the form of shape files, giving the outlines in terms of latitude and longitude coordinates.

These may be read in using the Shapefile.jl package.

Now we would like to combine the geographical and temporal (time) aspects. One way to do so is to animate time:

Day 1

However, we should always be wary about visualisations such as these. Perhaps we should be plotting cases per capita instead of absolute numbers of cases. Or should we divide by the area of the country? Some countries, such as China and Canada, are divided into states or regions in the original data set – but others, such as the US, are not. You should always check exactly what is being plotted!

Unfortunately, published visualisations often hide some of this information. This emphasises the need to be able to get our hands on the data, create our own visualisations and draw our own conclusions.