1 Introducing R

R is a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques. It has gained widespread use in academia and industry. R offers a wider array of functionality than a traditional statistics package, such as SPSS and is composed of core (base) functionality, and is expandable through libraries hosted on CRAN. CRAN is a network of ftp and web servers around the world that store identical, up-to-date, versions of code and documentation for R.

Commands are sent to R using either the terminal / command line or the R Console which is installed with R on either Windows or OS X. On Linux, there is no equivalent of the console, however, third party solutions exist. On your own machine, R can be installed from here.

Normally RStudio is used to implement R coding. RStudio is an integrated development environment (IDE) for R and provides a more user-friendly front-end to R than the front-end provided with R.

To run R or RStudio, just double click on the R or RStudio icon. Throughout this course, we will be using RStudio:

Fig. 1. RStudio features.

If you would like to know more about the various features of RStudio, watch this video

2 Setting the working directory

Before we start any analysis, ensure to set the path to the directory where we are working. We can easily do that with setwd(). Please replace in the following line the path to the folder where you have placed this file -and where the data folder lives.

#setwd('../data/sar.csv')
#setwd('.')

Note: It is good practice to not include spaces when naming folders and files. Use underscores or dots.

You can check your current working directory by typing:

getwd()

## [1] "/Users/franciscorowe/Dropbox/Francisco/Research/github_projects/sl/code"

3 R as a calculator

20 / 10

## [1] 2

20 * 10

## [1] 200

20 + 10

## [1] 30

20 - 10

## [1] 10

20^10

## [1] 1.024e+13

sqrt(20)

## [1] 4.472136

log(20)

## [1] 2.995732

log(20+10*(4/2))

## [1] 3.688879

4 R Scripts and Notebooks

An R script is a series of commands that you can execute at one time and help you save time. So you don’t repeat the same steps every time you want to execute the same process with different datasets. An R script is just a plain text file with R commands in it.

To create an R script in RStudio, you need to

• Open a new script file: File > New File > R Script

• Write some code on your new script window by typing eg. mtcars

• Run the script. Click anywhere on the line of code, then hit Ctrl + Enter (Windows) or Cmd + Enter (Mac) to run the command or select the code chunk and click run on the right-top corner of your script window. If do that, you should get:

mtcars

##                      mpg cyl  disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4           21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag       21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710          22.8   4 108.0  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive      21.4   6 258.0 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout   18.7   8 360.0 175 3.15 3.440 17.02  0  0    3    2
## Valiant             18.1   6 225.0 105 2.76 3.460 20.22  1  0    3    1
## Duster 360          14.3   8 360.0 245 3.21 3.570 15.84  0  0    3    4
## Merc 240D           24.4   4 146.7  62 3.69 3.190 20.00  1  0    4    2
## Merc 230            22.8   4 140.8  95 3.92 3.150 22.90  1  0    4    2
## Merc 280            19.2   6 167.6 123 3.92 3.440 18.30  1  0    4    4
## Merc 280C           17.8   6 167.6 123 3.92 3.440 18.90  1  0    4    4
## Merc 450SE          16.4   8 275.8 180 3.07 4.070 17.40  0  0    3    3
## Merc 450SL          17.3   8 275.8 180 3.07 3.730 17.60  0  0    3    3
## Merc 450SLC         15.2   8 275.8 180 3.07 3.780 18.00  0  0    3    3
## Cadillac Fleetwood  10.4   8 472.0 205 2.93 5.250 17.98  0  0    3    4
## Lincoln Continental 10.4   8 460.0 215 3.00 5.424 17.82  0  0    3    4
## Chrysler Imperial   14.7   8 440.0 230 3.23 5.345 17.42  0  0    3    4
## Fiat 128            32.4   4  78.7  66 4.08 2.200 19.47  1  1    4    1
## Honda Civic         30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
## Toyota Corolla      33.9   4  71.1  65 4.22 1.835 19.90  1  1    4    1
## Toyota Corona       21.5   4 120.1  97 3.70 2.465 20.01  1  0    3    1
## Dodge Challenger    15.5   8 318.0 150 2.76 3.520 16.87  0  0    3    2
## AMC Javelin         15.2   8 304.0 150 3.15 3.435 17.30  0  0    3    2
## Camaro Z28          13.3   8 350.0 245 3.73 3.840 15.41  0  0    3    4
## Pontiac Firebird    19.2   8 400.0 175 3.08 3.845 17.05  0  0    3    2
## Fiat X1-9           27.3   4  79.0  66 4.08 1.935 18.90  1  1    4    1
## Porsche 914-2       26.0   4 120.3  91 4.43 2.140 16.70  0  1    5    2
## Lotus Europa        30.4   4  95.1 113 3.77 1.513 16.90  1  1    5    2
## Ford Pantera L      15.8   8 351.0 264 4.22 3.170 14.50  0  1    5    4
## Ferrari Dino        19.7   6 145.0 175 3.62 2.770 15.50  0  1    5    6
## Maserati Bora       15.0   8 301.0 335 3.54 3.570 14.60  0  1    5    8
## Volvo 142E          21.4   4 121.0 109 4.11 2.780 18.60  1  1    4    2

• Save the script: File > Save As, select your required destination folder, and enter any filename that you like, provided that it ends with the file extension .R

An R Notebook is an R Markdown document with descriptive text and code chunks that can be executed independently and interactively, with output visible immediately beneath a code chunk - see Xie, Allaire, and Grolemund (⊕2019Xie, Yihui, JJ Allaire, and Garrett Grolemund. 2019. R Markdown: The Definitive Guide. CRC Press, Taylor & Francis, Chapman & Hall Book. https://bookdown.org/yihui/rmarkdown/.).

To create an R Notebook, you need to:

• Open a new script file: File > New File > R Notebook

Fig. 2. YAML metadata for notebooks.

• Insert code chunks, either:

1. use the Insert command on the editor toolbar;
2. use the keyboard shortcut Ctrl + Alt + I or Cmd + Option + I (Mac); or,
3. type the chunk delimiters ```{r} and ```

In a chunk code you can produce text output, tables, graphics and write code! You can control these outputs via chunk options which are provided inside the curly brackets eg.

Fig. 3. Code chunk example. Details on the various options: https://rmarkdown.rstudio.com/lesson-3.html

• Execute code: hit “Run Current Chunk”, Ctrl + Shift + Enter or Cmd + Shift + Enter (Mac)

• Save an R notebook: File > Save As. A notebook has a *.Rmd extension and when it is saved a *.nb.html file is automatically created. The latter is a self-contained HTML file which contains both a rendered copy of the notebook with all current chunk outputs and a copy of the *.Rmd file itself.

Rstudio also offers a Preview option on the toolbar which can be used to create pdf, html and word versions of the notebook. To do this, choose from the drop-down list menu knit to ...

5 Getting Help

You can use help or ? to ask for details for a specific function:

help(sqrt) #or ?sqrt

Example sqrt

example(sqrt)

## 
## sqrt> require(stats) # for spline
## 
## sqrt> require(graphics)
## 
## sqrt> xx <- -9:9
## 
## sqrt> plot(xx, sqrt(abs(xx)),  col = "red")

## 
## sqrt> lines(spline(xx, sqrt(abs(xx)), n=101), col = "pink")

6 Variables and objects

An object is a data structure having attributes and methods. In fact, everything in R is an object!

A variable is a type of data object. Data objects also include list, vector, matrices and text.

• Creating a data object

In R a variable can be created by using the symbol <- to assign a value to a variable name. The variable name is entered on the left <- and the value on the right. Note: Data objects can be given any name, provided that they start with a letter of the alphabet, and include only letters of the alphabet, numbers and the characters . and _. Hence AgeGroup, Age_Group and Age.Group are all valid names for an R data object. Note also that R is case-sensitive, to agegroup and AgeGroup would be treated as different data objects.

To save the value 28 to a variable (data object) labelled age, run the code:

age <- 28

• Inspecting a data object

To inspect the contents of the data object age run the following line of code:

age

## [1] 28

Find out what kind (class) of data object age is using:

class(age) 

## [1] "numeric"

Inspect the structure of the age data object:

str(age) 

##  num 28

• The vector data object

What if we have more than one response? We can use the c( ) function to combine multiple values into one data vector object:

age <- c(28, 36, 25, 24, 32)
age

## [1] 28 36 25 24 32

class(age) #Still numeric..

## [1] "numeric"

str(age) #..but now a vector (set) of 5 separate values

##  num [1:5] 28 36 25 24 32

Note that on each line in the code above any text following the # character is ignored by R when executing the code. Instead, text following a # can be used to add comments to the code to make clear what the code is doing. Two marks of good code are a clear layout and clear commentary on the code.

7 Basic Data Types

There are a number of data types. Four are the most common. In R, numeric is the default type for numbers. It stores all numbers as floating-point numbers (numbers with decimals). This is because most statistical calculations deal with numbers with up to two decimals.

• Numeric

num <- 4.5 # Decimal values
class(num)

## [1] "numeric"

• Integer

int <- as.integer(4) # Natural numbers. Note integers are also numerics.
class(int)

## [1] "integer"

• Character

cha <- "are you enjoying this?" # text or string. You can also type `as.character("are you enjoying this?")`
class(cha)

## [1] "character"

• Logical

log <- 2 < 1 # assigns TRUE or FALSE. In this case, FALSE as 2 is greater than 1
log

## [1] FALSE

class(log)

## [1] "logical"

TASK #1: Create a variable called income, with the following five respondent values: high, low, low, middle, high.

8 Random Variables

In statistics, we differentiate between data to capture:

• Qualitative attributes categorise objects eg.gender, marital status. To measure these attributes, we use Categorical data which can be divided into:

  • Nominal data in categories that have no inherent order eg. gender
  • Ordinal data in categories that have an inherent order eg. income bands

• Quantitative attributes:

  • Discrete data: count objects of a certain category eg. number of kids, cars
  • Continuous data: precise numeric measures eg. weight, income, length.

In R these three types of random variables are represented by the following types of R data object:

We have already encountered the R data type numeric. The next section introduces the factor data type.

9 Factor

gender <- c("female","male","male","female","female") # create a gender variable
gender <- factor(gender) # replace character vector with a factor version
gender

## [1] female male   male   female female
## Levels: female male

class(gender)

## [1] "factor"

str(gender)

##  Factor w/ 2 levels "female","male": 1 2 2 1 1

levels(gender)

## [1] "female" "male"

gender2 <- factor(gender, levels= c("male", "female"))
gender2

## [1] female male   male   female female
## Levels: male female

income <- ordered(income, levels = c("low", "middle", "high"))
income

## [1] high   low    low    middle high  
## Levels: low < middle < high

class(income)

## [1] "ordered" "factor"

str(income)

##  Ord.factor w/ 3 levels "low"<"middle"<..: 3 1 1 2 3

levels(income)

## [1] "low"    "middle" "high"

travel_mode <- c("train", "bicycle", "bus", "car", "car")

10 Data Frames

R stores different types of data using different types of data structure. Survey data are normally stored as a data.frame. A data.frame for a survey contains one row per respondent and one column per respondent attribute (eg. age, gender and income).

For example, if we have:

age <- c(28, 36, 25, 24, 32)
gender <- c("female", "male", "male", "female", "female")
income <- c("high", "low", "low", "middle", "high")

df <- data.frame(age, gender, income)
df # or use view(data)

##   age gender income
## 1  28 female   high
## 2  36   male    low
## 3  25   male    low
## 4  24 female middle
## 5  32 female   high

str(df) # or use glimpse(data) 

## 'data.frame':    5 obs. of  3 variables:
##  $ age   : num  28 36 25 24 32
##  $ gender: chr  "female" "male" "male" "female" ...
##  $ income: chr  "high" "low" "low" "middle" ...

df # whole data.frame

##   age gender income
## 1  28 female   high
## 2  36   male    low
## 3  25   male    low
## 4  24 female middle
## 5  32 female   high

df[1, 1] # contents of first row and column

## [1] 28

df[2, 2:3] # contents of the second row, second and third columns

##   gender income
## 2   male    low

df[1, ] # first row, ALL columns [the default if no columns specified]

##   age gender income
## 1  28 female   high

df[ ,1:2] # ALL rows; first and second columns

##   age gender
## 1  28 female
## 2  36   male
## 3  25   male
## 4  24 female
## 5  32 female

df[c(1,3,5), ] # rows 1,3,5; ALL columns

##   age gender income
## 1  28 female   high
## 3  25   male    low
## 5  32 female   high

df[ , 2] # ALL rows; second column (by default results containing only 

## [1] "female" "male"   "male"   "female" "female"

             #one column are converted back into a vector)
df[ , 2, drop=FALSE] # ALL rows; second column (returned as a data.frame)

##   gender
## 1 female
## 2   male
## 3   male
## 4 female
## 5 female

df[,"age"] # variable name in quotes inside the square brackets

## [1] 28 36 25 24 32

df$age # variable name prefixed with $ and appended to the data.frame name

## [1] 28 36 25 24 32

# or you can use attach
attach(df)

## The following objects are masked _by_ .GlobalEnv:
## 
##     age, gender, income

age # but be careful if you already have an age variable in your local workspace

## [1] 28 36 25 24 32

names(df)

## [1] "age"    "gender" "income"

11 Read Data

rm(list=ls()) # rm for targeted deletion / ls for listing all existing objects

load("../data/data_qlfs.RData") 
# NOTE: always ensure your are setting the correct directory leading to the data. 
# It may differ from your existing working directory

# 1
class(qlfs)
# 2 & 3
str(qlfs)

names(qlfs)

##  [1] "Case_ID"        "Hhold_ID"       "Family_ID"      "Person_ID"     
##  [5] "HHoldSize"      "FamilySize"     "Age"            "AgeGroup"      
##  [9] "Sex"            "MaritalStatus"  "EthnicGroup"    "CountryOfBirth"
## [13] "Religion"       "WorkStatus"     "NSSEC"          "HighestQual"   
## [17] "DegreeClass"    "HoursWorked"    "GrossPay"       "NetPay"        
## [21] "LastMoved"      "TravelMode"     "TravelTime"     "GovtRegion"    
## [25] "Tenure"         "FamilyType"     "YoungestChild"  "FamilyToddlers"

12 Data Distributions

We can think of two types of data distributions: theorethical distributions and empirical distributions.

# attach our dataset (ie. qlfs) to the R search path.
attach(qlfs)

# create table with counts
counts <- table(EthnicGroup, Sex)
counts

##                                        Sex
## EthnicGroup                              Male Female
##   White                                 34672  37951
##   Mixed/multiple ethnic background        267    360
##   Indian                                  836    851
##   Pakistani                               521    534
##   Bangladeshi                             190    201
##   Chinese                                 143    204
##   Other Asian background                  360    420
##   Black/African/Caribbean/Black British   755    964
##   Other ethnic group                      479    511

# row proportion
prop.table(counts, 1)

##                                        Sex
## EthnicGroup                                  Male    Female
##   White                                 0.4774245 0.5225755
##   Mixed/multiple ethnic background      0.4258373 0.5741627
##   Indian                                0.4955542 0.5044458
##   Pakistani                             0.4938389 0.5061611
##   Bangladeshi                           0.4859335 0.5140665
##   Chinese                               0.4121037 0.5878963
##   Other Asian background                0.4615385 0.5384615
##   Black/African/Caribbean/Black British 0.4392088 0.5607912
##   Other ethnic group                    0.4838384 0.5161616

# column proportion
prop.table(counts, 2)
# missing information ethinicity by sex
table(Sex, is.na(EthnicGroup))

# one variable
ggplot(data=qlfs) +
  geom_bar( aes( x= EthnicGroup) )

# two variables: split by sex
ggplot(data=qlfs) +
  geom_bar( aes( x= EthnicGroup, fill= Sex) )

# create histogram
hist(NetPay)

# create new data frame
df <- qlfs %>% filter(!is.na(NetPay)) %>% 
    filter(NetPay >= 0)
# remember to:
detach(qlfs)
# and:
attach(df)

# create new histogram 
  # 1) the base
hist_pay <- ggplot(data = df) +
  # 2) the histogram
  geom_histogram(bins = 100, aes(x = NetPay, y = ..density..)) +
  # 3) overlay density plot
  geom_density(alpha=0.5, colour="#FF6666", aes(x = NetPay))
hist_pay

box_pay <- ggplot(data = df) +
  geom_boxplot(aes(x = Sex, y= NetPay))
box_pay

13 Appendix: Concepts and Functions to Remember