Hence Knowledge Repo
  • Home
  1. Machine Learning / Engenharia
  2. Functional Programming Workshop
  • Processos
    • Gestão de Projetos
    • Padrões Hence
    • Ask Issues
    • Reproducible R scripts
  • Credito / Modelagem
    • Crédito Imobiliário - Métricas de Perda
    • Modelo de Crédito Imobiliário (Quod)
    • Probabilidade de recuperação de crédito usando análise de sobrevivência
    • Tutorial Análise de sobrevivencia
    • Advanced Survival Modelling for Consumer Credit
    • IFRS 9 (Book: Tiziano Bellini)
  • Machine Learning / Engenharia
    • Feature Engineering (Max Kuhn)
    • Machine Learning Engineering
    • NLP
    • R Packages Development Guide
    • Functional Programming Workshop
  • Relatorios
    • Recuperação de crédito e Perda Esperada - Direcional Taxas

On this page

  • 1 Imperative vs. Declarative Programming
    • 1.1 Imperative
    • 1.2 Declarative
  • 2 Functional Programming
  • 3 Writing functions in R
  • 4 The purrr package
    • 4.1 The map
    • 4.2 The reduce family
    • 4.3 The keep family
    • 4.4 The compose family
    • 4.5 The list family
  1. Machine Learning / Engenharia
  2. Functional Programming Workshop

Functional Programming Workshop

1 Imperative vs. Declarative Programming

1.1 Imperative

nums <- numeric()
for (i in 1:20) {
  if(i < 10) {
    nums[i] <- i
  }
}
print(nums)
[1] 1 2 3 4 5 6 7 8 9

1.2 Declarative

nums <- 1:20 %>% purrr::keep(~.x < 10)
print(nums)
[1] 1 2 3 4 5 6 7 8 9
# OR
nums <- c(1:20)[c(1:20) < 10]
print(nums)
[1] 1 2 3 4 5 6 7 8 9

2 Functional Programming

The functional programming paradigm borrows the mathematical idea of a function: a function will always return the same output if the same input is given. There is no way f(x) = y and f(x) = a. Functions that respect this rule are usually called pure functions. This makes it possible to treat functions as first class objects, that is to treat them the same way objects such as integers or strings are treated. In the example below, functions are given as arguments to the apply_stat function.

apply_stat <- function(fun, x) {
  stat <- fun(x)
  return(stat)
}

mean_nums <- apply_stat(mean, nums)
sd_nums   <- apply_stat(sd,   nums)
min_nums  <- apply_stat(min,  nums)
max_nums  <- apply_stat(max,  nums)

mean_nums; sd_nums; min_nums; max_nums
[1] 5
[1] 2.738613
[1] 1
[1] 9

3 Writing functions in R

References:

  • https://www.youtube.com/watch?v=3uK1OzA7CTs

In order to write pure functions, some good practices are recommended:

  • Avoid using global variables inside functions:
    • Use default values if needed
# Don't
bonus <- 45

adds_bonus <- function(salary){
  bonus_salary <- salary + bonus
  return(bonus_salary)
}

adds_bonus(100)
[1] 145
# Do
adds_bonus_2 <- function(salary, bonus = 45){
  bonus_salary <- salary + bonus
  return(bonus_salary)
}

adds_bonus_2(100, 45)
[1] 145
  • Avoid writing functions that do more than one thing and explicitly state what the function returns
# Don't
melts_saves_df <- function(df){
  
  melted_df <- df %>%
    reshape2::melt()
  
  write.csv(melted_df, "melted_df.csv")
  
}

melts_saves_df(mtcars)
Error in `loadNamespace()`:
! there is no package called 'reshape2'
# Do
melts_df <- function(df) {
  melted_df <- df %>%
    reshape2::melt()
  return(melted_df)
}

saves_melted_df <- function(df, file_name){
  write.csv(df, file_name)
}

melted_df <- melts_df(mtcars)
Error in `loadNamespace()`:
! there is no package called 'reshape2'
saves_melted_df(melted_df, "melted_df2.csv")
Error:
! object 'melted_df' not found

You can also apply both functions with the pipe:

mtcars |> 
  melts_df() |> 
  saves_melted_df(file_name = "melted_df2.csv")
Error in `loadNamespace()`:
! there is no package called 'reshape2'

4 The purrr package

References:

  • https://www.youtube.com/watch?v=H3ao7LzcvW8
  • https://lente.dev/advanced-purrr.pdf
  • https://purrr.tidyverse.org/reference/index.html
  • https://www.youtube.com/watch?v=vb1lD9_AFcU&t=2708s

The purrr package provides functions that allow R code to be written in a simple and functional way. With purrr, we can:

  • Work with lists in an efficient way;
  • Replace the usage of loops or avoiding repetitive work;
  • Improve code readability and generalization;

The functions in purrr can be divided into a few families, where the map family is the most important one.

4.1 The map

4.1.1 map

The map function takes as arguments a vector or list and a function. It will apply this function to the elements of the vector or list. The main point of map is to apply the same functions many times changing one or more parameters, “mapping” the function to these parameters.

In the example below, we use map to calculate the statistics of a given numeric vector.

stats <-
  list(min, max, median, mean, sd) %>%
  purrr::map(apply_stat, nums)

Which is equivalent to the following, as the expression inside the map function is the same as what the apply_stat() does. This means that we can use inline functions inside map (which don’t exist in the environment), using the generic “.x” to indicate what is the function parameter:

stats_v2 <-
  list(min, max, median, mean, sd) %>%
  purrr::map(~.x(nums))

# min
stats[[1]]; stats_v2[[1]]
[1] 1
[1] 1
# max
stats[[2]]; stats_v2[[2]]
[1] 9
[1] 9
# median
stats[[3]]; stats_v2[[3]]
[1] 5
[1] 5

In order to easily access the results, we can use the get function from base R:

stats_names <- c("min", "max", "median", "mean", "sd")

named_stats <-
  stats_names %>%
  map(~get(.x)(nums)) %>%
  set_names(stats_names)

names(named_stats)
[1] "min"    "max"    "median" "mean"   "sd"    
named_stats$min; named_stats$median
[1] 1
[1] 5

Above, we had a list of functions to apply to same vector, but we can also have a list of vectors and apply the same function to all of them:

list(c(1:10), rnorm(10), c(15:35)) %>%
  purrr::map(mean)
[[1]]
[1] 5.5

[[2]]
[1] -0.3765446

[[3]]
[1] 25

Thus, we can see that map takes vectors or lists and pass them as the first argument to a function: map(.x, .f). However, if the function .f accepts more than one argument, these arguments can be passed to .f: map(.x, .f, argument = value). In this case, argument = value will be passed to .f every time it called:

list(c(1:10,NA), c(rnorm(10),NA), c(15:35,NA)) %>%
  purrr::map(mean)
[[1]]
[1] NA

[[2]]
[1] NA

[[3]]
[1] NA

Now, setting na.rm = T

list(c(1:10,NA), c(rnorm(10),NA), c(15:35,NA)) %>%
  purrr::map(mean, na.rm = T)
[[1]]
[1] 5.5

[[2]]
[1] -0.1667445

[[3]]
[1] 25
list(iris, mtcars) |> 
  purrr::map(head, n  = 1)
[[1]]
  Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1          5.1         3.5          1.4         0.2  setosa

[[2]]
          mpg cyl disp  hp drat   wt  qsec vs am gear carb
Mazda RX4  21   6  160 110  3.9 2.62 16.46  0  1    4    4

There are several variations of the map function: map2, map_if, map_at. They all have the same principle: apply the same function to the element of a vector or list. map_if will apply the function to the elements of the list that satisfy a predicate function .p. map_at will apply the function to elements of the list at certain positions, the ones specified in the .at argument. Let’s look into map2 with more detail.

4.1.2 map2

map2 was built to handle situations in which one needs to apply the same function to all the element of a list, but the function takes more than one argument and each element of the list takes a different value to this argument. Suppose you have a list of data frames and you want to save them in your directory with different names. In the example below, .x will be passed as the first argument to write.csv and .y will be passed as the second argument to write.csv

Ex 1:

dfs <- list(
  as.data.table(mtcars,keep.rownames = TRUE),
  data.frame(AirPassengers)
  )

file_names <- c("mtcars_df.csv","air_passengers_df.csv")

map2(.x = dfs, .y = file_names, write.csv)
[[1]]
NULL

[[2]]
NULL

Note that the row names were saved to both csv files. We can set the argument row.names = FALSE to both calls of the function like this:

file_names <- c("mtcars_df_sem_rn.csv","air_passengers_df_sem_rn.csv")

map2(.x = dfs, .y = file_names, write.csv, row.names = FALSE)
[[1]]
NULL

[[2]]
NULL

Ex 2:

x <- list(1, 10, 100)
y <- list(1, 2, 3)
purrr::map2(x, y, ~.x + .y)
[[1]]
[1] 2

[[2]]
[1] 12

[[3]]
[1] 103

If we want to map more than two arguments, we can use pmap(), the generalization for a p number of parameters:

And, where the names of the list (e.g. x, y, z) will already be ‘matched’ by the function parameter names:

x <- list(1, 10, 100)
y <- list(1,   2,  3)
z <- list(15, 25, 35)

ops <- function(x, y, z){ x + (y * z)}
purrr::pmap(list(x = x, y = y, z = z), ops)
[[1]]
[1] 16

[[2]]
[1] 60

[[3]]
[1] 205

4.2 The reduce family

The reduce family functions are built to “reduce a list to a single value by iteratively applying a binary function”, according to its help page.

Ex 1:

letras <- list(
  LETTERS,
  LETTERS[1:10],
  LETTERS[7:14]
)

reduce(letras, intersect)
[1] "G" "H" "I" "J"

Ex 2:

dfs_list <- list(
  minusculas = data.table(id = 1:10, minusculas = letters[1:10]),
  maiusculas = data.table(id = 1:10, maiusculas = LETTERS[1:10]),
  nomes = data.table(
    id = 1:5,
    nomes = c("Bruna","Igor","Milton","Rodrigo","Victor"))
)

reduce(dfs_list, merge, by = "id")
Key: <id>
      id minusculas maiusculas   nomes
   <int>     <char>     <char>  <char>
1:     1          a          A   Bruna
2:     2          b          B    Igor
3:     3          c          C  Milton
4:     4          d          D Rodrigo
5:     5          e          E  Victor

Like in map, we can set the values to the other argument of the function .f

reduce(dfs_list, merge, by = "id", all = TRUE)
Key: <id>
       id minusculas maiusculas   nomes
    <int>     <char>     <char>  <char>
 1:     1          a          A   Bruna
 2:     2          b          B    Igor
 3:     3          c          C  Milton
 4:     4          d          D Rodrigo
 5:     5          e          E  Victor
 6:     6          f          F    <NA>
 7:     7          g          G    <NA>
 8:     8          h          H    <NA>
 9:     9          i          I    <NA>
10:    10          j          J    <NA>

4.3 The keep family

These functions are used to filter vectors:

# Let's use this space to introduce and explain the pluck family. pluck() is a
# function used to pull elements of a list. pluck(.x, ...), in which .x is a list
# and ... accepts a numeric, indicating the position of the element you want to pull,
# or a string, indicating the name of the element you want to pull

# Extracting the names that contain the letter "R". Note that stringr::str_detect
# is case sensitive.

dfs_list %>%
  purrr::pluck("nomes") %>%
  dplyr::pull(nomes) %>%
  purrr::keep(stringr::str_detect, "R|r")
[1] "Bruna"   "Igor"    "Rodrigo" "Victor" 

or:

dfs_list %>%
  purrr::pluck("nomes") %>%
  dplyr::pull(nomes) %>%
  purrr::discard(stringr::str_detect, "R|r")
[1] "Milton"

4.4 The compose family

The compose family is used to create composed functions, just like in mathematics.

lista <- list(
  c(rep("A",4)),
  c("a",letters),
  dfs_list$nomes$nomes
)

lista %>%
  map(compose(length, unique))
[[1]]
[1] 1

[[2]]
[1] 26

[[3]]
[1] 5

which is equivalent to:

lista %>%
  map(unique) %>%
  map(length)
[[1]]
[1] 1

[[2]]
[1] 26

[[3]]
[1] 5

So far we’ve looked into functions that allow us to apply functions into elements of lists. However it is also useful sometimes to alter the structure of a list. Let’s take a look into these functions:

4.5 The list family

4.5.1 flatten

flatten is used to trasnform a deep list (list of lists) into a single list:

lista <- 
  list(
    a = list(nums = 1:5),
    b = list(nums = 6:10)
  )

purrr::flatten(lista)
$nums
[1] 1 2 3 4 5

$nums
[1]  6  7  8  9 10

4.5.2 transpose

transpose is used to invert the structure of the list:

purrr::transpose(lista)
$nums
$nums$a
[1] 1 2 3 4 5

$nums$b
[1]  6  7  8  9 10