1 Padrões Hence

Estrutura de diretórios
Nome de arquivo
Script R

1.1 Messagem do comando ‘git commit -m’

[optional scope]:

[optional body]

[optional footer(s)]

Types:

init: new project.
fix:
feature:
breaking change:

Type (Must be one of the following):

build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
docs: Documentation only changes
feat: A new feature
fix: A bug fix
perf: A code change that improves performance
refactor: A code change that neither fixes a bug nor adds a feature
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
test: Adding missing tests or correcting existing tests

Referências:

https://www.conventionalcommits.org/en/v1.0.0/
https://github.com/angular/angular/blob/22b96b9/CONTRIBUTING.md#commit

1.2 Code Organization

Put all library() calls and any hard-coded variables at the top of the script.
Use RStudio projects to organize your scripts, data, and output.
Modularize your code.
Configurações RStudio:
- Do not save your working directory.
  - RStudio -> Tools
    
    -> Global options -> General: Save workspace to .RData on exit=Never
    
    -> Global options -> Code -> Saving: Default text encoding=ASCII

1.3 Code itself

Use library() NOT require() when loading packages in scripts.
Do not use functions that change someone’s computer (e.g. install.packages, setwd, or rm). rm(list = ls()) only in the first script of a pipeline.
Comment incessantly but in one line. Do not comment obvious statements.
Follow a style and be consistent.

1.4 Code Style

Customizing styler package

1.4.1 File data name (validado)

empresa_AAAAMM-<nome significativo do arquivo com palavras separas por _>
lower case
do not use spaces
Manter o mesmo nome para arquivos ‘raw’ apenas substituindo espaços por _
separar grupos de nomes por - e dentro do grupo separar as palavras por _

1.4.2 Script file name

Geração de arquivos:

NN_etlN_<objetivo do script>-sufixo.R
- onde:
  - NN = 01, 02, … sequencial de execução do pipeline
  - sufixo opcional
- exemplo:
  - 00_etl2_adaptacao_template-direcional_202209-carteira_tab_direta.R => sufixo = direcional_202209-carteira_tab_direta
  - 00_etl1_raw_to_parquet.R => sufixo não informado

1.4.3 Organização do código

Comentar blocos de comandos de forma coesa e funcional utilizando identação com #.

Veja exemplo no arquivo:

hence-finance/ cessao-recebiveis-imobiliarios/ shiny-relatorio-cri/ src/ R/ 08_estrutura_dados_relatorio.R

1.4.4 Object names

snake_case.
variable names should be nouns and function names should be verbs.

1.4.5 Spacing

Commas: Always put a space after a comma, never before. Ex.: x[, 1]

1.4.6 Parentheses

mean(x, na.rm = TRUE)

if (debug) {
  show(x)
}

function(x) {}

1.4.7 {{ }}

max_by <- function(data, var, by) {
  data %>%
    group_by({{ by }}) %>%
    summarise(maximum = max({{ var }}, na.rm = TRUE))
}

1.4.8 Infix operators

The operators with high precedence: ::, :::, $, @, [, [[, ^, unary -, unary +, and : should never be surrounded by spaces:

height <- (feet * 12) + inches 
sqrt(x^2 + y^2)
df$z
x <- 1:10
  
~foo
tribble(
   ~col1, ~col2,
     "a",   "b"
)

complex right-hand side do need a space:

~ .x + .y

When used in tidy evaluation !! (bang-bang) and !!! (bang-bang-bang) (because have precedence equivalent to unary -/+)
- call(!!xyz)
Adding extra spaces is ok if it improves alignment of = or <-.

list(
  total = a + b + c,
  mean  = (a + b + c) / n
)

1.4.9 Function calls

Avoid assignment in function calls.
Funções que recebem um dataset devem definir este parâmetro como o primeiro.
Parâmetros devem iniciar com ‘p_’. Ex.: ^function(p_bucket, p_group)

1.4.10 Control flow

{ should be the last character on the line. Related code (e.g., an if clause, a function declaration, a trailing comma, …) must be on the same line as the opening brace.
The contents should be indented by two spaces.
} should be the first character on the line.

if (y < 0 && debug) {
  message("y is negative")
}

if (y == 0) {
  if (x > 0) {
    log(x)
  } else {
    message("x is negative or zero")
  }
} else {
  y^x
}

test_that("call1 returns an ordered factor", {
  expect_s3_class(call1(x, y), c("factor", "ordered"))
})

tryCatch(
  {
    x <- scan()
    cat("Total: ", sum(x), "\n", sep = "")
  },
  interrupt = function(e) {
    message("Aborted by user")
  }
)

1.4.11 If statements

If used, else should be on the same line as }.
& and | should never be used inside of an if clause because they can return vectors. Always use && and || instead.

x <- 5
if (x > 10) {
  message <- "big"
} else {
  message <- "small"
}

Avoid implicit type coercion (e.g. from numeric to logical) in if statements:

# Good
if (length(x) > 0) {
  # do something
}

NULL

# Bad
if (length(x)) {
  # do something
}

NULL

Switch statements
- Avoid position-based switch() statements (i.e. prefer names).
- Each element should go on its own line.
- Elements that fall through to the following element should have a space after =.
- Provide a fall-through error, unless you have previously validated the input.

# Good 
switch(x, 
  a = ,
  b = 1, 
  c = 2,
  stop("Unknown `x`", call. = FALSE)
)

Long lines
- Strive to limit your code to 80 characters per line.
- Para situações que não em cabe em uma linha de 80 caracteres, coloque um parâmetro por linha. Use o bom censo e priorize a legibilidade.

# Good
do_something_very_complicated(
  something = "that",
  requires = many,
  arguments = "some of which may be long"
)

1.4.12 Semicolons

Don’t put ; at the end of a line, and don’t use ; to put multiple commands on one line.

1.4.13 Assignment

Use <-, not =, for assignment.

1.4.14 Data

Use “, not ’, for quoting text. The only exception is when the text already contains double quotes and no single quotes.

# Good
"Text"
'Text with "quotes"'
'<a href="http://style.tidyverse.org">A link</a>'

# Bad
'Text'
'Text with "double" and \'single\' quotes'

1.4.15 Comments

Comentar grupos correlatos de comandos conforme script shiny-relatorio-cri/src/R/08_estrutura_dados_relatorio.R.

1.4.16 Functions

Naming

As well as following the general advice for object names, strive to use verbs for function names:

# Good
add_row()
permute()

# Bad
row_adder()
permutation()

Double-indent: Place each argument of its own double indented line.

long_function_name <- function(
    a = "a long argument",
    b = "another argument",
    c = "another long argument") {
  # As usual code is indented by two spaces.
}

return()
- usar sempre o return() para deixar claro o que é retornado pela função.
If your function is called primarily for its side-effects (like printing, plotting, or saving to disk), it should return the first argument invisibly. This makes it possible to use the function as part of a pipe. print methods should usually do this, like this example from httr:

print.url <- function(x, ...) {
  cat("Url: ", build_url(x), "\n", sep = "")
  invisible(x)
}

comments
- free
parameters
- o nome dos parâmetros deve ter o prefixo “p_”

hence_s3_read_file <- function(p_bucket,
                               p_group,
                               p_project,
                               p_dat_folder,
                               p_file_name,
                               p_method = "paws",
                               p_file_type = "parquet") {

  bucket <- p_bucket
  group <- p_group
  project <- p_project
  dat_folder <- p_dat_folder
  file_name <- p_file_name
  method <- p_method
  file_type <- p_file_type

  ...
}

1.4.17 Pipes

Usar o pipe do pacote magrittr, ou seja, %>%.
Whitespace
- %>% should always have a space before it, and should usually be followed by a new line. Caso %>% esteja sendo usado dentro de uma etapa, colocar em uma linha se couber.
- After the first step, each line should be indented by two spaces.

# passa de 80 caracteres
resultado <- dados %>%
  filter(conta %in% (eval(parse(text = text)) %>%
                       str_split(",") %>%
                       unlist)) %>%
  mutate(valor = credito - debito) %>%
  select(conta,ano_mes,valor) %>%
  group_by(ano_mes) %>%
  summarise(valor = sum(valor)) %>%
  set_names(c("mes","valor")) %>%
  setDT %>%
  .[month(mes) == 12, valor := NA] %>%
  fill(valor,.direction = "down")

# não passa de 80 caracteres
dados_brutos %>%
  map2(.y = dados_brutos %>% names %>% make_clean_names,
       function(arquivo,nome){
         hence_s3_write_file(
           p_bucket = g_bucket,
           p_group = g_group,
           p_project = g_project,
           p_dat_folder = g_dir_etl3,
           p_file_name = paste0("cosimat_202210-",nome,".parquet"),
           p_data = arquivo,
           p_file_type = "parquet")
       })

# Good
iris %>%
  group_by(Species) %>%
  summarize_if(is.numeric, mean) %>%
  ungroup() %>%
  gather(measure, value, -Species) %>%
  arrange(value)

# Bad
iris %>% group_by(Species) %>% summarize_all(mean) %>%
ungroup %>% gather(measure, value, -Species) %>%
arrange(value)

Assignment

iris_long <-
  iris %>%
  gather(measure, value, -Species) %>%
  arrange(-value)

iris_long <- iris %>%
  gather(measure, value, -Species) %>%
  arrange(-value)

iris %>%
  gather(measure, value, -Species) %>%
  arrange(-value) ->
  iris_long

ggplot2
- ggplot2 allows you to do data manipulation, such as filtering or slicing, within the data argument. Avoid this, and instead do the data manipulation in a pipeline before starting plotting.

# Good
iris %>%
  filter(Species == "setosa") %>%
  ggplot(aes(x = Sepal.Width, y = Sepal.Length)) +
  geom_point()

# Bad
ggplot(filter(iris, Species == "setosa"), aes(x = Sepal.Width, y = Sepal.Length)) +
  geom_point()

‘+’ deve seguir o mesmo estilo definido para o pipe.

1.4.18 Files

Writing functions in R4DS

Names
- If a file contains a single function, give the file the same name as the function.
- If a file contains multiple related functions, give it a concise, but evocative name.
- Deprecated functions should live in a file with deprec- prefix.
- Compatibility functions should live in a file with compat- prefix.

1.4.19 Documentation

Padrão roxygen2

1.4.20 Tests

Organisation
- The organisation of test files should match the organisation of R/ files: if a function lives in R/foofy.R, then its tests should live in tests/testthat/test-foofy.R.
- Use usethis::use_test() to automatically create a file with the correct name.
- The file name will be displayed in output in order to get context.

1.4.21 Error messages

Utilizar o comando stop para apresentar a mensagem de erro.

1.4.22 News

Each user-facing change to a package should be accompanied by a bullet in NEWS.md. Minor changes to documentation don’t need to be documented, but it’s worthwhile to draw attention to sweeping changes and to new vignettes.

1.4.23 Git commit messages

Mensagem clara e suscinta.

1.4.24 Gráficos

ggplot com plotly
thema: criar pacote
cor hence
fonte
data.table para as estruturas de dados em etl3

1.4.25 Análise Sobrevivência

Nome das variáveis principais do modelo:
- dat_inicio: data de início da análise.
- dat_evento: data em que ocorreu o evento ou data da censura.
- tempo_ate_evento: tempo entre a data de início e data de ocorrência do evento.
- evento: flag de identificação da ocorrência do evento: 1 = ocorreu o evento, 0 = não ocorreu o evento

1.4.26 Shiny

Funções genéricas devem iniciar com hence_.

1.4.27 Packages

No visible binding for global variable

R CMD check NOTE: no visible global function definition for ‘.’ #5436

another work-around is to use . <- NULL

Our recommended default is to call external functions using the package::function() syntax:

somefunction <- function(...) {
  ...
  x <- aaapkg::aaa_fun(...)
  ...
}

If you @import many packages, it increases the chance of function name conflicts. Save this for very special situations.

When developing your package, if you are experiencing these unbound global variables NOTEs you should (calcula_cesh: no visible binding for global variable ‘parcela’ where ‘parcela’ is a data.table variable):

Strive to define any unbound variables locally within a function.
Ensure that any functions or data from external packages (including utils, stats, etc.) have the correct @importFrom tag
Do not suppress this check in the .Renviron file and the solutions proposed here should remove the current need to do so
Any package wide unbound variables, which are typically syntactic sugar (e.g. :=), should be defined within the package description file inside a globalVariables() function, which should be a very short and maintainable list.

2 Etapas de criação de modelo

Preparação de dados
Análise exploratória de dados
Certificação a qualidade da base de dados
Aplicação de filtros
Separação do dataset em treinamento e teste
- To partition the data, the splitting of the orignal data set will be done in a stratified manner by making random splits in each of the outcome classes. This will keep the proportion of stroke patients approximately the same.
Preprocess the predictors
- Understand important predictor characteristics such as:
  - their individual distributions
  - the degree of missingness within each predictor
  - potentially unusual values within predictors
  - relationships between predictors, and
  - the relationship between each predictor and the response and so on.
  - Undoubtedly, as the number of predictors increases, our ability to carefully curate each individual predictor rapidly declines. But automated tools and visualizations are available that implement good practices for working through the initial exploration process such as Kuhn (2008) “The Caret Package.” and Wickham and Grolemund (2016) “R for Data Science: Import, Tidy, Transform, Visualize, and Model Data.”.
Explore relationships among the predictors and the response in the training and test set.
Feature engineering:
- normalização e mudança de escada.
- criação de novas variáveis.
- transformação
Feature selection
Validação das variáveis nas base de treinamento e teste
- distribuição da variável resposta
- distribuição das variáveis preditoras
Modelagem/treinamento do modelo
- tuning dos hiperparâmetros (cross-validation)
- geração de métricas de validação do modelo