A tour of {knitr} engines: {knitr} not only knits R

class: center, middle, inverse, title-slide

.title[
# A tour of {knitr} engines: {knitr} not only knits R
]
.author[
### Christophe Dervieux
]
.date[
### userR!2022 - 22nd of June
]

---

# {knitr} in the the R Markdown workflow

.center.f3[.ba.b--blue.pa2.br4[`knitr::knit()`] + Pandoc (+ LaTeX for PDF) = `rmarkdown::render()`]

.fl.w-50.pa2[
.center[![:scale 100%, R Markdown wizard illustration by Alison Horst](rmarkdown_wizards.png)]
.source-fig.center[Source: https://github.com/allisonhorst/stats-illustrations]
]

.fl.w-50.pa2[
.center[![:scale 80%, Diagram showing the different step of rendering, using knitr first from Rmd to md, then Pandoc from md to output format like HTML, DOC and PDF (using LaTeX)](https://bookdown.org/yihui/rmarkdown-cookbook/images/workflow.png)]
.source-fig.center[
source: [R Markdown Cookbook](https://bookdown.org/yihui/rmarkdown-cookbook)
]
]

???

First a bit of context regarding **knitr** in the R Markdown process. R Markdown
is a tool to create some publication in several output formats by writing text
and executable code together in the same document. **knitr** is the tool in the
chain that will bring the literate programming feature by sewing source code and
result with other text in the knitting process.

---

# What is a {knitr} engine ?

.subtitle[Using R as example]

.fl.w-50.pa2[

````markdown
```{r setup, include = FALSE}
knitr::opts_chunk$set(echo = FALSE)
```
````
]

.fl.w-50.pa2[

````markdown
```{<engine> <label>, <keys = values>}
<code content>
```
````
]

.fl.w-100[
* `<engine>` defines **how** the `<code content>` will be processed,
* `<keys = values>` are **configurations** for the engine,
* `<label>` is the name of the chunk</br>(equivalent to `label = <label>`).
]

???

Usually, R engine is used with **knitr** chunks to evaluate R code and showing source and outputs depending on providing options. Syntax looks this way with engine, a label and some option. An engine is the main component of **knitr** which will take code chunk content and process it so that source and results can be sewed in the document as text in the Markdown output.

---

# There is more than just R !

.subtitle[Meet the other engines !]

A subset of available engines

```r
names(knitr::knit_engines$get())
```

```
##  [1] "awk"       "bash"      "coffee"    "gawk"      "groovy"    "haskell"  
##  [7] "lein"      "mysql"     "node"      "octave"    "perl"      "psql"     
## [13] "Rscript"   "ruby"      "sas"       "scala"     "sed"       "sh"       
## [19] "stata"     "zsh"       "asis"      "asy"       "block"     "block2"   
## [25] "bslib"     "c"         "cat"       "cc"        "comment"   "css"      
## [31] "ditaa"     "dot"       "embed"     "exec"      "fortran"   "fortran95"
## [37] "go"        "highlight" "js"        "julia"     "python"    "R"        
## [43] "Rcpp"      "sass"      "scss"      "sql"       "stan"      "targets"  
## [49] "tikz"      "verbatim"
```

.center[Let's take a tour! 🚌]

???

This principle is used for various engines and not just evaluating code chunk content with R.

Let's take a tour !

---
layout: true

# Processing chunk content as-is

---

.subtitle[Meet the `verbatim` engine]

.center.f2[Include chunk content in a code block]

.fl.w-50.pa2[
.ba[
.title-file[.Rmd before knitting]

`````markdown
Let's show an example of Rmd file content:

*````{verbatim, lang = "markdown"}
We can output arbitrary content **verbatim**.

```{r}
1 + 1
```

The content can contain inline code like
`r pi * 5^2`, too.
````
`````
]
]

.fl.w-50.pa2[
.ba[
.title-file[.md after knitting]

`````markdown
Let's show an example of Rmd file content:

*````markdown
We can output arbitrary content **verbatim**.

```{r}
1 + 1
```

The content can contain inline code like
`r pi * 5^2`, too.
````
`````
]
]

---

.subtitle[Meet the `embed` engine]

.center.f2[Include file content in a code block]

.ba[
.title-file[.Rmd before knitting]

````markdown
*```{embed, file = "macros.js"}
```
````
]

.ba.mt2[
.title-file[.md after knitting]

````default
*```js
remark.macros.scale = function(w, alt="") {
  var url = this;
  return '<img src="' + url + '" style="width: ' + w + ';" alt="' + alt + '" />';
};
remark.macros.scaleh = function(h, alt="") {
  var url = this;
  return '<img src="' + url + '" style="height: ' + h + ';" alt="' + alt + '" />';
};
```
````
]

---

.subtitle[Look at the `asis` engine]

.fl.w-60.pa2[

````default
```{r}
getRandomNumber <- function() {
  sample(1:6, 1)
}
```
*```{asis, echo = getRandomNumber() == 4}
According to https://xkcd.com/221/,
we just generated a **true** random number!
```

````
]

.fl.w-40.pa2[
![](https://imgs.xkcd.com/comics/random_number.png)
.source-fig.right[https://xkcd.com/221/]
]

---
.subtitle[Look at the `asis` engine]

.fl.w-50.pa2[
.f3[For `getRandomNumber() != 4`]

````default

```r
getRandomNumber <- function() {
  sample(1:6, 1)
}
```

````
]

.fl.w-50.pa2[
.f3[For `getRandomNumber() == 4`]

````default

```r
getRandomNumber <- function() {
  sample(1:6, 1)
}
```

According to https://xkcd.com/221/,
we just generated a **true** random number!

````
]

???

https://bookdown.org/yihui/rmarkdown-cookbook/eng-asis.html

---
layout: false
# Adding dependencies

.subtitle[Include CSS and JS easily in HTML]

.fl.w-50.pa2[

````markdown
```{css, echo=FALSE}
/* Some CSS code that will will be included in HTML document a `<style>` tag. */
```
````
]

.fl.w-50.pa2[

````markdown
```{js, echo=FALSE}
// some JS code that will be included in HTML document in a`<script>` tag.
```
````
]

.fl.w-100.pa2[
Useful to customize output directly from the Rmd file without external resource
]

---
layout: true

# Running other tools than R

---

.subtitle[Some built-in support]

.fl.w-50.pa2[

````markdown
```{python}
import os
os.env
```
````
]

.fl.w-50.pa2[

````markdown
```{stata}
sysuse auto
summarize
```
````
]

.fl.w-50.pa2[

````markdown
```{bash}
ls *.Rmd | head -n 5
```
````
]

.fl.w-50.pa2[

````markdown
```{perl}
$test = "jello world";
$test =~ s/j/h/;
print $test
```
````
]

.fl.w-100[
Chunk content is passed to the tools through `system2()`
]

---

.subtitle[Extend using `exec` engine]

Using `node` CLI (which require `.js` extension for scripts)

````markdown
```{exec, command='node', engine.opts = list(ext = ".js")}
function Display(x) { console.log(`Your number is ${x}`); }
Display(100);
```
````
````markdown
```javascript
function Display(x) { console.log(`Your number is ${x}`); }
Display(100);
```

```
## Your number is 100
```
````

More in [knitr-examples repo](https://github.com/yihui/knitr-examples): [`124-exec-engine.Rmd`](https://github.com/yihui/knitr-examples/blob/master/124-exec-engine.Rmd) 
???

The `exec` engine allows to execute an arbitrary command on the code chunk
content, optionally with arguments. If any, they are passed to `engine.opts` and
they could be one of the following:

The `exec` engine allows to execute an arbitrary command on the code chunk
content, optionally with arguments.

In your Rmd file

---
layout: false
# Working with interoperability

.subtitle[More engines powered by other 📦]

* Use the .color1[`sql`]  engine to run queries using **DBI** on compatible 
.icon-link[databases [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://bookdown.org/yihui/rmarkdown/language-engines.html#sql)]
* Use the .color1[`python`]  engine with **reticulate** to work seemlessly with R and Python chunks 
.icon-link[together [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://rstudio.github.io/reticulate/articles/r_markdown.html)]
* Use the .color1[`scss`]  or .color1[`sass`]  engine to process a chunk content with **sass** package to insert a CSS in 
.icon-link[HTML [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://rstudio.github.io/sass/articles/sass.html#rmarkdown)]
* Use the .color1[`bslib`]  engine to add rules to **bslib** themes withing 
.icon-link[Rmd [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://rstudio.github.io/bslib/articles/bslib.html#advanced)]

???

Some R packages are great to work with other tools like **DBI** for SQL queries, **reticulate** for working with Python from R, **sass** to work with SASS or **bslib** to customize Bootstrap website. **knitr** engines can be integrated with other Packages to bring more feature from within a Rmd document.

---
layout: false

# Extending {knitr} with custom engines

.subtitle[Any package can provide custom way to process chunk content]
 
* **glue** has a .color1[`glue`]  engine to process chunk content as if passed to `glue::glue()` 
.icon-link[function [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://glue.tidyverse.org/articles/engines.html)]
* **texPreview** has a .color1[`texpreview`] engine to render TeX snippet from code chunk, in non-LaTeX output 
.icon-link[document [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://yonicd.github.io/texPreview/articles/engine.html)]
* **targets** offers a .color1[`targets`] engine so that literate programming can be used to create a **targets** workflow 
.icon-link[(Target Markdown) [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://books.ropensci.org/targets/markdown.html#target-markdown)]
* **d3** has a .color1[`d3`] engine where chunk content will be processed with 
.icon-link[**r2d3** [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M432,320H400a16,16,0,0,0-16,16V448H64V128H208a16,16,0,0,0,16-16V80a16,16,0,0,0-16-16H48A48,48,0,0,0,0,112V464a48,48,0,0,0,48,48H400a48,48,0,0,0,48-48V336A16,16,0,0,0,432,320ZM488,0h-128c-21.37,0-32.05,25.91-17,41l35.73,35.73L135,320.37a24,24,0,0,0,0,34L157.67,377a24,24,0,0,0,34,0L435.28,133.32,471,169c15,15,41,4.5,41-17V24A24,24,0,0,0,488,0Z"/></svg>](https://bookdown.org/yihui/rmarkdown-cookbook/d3.html#d3)]

???

https://yonicd.github.io/texPreview/

https://books.ropensci.org/targets/markdown.html

---
layout: true

# Extending {knitr} with custom engines

.subtitle[How to create a new engine ?]

---

```r
knitr::knit_engines$set(foo = function(options) {
  # the source code is in options$code; just do
  # whatever you want with it
})
```

* Use `knit_engines$set()` to register by name
* All knitr options are passed to the engine
* Code chunk content is in `options$code`

---

.fl.w-100.pa2[
This engine will take the chunk content and make it upper case.
]

.fl.w-60.pa2[

.v-mid[

```r
knitr::knit_engines$set(upper = function(options) {
  code <- paste(options$code, collapse = "\n")
  # Allow to hide result
  if (options$results == 'hide') return()
  # Allow to prevent processing
  if (options$eval) {}
    toupper(code) 
  } else {
    code
  }
})
```
]
]

.fl.w-40.pa2[

````markdown
```{upper}
Hello, **knitr** engines!
```
````

.small.center[<svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M413.1 222.5l22.2 22.2c9.4 9.4 9.4 24.6 0 33.9L241 473c-9.4 9.4-24.6 9.4-33.9 0L12.7 278.6c-9.4-9.4-9.4-24.6 0-33.9l22.2-22.2c9.5-9.5 25-9.3 34.3.4L184 343.4V56c0-13.3 10.7-24 24-24h32c13.3 0 24 10.7 24 24v287.4l114.8-120.5c9.3-9.8 24.8-10 34.3-.4z"/></svg>]

```markdown
HELLO, **KNITR** ENGINES!
```
]

---

.fl.w-100.pa2[A custom `py` engine to run `python`]

.fl.w-60.pa2[

```r
knitr::knit_engines$set(py = function(options) {
  code <- paste(options$code, collapse = '\n')
  out  <- system2(
    'python', c('-c', shQuote(code)), stdout = TRUE
  )
  knitr::engine_output(options, code, out)
})
```
]

.fl.w-40.pa2[

````markdown
```{py}
import os
print(os.environ.get('USERNAME'))
```
````

````markdown
```py
import os
print(os.environ.get('USERNAME'))
```

```
## chris
```
````
]

???

https://bookdown.org/yihui/rmarkdown-cookbook/custom-engine.html

`knit_engines` is the way to register an engine by name for **knitr** to know it is available. Full set of `options` is passed and `options$code` contains the chunk content. Do what you need with that to process it and outputting a what will be sewed in the document.

---
layout: false
class: center, middle, annexe
count: false

# Thank you !

.f3[https://cderv.rbind.io/slides/user2022-knitr-engines]

.f3[https://github.com/cderv/user2022-knitr-engines]

.pull-left[
.center[
<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg></br>
[@cderv](https://github.com/cderv)
]
]

.pull-right[
.center[
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg></br>
[@chrisderv](https://twitter.com/chrisderv)
]
]