Get a YouTube API key

As is often the case with anything to do with Google, first, you must get credentials. In this example, we were interested in an API key that would work with the YouTube Data API v3.

1. Log into the Google Cloud Platform Console.
2. Create a new project.
3. From your dashboard, click to ‘Go to APIs Overview’ and navigate to ‘Credentials’ in the sidebar.
4. At the top, click ‘Create Credentials’ and choose ‘API key’.
5. Your API key will pop up. Save it for reference later in this walkthrough.
6. In the pop-up window (or in the hamburger menu for the API key), click ‘Edit API key’.
7. Scroll to the bottom and under ‘API restrictions’, select ‘Restrict key’, select ‘YouTube Data API v3’ from the dropdown menu, click ‘OK’ to exit the dropdown menu, then click ‘Save’. This will restrict your API key so that it can only be used with the YouTube Data API v3, which will limit its usefulness if anyone gets ahold of your key.

Store your YouTube API key

Now that you have an API key, you will want to save it in your R environment (.Renviron) using usethis::edit_r_environ() if working interactively or in your repository secrets if using GitHub Actions. Storing the API key as an environment variable allows you to access it easily without the need to manually input it every time you run your code. GitHub Actions can interact with the API without manual intervention, and storing it as a repository secret keeps the key separate from your source code so it is not exposed to others.

To do this, follow the instructions below:

1. Install the CRAN version of {httr2} and the GitHub version of {tuber}. {pak} allows for easy installation of packages, whether they are on CRAN, GitHub, pull requests, etc.

# install.packages("pak")      # Install {pak} if not yet installed
pak::pak(httr2, soodoku/tuber) # CRAN and GitHub syntax side by side!

2. On your local machine, save your YouTube API key in .Renviron. Running the code below will prompt you to save the API key. Paste your API key into the pop-up window:

tuber::yt_set_key()

Now your YouTube API key is saved as YOUTUBE_KEY in .Renviron!

Pull and clean data

Woohoo, you are credentialed! It’s time for the fun part — pulling and cleaning data. I did this in a script called data_processing.R.

In addition to {tuber}, here are other packages that will aid you in the cleaning process:

library(tuber) # GitHub version, installed with pak::pak(soodoku/tuber)
library(readr)
library(dplyr)
library(stringr)
library(DT)

Create a spreadsheet with the channels you are interested in. I created this manually — I just searched on YouTube for all the R-Ladies channels I could find. To make this workflow as programmatic as possible, I only included the chapter name, thumbnail, and channel ID.

dat <-
  read_csv("https://github.com/ivelasq/rladies-video-feed/blob/main/r-ladies_channels.csv")
  
head(dat)
  chapter            image                                              id   
  <chr>              <chr>                                              <chr>
1 R-Ladies Global    https://yt3.ggpht.com/ytc/AKedOLRM4FPiPcPBdFUmYWR… UCDg…
2 R-Ladies Baltimore https://yt3.ggpht.com/RQifC3qp_7AFeTc48-QV1i4WBNM… UC9U…
3 R-Ladies Philly    https://yt3.ggpht.com/ytc/AKedOLShIKBvPUKqbvm_Cpu… UCPq…
4 R-Ladies STL       https://yt3.ggpht.com/ytc/AKedOLSBA7rlz1yvbIJ7TVN… UCQ7…
5 R-Ladies Sydney    https://yt3.ggpht.com/ytc/AKedOLQnaU-dJbb14j2RE6W… UCkr…
6 R-Ladies Vancouver https://yt3.ggpht.com/3yf0Zo8-VKffrG-dRT_Gs85xX_x… UCX5…

Now, create a few more columns with dplyr::mutate() that expand the URLs into HTML format:

dat_urls <-
  dat %>%
  dplyr::mutate(
    feed = paste0("https://www.youtube.com/feeds/videos.xml?channel_id=", id),
    feed_url = paste0("yt:channel:", id),
    channel_image_url = paste0(
      "<img src='",
      image,
      "' alt='Hex Sticker for Chapter' width='40'></img>",
      " <a href='https://www.youtube.com/channel/",
      id,
      "' target='_blank'>",
      chapter,
      "</a><br>"
    ),
  )

The {tuber} documentation describes the many functions available to you. Many of these require OAuth 2.0 authentication. Since I had only the channel IDs, I wanted to use list_channel_videos() to get a complete list of videos from the channels, which thankfully works with an API key!

For example, for the R-Ladies Global channel, you could run list_channel_videos() to get its (currently 165) videos:

tuber::list_channel_videos("UCDgj5-mFohWZ5irWSFMFcng", max_results = 200, auth = "key")

What if you want the results for all the channels we have in our spreadsheet? I used a loop for this.

1dat_videos <- NULL

2for (i in 1:nrow(dat_urls)) {
  tmp <-
3    dat_urls[i, ]["id"] %>%
4    dplyr::pull() %>%
5    tuber::list_channel_videos(
      .,
      part = "snippet",
      config = list('maxResults' = 200),
      auth = "key"
    )
6  dat_videos <- dplyr::bind_rows(dat_videos, tmp)
}

1

Starting with an empty data frame called dat_videos,

2

For each row in dat_urls (which contains the channel IDs),

3

Find the channel ID,

4

Pull the expanded URL,

5

Run it through list_channel_videos(), and

6

Add the information as rows to dat_videos.

The arguments in list_channel_videos() gave me all the columns I am interested in with part = "snippet". Notice that there’s a default limit for the number of videos pulled from the API. I bumped that up a bit with the maxResults argument.

Then, I brought back the URL info:

dat_join <-
  dat_videos %>%
  dplyr::left_join(., dat_urls, by = join_by("snippet.channelId" == "id"))

This results in a lot of information for each video. I cleaned it up a bit so that the data frame contained only the relevant columns in HTML format.

dat_dashboard <-
  dat_join %>%
  dplyr::mutate(
    video_url = paste0(
      "<a href='https://www.youtube.com/watch?v=",
      snippet.resourceId.videoId,
      "' target='_blank'>",
      snippet.title,
      "</a>"
    ),
    channel_url = paste0(
      "<img src='",
      image,
      "' alt='Hex Sticker for Chapter' width='40'></img>",
      "<a href='https://www.youtube.com/channel/",
      snippet.channelId,
      "' target='_blank'>",
      chapter,
      "</a>"
    ),
    date = as.Date(str_sub(snippet.publishedAt, 1, 10))
  ) %>%
  dplyr::arrange(desc(snippet.publishedAt)) %>%
  dplyr::select(date, chapter, channel_url, video_url, channel_image_url)

See the final data processing file on GitHub.

Create a flexdashboard

You have the YouTube data — time to create a pretty dashboard!

My flexdashboard started as R Markdown files often do: with a YAML header. I specified an orientation (columns) and added the link to the GitHub repository in the navigation bar.

---
title: "R-Ladies YouTube Video Feed"
output: 
  flexdashboard::flex_dashboard:
    orientation: columns
    navbar:
      - { icon: "fa-github", href: "https://github.com/ivelasq/rladies-video-feed", align: right }
---

---
title: "R-Ladies YouTube Video Feed"
output: 
  flexdashboard::flex_dashboard:
    orientation: columns
    navbar:
      - { icon: "fa-github", href: "https://github.com/ivelasq/rladies-video-feed", align: right }
    theme:
      version: 4
      bg: "#FDF7F7"
      fg: "#88398A" # purple
      primary: "#88398A" # purple
      base_font:
        google: "Lato"
---

library(flexdashboard)
library(bslib)
source("data-processing.R", local = knitr::knit_global())

Design the flexdashboard layout

This dashboard has a simple layout: just a sidebar and the main section. I recommend checking out the flexdashboard documentation to see all the layout options available to you.

This code builds out the sidebar section. I created a list of each R-Ladies chapter that I was able to find and arranged them by name. With htmltools::HTML(), the dashboard can render the URLs as HTML (the reason for all that manipulation earlier on). We learned that the .noWS = "outside" argument is crucial for deploying the dashboard with GitHub Actions. It omits extra whitespace around the HTML, which ensures that the dashboard is committed only when the number of videos has changed (rather than creating spurious commits each time data is pulled, even if the number of videos is unchanged).

Channels {.sidebar}
-----------------------------------------------------------------------

The purpose of this dashboard is to provide a running feed of R-Ladies videos posted to YouTube. It is refreshed every 24 hours.

Currently, the feed includes these channels:

```{r}
dat_join %>% 
  dplyr::arrange(chapter) %>% 
  dplyr::distinct(channel_image_url) %>% 
  dplyr::pull() %>% 
  htmltools::HTML(.noWS = "outside")
```

For the main body, I used {DT} to create a table for the information in our clean dataset.

Column {data-width=900}
-----------------------------------------------------------------------

### By default, the list is sorted by latest video.

<style>
.dataTables_scrollBody {
    max-height: 100% !important;
}
</style>

```{r}
dat_dashboard %>%
  dplyr::select(-chapter, -channel_image_url) %>%
  DT::datatable(
    colnames = c("Date", "Channel", "Video"),
    filter = "top",
    escape = FALSE, # <1>
    height = "1000", # <2>
    elementId = "dashboard",
    options = list(columnDefs = list( # <3>
      list(className = "dt-middle", targets = "_all")
    ))
  )
```

There are a lot of customization options! A couple of fun tricks to notice:

1. escape = FALSE renders the URLs within the table as HTML.
2. height = "1000" makes it expand to the entire column height.
3. The list within options aligns the text within columns (in this case, to be in the middle of the cell for all columns).¹

And that’s it! See the final dashboard code on GitHub.

Try it out — search “Shiny” to see any video with Shiny in the title, or “Ecuador” to see all the videos from R-Ladies Ecuador!

Create a GitHub Action workflow

Great, we have a completed dashboard! Now, what if we want to keep it updated without manually re-knitting the dashboard? The next step is to create a GitHub Action. GitHub Actions are workflows that you create and configure to automate various tasks within your GitHub repositories. They are event-driven and can be triggered by actions such as code commits, pull requests, issue updates, or scheduled intervals.

on:
  schedule:
1    - cron: '0 0 * * *'

name: rladies-videos-bot

jobs:
  rladies-videos-bot:
    runs-on: ubuntu-latest
    env:
      GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
      YOUTUBE_KEY: ${{ secrets.YOUTUBE_KEY }}
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: r-lib/actions/setup-pandoc@v2
      - uses: r-lib/actions/setup-r@v2
        with:
          use-public-rspm: true
2      - uses: r-lib/actions/setup-r-dependencies@v2
        with:
3          extra-packages: soodoku/tuber
      - name: Render and commit dashboard
        run: |
          Rscript -e 'rmarkdown::render("index.Rmd")'
          git config --local user.name "$GITHUB_ACTOR"
          git config --local user.email "[email protected]"
          git add index.html
          git commit -m '[Bot] Render and commit dashboard' || echo "No changes to commit"
          git push origin || echo "No changes to commit"

Imports: 
    bslib,
    dplyr,
    DT,
    flexdashboard,
    htmltools,
    httr2,
    jsonlite,
    readr,
    rmarkdown,
    stringr

Host the dashboard

To host the dashboard, I used GitHub Pages. In the GitHub repository for your dashboard, go to ‘Settings’, then ‘Pages’. Choose the branch and folder of your flexdashboard index.html output file, click ‘Save’, and then you will have a URL to showcase your work.

Here is the final link for this dashboard: https://ivelasq.github.io/rladies-video-feed.

Summary

In summary, we have explored the powerful combination of {flexdashboard}, GitHub Actions (via r-lib/actions), and {tuber} to create an automated YouTube dashboard.

• With {flexdashboard}’s customizability and R Markdown support, we can create a visually appealing, interactive dashboard.
• The {tuber} package’s integration with the YouTube Data API and new functions we contributed to permit YouTube API key authentication allow us to easily fetch data and keep track of YouTube videos.
• With the automation capabilities of GitHub Actions, we can schedule dashboard updates and stay up-to-date with new YouTube videos.

It took a while, but we got there! Many thanks to @gvelasq’s support in getting this through the finish line.

Give this tutorial a try and unlock the potential of automated dashboards for your favorite YouTube channels! I’d love to see what you create!