Counting Kaggle Followers through Meta Kaggle

Follow the white rabbit

The wondrous land of Kaggle community connections is chiefly contained in the file UserFollowers.csv. Let’s download and read it:

follow <- vroom("../../static/files/UserFollowers.csv", col_types = cols())

Here are the first few rows of that table:

follow |> 
  head(10) |> 
  gt() |> 
  opt_row_striping()

It is constructed from the point of view of the individual user, and their UserId and CreationDate (i.e join date), which has decided to follow other Kagglers with a certain FollowingUserId. So if we want to aggregate it from the point of view of the Kagglers that are being followed, we count the FollowingUserId instances.

The goal is to have a table of the top 500 followed Kagglers. But we also want to join this table later to another table that contains more user info; but that doesn’t have some Kagglers that no longer have an account on the platform (for whichever reason). So we’re being a bit more generous and start with the top 700 here. The first 5 rows of that table look as follows:

top500 <- follow |> 
  count(FollowingUserId, sort = TRUE) |> 
  slice_head(n = 700)

top500 |> 
  head(5) |> 
  gt()

The most popular Kagglers have more than 10,000 followers. That’s very impressive! But who exactly are they? This table only contains their user ID. If you’re on Kaggle, then you might have an idea of who might be hiding behind these numbers. But we’d like to know for sure. This is where another Meta Kaggle table comes to the rescue.

Join the fun

What we need is the Users.csv, which contains the details for each of the (more than 10 million!) Kagglers. (This is a large file, and I recommend downloading the most recent version). Let’s read that in as well:

users <- vroom("./Users.csv", col_types = cols())

The user ID here is encoded as Id, and we also have the UserName (your unique user name), DisplayName (how your name is shown on your Kaggle profile, can be changed), and the date of joining Kaggle (RegisterDate) together with the current PerformanceTier. The latter goes from 0 to 4 for Novice, Contributor, Expert, Master, and Grandmaster; with tier 5 indicating a member of the Kaggle team.

Here are some sample rows, which already contain a number of familiar names:

users |> 
  head(50) |> 
  gt() |> 
  opt_row_striping() |> 
  tab_options(container.height = px(400))

Now we’re joining this table to our followers ranking on "FollowingUserId" = "Id", and take the top 500 users that are present in both tables:

top500 <- top500 |> 
  left_join(users, by = c("FollowingUserId" = "Id")) |> 
    filter(!is.na((UserName))) |>
    slice_head(n = 500) |> 
    rownames_to_column("rank")

This is the result, the full table of the top 500 followed Kaggle users, ranked by number of followers:

top500 |> 
  select(rank, UserName, DisplayName, RegisterDate, n) |> 
  gt() |> 
  opt_row_striping() |> 
  tab_options(container.height = px(400))

Big names in the top 10, and among them yours truly at rank 7. Which is mind-blowing, to be honest. Among the ten most followed Kagglers in the world. Wow. I hope that I can give interesting ideas and inspiration back to the community.

Seeing Abhishek Thakur in 1st place is no surprise. He was the first 3 x Grandmaster and 4 x Grandmaster, and constantly shares great knowledge on Kaggle and through his popular Youtube channel. SRK, another 4 x GM, was already a Kaggle legend when I joined the platform. And bestfitting has just reclaimed the title of Competitions #1, which they had previously held for several years.

Feel free to scroll through the table and you will recognise many names from Kaggle’s past or present.

Get ready for networking

To wrap up this post, we will prepare a compact dataset which we can then feed into the network analysis tools that I want to practice in the upcoming post. We could do that already based on the UserFollowers.csv file only. But I’d like to work with the user names instead.

In this notation, user_1 is being followed by user_2. We’re also keeping track of the CreationDate, which is the date at which the follow began. And of the Kaggle tiers (e.g. Expert, Master) of both users. The new dataset only contains relationships between the top 500 followed Kagglers.

follow_network <- follow |> 
  left_join(top500 |> select(user_1 = UserName,
                             tier_1 = PerformanceTier,
                             followers_1 = n,
                             FollowingUserId), by = "FollowingUserId") |> 
  left_join(top500 |> select(user_2 = UserName,
                             tier_2 = PerformanceTier,
                             followers_2 = n,
                             UserId = FollowingUserId), by = "UserId") |> 
  filter(!is.na(user_1) & !is.na(user_2)) |> 
  mutate(date = mdy(CreationDate)) |> 
  select(user_1, user_2, tier_1, tier_2, followers_1, followers_2, date)

Here are the first few rows to illustrate what the file looks like. Those rows show the Kaggle founder and now former CEO Anthony Goldblum following his Co-Founder Ben Hamner and a few other people:

follow_network |> 
  head(5) |> 
  gt() |> 
  opt_row_striping()

This is all for the scope of this post. See the next posts for a continuation of this analysis and a deeper dive into Kaggle community dynamics. Have fun!