In a previous post, I outlined how to load daily Adobe Analytics Clickstream data feeds into a PostgreSQL database. While this isn’t a long-term scalable solution for large e-commerce companies doing millions of page views per day, for exploratory analysis a relational database structure can work well until a ... [Read more...]

As an author of several open-source software projects, I’ve taken for granted that people using the software share the same community values as I do. Open-source authors provide their code “free” to the community so that others may benefit without having to re-invent the wheel. The only expectation (but ... [Read more...]

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself. As part of getting RSiteCatalyst 1.4.8 ready for CRAN, I’ve managed to accumulate hundreds of testthat tests across 63 test files. Each of these tests runs on Travis ... [Read more...]

For being in RSiteCatalyst retirement, I’m ending up working on more functionality lately ¯_(ツ)_/¯. Here are the changes for RSiteCatalyst 1.4.8, which should be available on CRAN shortly: Segment Stacking RSiteCatalyst now has the ability to take multiple values in the segment.id keyword for the Queue* functions. This functionality was ... [Read more...]

In my previous post about the Adobe Analytics Clickstream Data Feed, I showed how it was possible to take a single day worth of data and build a dataframe in R. However, most likely your analysis will require using multiple days/weeks/months of data, and given the size and ... [Read more...]

@randyzwitch Do you know if anyone has gotten RSiteCat running in a Jupyter Notebook that ran RPY2? Tired of using 2 different environments — Adam Gitzes (@FootballActuary) February 18, 2016 This will be a very short post, because the only “new” information I’m going to provide is the minimal example to answer the ... [Read more...]

It seems as though I missed release notes for version RSiteCatalyst 1.4.6, so we’ll do those and RSiteCatalyst 1.4.7 (now on CRAN) and the same time… RSiteCatalyst 1.4.6 This release was mostly tweaking some settings, specifically: Adding a second `top` argument within the Queue* functions for more control on results returned. It ... [Read more...]

More often that I would like, I receive datasets where the data has only been partially cleaned, such as the picture on the right: hundreds, thousands…even millions of tiny files. Usually when this happens, the data all have the same format (such as having being generated by sensors or ... [Read more...]

It’s only been a month since the last RSiteCatalyst update, and this update is also a pretty minor update in terms of functionality. Set Your Own Endpoint For the overseas users (or companies with weird setups), you can now use the endpoint argument in the SCAuth() function to specify ... [Read more...]

As I’ve become more serious about contributing in the open-source community, having quality tests for my packages has been something I’ve spent much more time on than when I was just writing quick-and-dirty code for my own purposes. My most used open-sourced package is RSiteCatalyst, which accesses the ... [Read more...]

“Well, first you need a TMS and a three-tiered data layer, then some jQuery with a node backend to inject customer data into the page asynchronously if you want to avoid cookie-based limitations with cross-domain tracking and be Internet Explorer 4 compatible…” Blah Blah Blah. There’s a whole cottage industry ... [Read more...]

It’s been about six months since the last RSiteCatalyst update, and this update is really just a single bug fix, but a big bug fix at that! Sparse Data = Opaque Error Messages Numerous people have reported receiving an error message from RSiteCatalyst similar to the following: 'names' attribute [1] must ... [Read more...]

          Mmmmm, chartjunk! Rebooting Vega.jl Recently, I’ve found myself without a project to hack on, and I’ve always been interested in learning more about browser-based visualization. So I decided to revive the work that John Myles White had done in building Vega.jl nearly two years ago. And ... [Read more...]

Thanks to user dnlbrky, we now have a third way to accomplish sessionizing log data for any arbitrary time out period (see methods 1 and 2), this time using data.table from R along with magrittr for piping: I agree with dnlbrky in that this feels a little better than the dplyr ... [Read more...]

Last week, I wrote a blog post showing how to sessionize log data using standard SQL. The main idea of that post is that if your analytics platform supports window functions (like Postgres and Hive do), you can make quick work out of sessionizing logs. Here’s the winning query:... [Read more...]

It’s a new year, so…new version of RSiteCatalyst on CRAN! For the most part, this release fixes a handful of bugs that weren’t noticed with the prior release 1.4.2 (oops!), but there are pieces of additional functionality. New functionality: Data Feed monitoring For those of you having hourly ... [Read more...]

Admission: I didn’t really know how computers worked until around 2012. For the majority of my career, I’ve worked for large companies with centralized IT functions. Like many statisticians, I fell into a comfortable position of learning SAS in a Windows environment, had Ops people to fix any Unix ... [Read more...]

RSiteCatalyst version 1.4.2 is now available on CRAN. This update was primarily bug fixes with one additional feature added. Fixed QueueRanked function to allow multiple SAINT classifications to be specified. This allows for breaking down a SAINT classification with another SAINT classification, such as breaking down tracking codes by marketing channel ... [Read more...]

Changes Version 1.4.1 of RSiteCatalyst is now available on CRAN. There were a handful of bug fixes and new features added, including: Fixed bug in QueueRanked function where only 10 results were returned when requesting multiple element reports. Function now returns up to 50,000 per breakdown (API limit) Created better error message to ... [Read more...]

A couple of weeks ago, Twitter open-sourced their BreakoutDetection package for R, a package designed to determine shifts in time-series data. The Twitter announcement does a great job of explaining the main technique for detection (E-Divisive with Medians), so I won’t rehash that material here. Rather, I wanted to ... [Read more...]