Recent revelations about PRISM, the NSA’s massive program of surveillance of civilian communications have caused quite a stir. And rightfully so, as it appears that the agency has been granted warrantless direct access to just about any form of digital communication engaged in by American citizens, and that their ... [Read more...]

Read Part 1 When making a statement of the form “1/2 is the correct probability that this coin will land tails”, there are a few things which are left unsaid, but which are typically implied. The statement is one about the probability of an unknown event occurring, and it would seem reasonable ... [Read more...]

I am currently working on a validation metric for binary prediction models. That is, models which make predictions about outcomes that can take on either of two possible states (eg Dead/not dead, heads/tails, cat in picture/no cat in picture, etc.) The most commonly used metric for this ... [Read more...]

Reblogged from Zero to R Hero: Canadian Aquatic Invasive Species Networks Annual General Meeting in Kananaskis, Alberta. May 03, 3:25-5:30. This 2-hour workshop will focus on how and why we do numerical simulation in R. Time permitting, we will also look at how to build and fit likelihood based statistical models. ... [Read more...]

I’ve been showing my new favourite toys to just about anyone foolish enough to actually engage me in conversation. I described how my shiny new set of non-transitive dice work here, complete with a map showing all the relevant probabilities. All was neat and tidy and wonderful until fellow ... [Read more...]

A very special package that I am rather excited about arrived in the mail recently. The package contained a set of 6-sided dice. These dice, however, don’t have the standard numbers one to six on their faces. Instead, they have assorted numbers between zero and nine. Here’s the ... [Read more...]

UPDATE: The day was great! There are many people doing really amazing things with open data and it was amazing to meet them. Here are my slides from the panel talk. Next Saturday, I’ll be sitting on a panel discussing future avenues for open data at ODX13. From the ... [Read more...]

We had a great turnout yesterday for our Zero to R Hero workshop at the Quebec Centre for Biodiversity Science. We went from the absolute basics of the command line, to the intricacies of importing data, and finally we had a look at plotting using ggplot2. We didn’t have ... [Read more...]

This puzzle came up in the New York Times Number Play blog. It goes like this: An entrepreneur has devised a gambling machine that chooses two independent random variables x and y that are uniformly and independently distributed between 0 and 100. He plans to tell any customer the value of x ... [Read more...]

This Thursday (Jan 24th), 5:30pm, the good folks at Wajam are hosting a meetup of the Montreal R User Group. The event will be at Bolidea at 4115 St Laurent, Montréal, QC. Be sure to RSVP. From Benjamin Rollert: This is an opportunity for people interested in R to hang ... [Read more...]

Well, that’s embarrassing. A little tweak to my dark matter model resulted in a leaderboard score in the top 10. The only problem is that the contest closed about an hour ago. I ran this final prediction earlier today but then simply forgot to go back to it and submit!! ... [Read more...]

In the search for dark matter, I have been having mixed success. It seems that locating DM in single halo skies is a fairly straightforward problem. However, when there are more than one halo, things get quite a bit trickier. As I have advocated many times before, including here and ... [Read more...]

At a meeting last night with some collaborators at the Vélobstacles project, I was excitedly told about the magic of IPython and it’s notebook functionality for reproducible research. This sounds familiar, I thought to myself. Using a literate programming approach to integrate computation with the communication of methodology ... [Read more...]

Last night we had a great meet up of the Montreal R User Group. I got things started with a little presentation asking the question “What does R do?” (slides). I made the presentation using Montreal R User Group member Ramnath Vaidyanathan‘s Slidify package. Slidify allows you to generate ... [Read more...]

In today’s XKCD, a pair of (presumably) physicists are told by their neutrino detector that the sun has gone nova. Problem is, the machine rolls two dice and if they both come up six it lies, otherwise it tells the truth. The Frequentist reasons that the probability of obtaining ... [Read more...]

I am working on a project that requires the generation of Bernoulli outcomes. Typically, I would go about this using the built in sample() function like so: This works great and is fast, even for large n. Problem is, I want to generate each sample with its own unique probability. ... [Read more...]

After a bit of a summer lull, the Montreal R User Group is meeting up again! We’re trying out a new venue this time. Notman House is the home of the web in Montreal. They hold hackathons and other tech user group meetups, and they are all around great ... [Read more...]

I recently posted the slides from a guest lecture that I gave on Bayesian methods for biologists/ecologist. In an effort to promote active learning, the class was not a straight forward lecture, but rather a combination of informational input from me and opportunities for students to engage with the ... [Read more...]

This is a talk I gave this week in Advanced Biostatistics at McGill. The goal was to provide an gentle introduction to Bayesian methodology and to demonstrate how it is used for inference and prediction. There is a link to an accompanying R script in the slides [Read more...]

The three benchmark algorithms for predicting the location of dark matter halos are, for the most part, all over the map. Most of the test skies look something like this: There are, however, some skies with rather strong halo signals that get a decent amount of agreement: The Lenstool MLE ... [Read more...]