Beta Distribution and the NJ U.S. Senate Election

The beta distribution is highly flexible distribution and applies to many situations and environments. The beta distribution applies well when there are percentages. The upcoming New Jersey U.S. Senate election on Wednesday fits that criterion quite well. So here I applied the beta distribution to some pre-election polls where the numbers were obtained through the poll aggregator www.realclearpolitics.com.

The candidates for New Jersey election this Wednesday — to fill the vacant seat left by the death of Frank Lautenberg — are Cory Booker and Steve Lonegan. Though there are other third-party candidates running the race it is effectively between Booker and Lonegan. Though more complex models can be used reducing the candidates to two the beta distribution can be applied to these data and the outcomes and a simple simulation can be achieved using the given data.

Some Historical Notes

This general election is on a non-standard Election Day (Wednesday, October 16th). It happens to be the first time that a New Jersey general election has been held on a Wednesday. Aside from the current Republican senator who was appointed by Chris Christie the last time there was a Republican U.S. Senator in New Jersey was back in the early 1980′s and even then he too was appointed to the office.

The Beta Distribution

As can be seen from the elections since 1990 the democratic candidate has won by an average of about 8.9%.

2012 — Menendez: 58.9% v. Kyrillos: 39.4%
2008 — Lautenberg: 55.5% v. Zimmer: 42.5%
2006 — Menendez: 53.3% v. Kean Jr.: 44.3%
2002 — Lautenberg: 53.9% v. Forrester: 44.0%
2000 — Corzine: 50.1% v. Franks: 47.1%
1996 — Torricelli: 52.7% v. Zimmer: 42.6%
1994 — Lautenberg: 50.3% v. Haytaian: 47.0%
1990 — Bradley: 50.5% v. Whitman: 47.4%

Based on recent pre-election polling it looks like Booker will likely win by a similar margin and maybe a little higher than the average of 8.9% and, based on pre-election polls, closer to 12 percentage points.  The marginal difference between Booker and Lonegan is distributed as a beta distribution and we can see that the threshold of zero (0) is out in the far tail of the distribution.  So based on historical election and current pre-election polling it seems that the likelihood that Booker will win is very high.

Example Code

[sourcecode language=”css”]<br /> library(MCMCpack)<br /> ## Set up several of the recent polls but will only work with the most recent on<br /> raw.1 = NULL<br /> raw.1 = data.frame( rbind(<br /> Quinnipiac = c(.53,.41,899),<br /> RSC = c(.50,.39,729),<br /> FD= c(.45,.29,702),<br /> Mon = c(.53, .40,571)<br /> )<br /> )<br /> raw.1 = rbind(raw.1, c(apply(raw.1[,1:2],2,weighted.mean,raw.1[,3]),sum(raw.1[,3])))<br /> names(raw.1) = c(&#8220;Cand1&#8243;,&#8221;Cand2&#8243;,&#8221;size&#8221;)<br /> raw.1$Other.und = 1-raw.1$Cand1-raw.1$Cand2<br /> raw.1.no.und = data.frame(raw.1[5,1:2] + raw.1[5,1:2]/sum(raw.1[5,1:2])*raw.1[5,4],size=raw.1[5,3],Other.und=0)<br /> raw = rbind(raw.1, raw.1.no.und)<br /> ###################################################################<br /> ## More than two candidates so Beta distribution won&#8217;t work<br /> ## Function to randomly generate data from a dirichlet distribution<br /> ###################################################################<br /> j= 4<br /> prob.win = function(j,export=1){<br /> p=rdirichlet(100000,<br /> raw$size[j] *<br /> c(raw$Cand1[j], raw$Cand2[j], 1-raw$Cand1[j]-raw$Cand2[j])+1<br /> )<br /> if(export==1){<br /> mean(p[,1]&gt;p[,2])<br /> } else {<br /> return(p)<br /> }<br /> }</p> <p>( cand1.win.probs = sapply(1:nrow(raw),prob.win) )</p> <p>sim.dir = prob.win(4,export=2) ## set simulated data for plotting and determining parameters<br /> sim.dir.diff = sim.dir[,1]-sim.dir[,2] ## Get the marginal. From a Dirichlet the is distributed as a Beta.<br /> sim.dir = cbind(sim.dir, sim.dir[,1]-sim.dir[,2])<br /> ## The shape parameters (shape1 and shape2) might need some manual adjusting and tweaking.<br /> ## In this case I ran the function a few time to set the start value close to the output<br /> fit.distr.1 = fitdistr(sim.dir[,1], &#8220;beta&#8221;,<br /> start=list(shape1=302,shape2=270))<br /> fit.distr.2 = fitdistr(sim.dir[,2], &#8220;beta&#8221;,<br /> start=list(shape1=229,shape2=343))<br /> fit.distr.margin = fitdistr(sim.dir[,4], &#8220;beta&#8221;,<br /> start=list(shape1=5,shape2=5))<br /> ## Could also draw a histogram of simulated data<br /> curve(dbeta(x,fit.distr.1$estimate[1],fit.distr.1$estimate[2]),<br /> ylim=c(0,20), xlim=c(.3,.6), col=&#8217;blue&#8217;, lty=1, lwd=2, ylab=&#8221;Density&#8221;, xlab=&#8221;theta&#8221;,<br /> main=&#8221;Distribution of the NJ U.S. Senate Election 2013&#8243;,<br /> sub=paste(&#8220;Probability that Booker beats Lonegan: &#8220;, round(cand1.win.probs[6],2) ) ) ## Candidate 1<br /> curve(dbeta(x,fit.distr.2$estimate[1],fit.distr.2$estimate[2]), add=T, col=&#8217;red&#8217;, lty=2, lwd=2) ## Candidate 2</p> <p>abline(v=c(median(sim.dir[,1]), median(sim.dir[,2])), col=c(&#8216;blue&#8217;,’red&#8217;), lwd=2, lty=c(1,2,3))<br /> legend(&#8220;topleft&#8221;,c(&#8220;Booker&#8221;,&#8221;Lonegan&#8221;), lwd=2, col=c(&#8216;blue&#8217;,’red&#8217;), lty=c(1,2))<br /> ## Draw a histogram of simulated data<br /> hist(sim.dir[,4], nclass=100, main=&#8221;Histogram of the Candidate Differences&#8221;, xlab=&#8221;Candidate Difference&#8221;)<br /> abline(v=0, col=c(&#8216;black&#8217;), lwd=2, lty=c(1))<br /> [/sourcecode]