A famous finance professor once told us that good diversification meant holding everything in the world. Fine, but in what proportion? Suppose you could invest in every country in the world. How much would you invest in each? In a market-capitalization weighted index, you'd invest in each country in proportion ... [Read more...]

I studied psychology in Marburg (Germany). At university we exclusively used SPSS. The courses were among my classmates usually not the most popular. After over 10 years in management consulting, I have noticed that little has changed to this. It’s a shame because a lot of interesting data in consulting ... [Read more...]

It is not uncommon to have two computers at work, four at home and a server out on the wild, wild internet (that's what we have, anyway ... wait, we forgot one in London). How to keep all these files in sync? Here are our file synchronization tips. [Read more...]

Best graph ever. LARGEST EVER DIFFERENCE BETWEEN 328 and 327 SPOTTED IN NEW YORK CITY [Read more...]

In creating the numbers tool I wanted to be able to do two things, 1) obtain information about what source did by matching the numeric literals it contained against a database of ‘interesting’ values (now with over 14,000 entries) and 2) flag possible incorrect numeric literals (e.g., 3.1459265 when 3.14159265 had been intended in ... [Read more...]

Yesterday Alasdair Allan and Pete Warden shocked the world by revealing that iPhones and iPads have been keeping track of our every move, and saving the data in obfuscated back up files. As my friend Vince Buffalo mentioned on Twitter, part of me was disgusted by the secret stalking Steve ... [Read more...]

Like many people who crunch numbers frequently, I have increasingly been integrating Amazon’s cloud computing services into my daily workflow. In particular, I have been using their elastic cloud computing (EC2) on a regular basis. The service is an excellent way to offload computationally intensive work from your laptop ... [Read more...]

Recently, the good folks at Infochimps.com rolled out a series of new APIs to add to their already impressive set of data resources. I have been in a perpetual state of catch-up since the new year, so I have only now got around to adding some of these new ... [Read more...]

The housing bubble by city. Miami sailed high and fell far. Detroit rose modestly and but dropped more than it went up. Dallas held steady. DC is enjoying a bit of renewed growth, but are in and New York yet to fall? [Read more...]

RESULTS OF THE GREAT AREA PLOT QUIZ If you are the type of reader who remembers things from last week, you may remember the great area plot quiz we had running. This week, we are excited to announce that the results are in. The plot above shows answers to the ... [Read more...]

Tomorrow I am headed out of town for a few weeks, so posting will be lighter than the usual lightness. My first stop is Chicago, and tomorrow night I will be speaking about my R package for the infochimps API at the Chicago R Users Group. If you are in ... [Read more...]

HOW TO DISPLAY A LINE PLOT WITH COUNT INFORMATION? In a previously-mentioned paper Sharad and your DSN editor are writing up, there is the above line plot with points. The area of each point shows the count of observations. It’s done in R with ggplot2 (hooray for Hadley). We ... [Read more...]

I have finally taken the plunge and created my first R package! As frequent readers will know, I often sing the praises of infochimps, a startup out of Austin, TX attempting to be the world’s data clearinghouse. While infochimps is an excellent resource for data sets, they also provide ... [Read more...]

HOMOPHILY + MAPS WITHOUT MAPPING SOFTWARE In the past, Decision Science News has posted about homophily (“birds of a feather shop together“) and cool, lightweight visualizations (“maps without map packages in R“). Today, both topics come together in Eric Fischer’s fascinating set of images on Flickr called “Race and Ethnicity”(*).  ... [Read more...]

PREDICTING CONSUMER BEHAVIOR FROM SOCIAL NETWORKS This week, Decision Science News is doing a special cross-posting with Messy Matters. The post below is by Sharad Goel and describes work that he and your Decision Science News editor Dan Goldstein are jointly undertaking at Yahoo! Do you know what the #$*! your ... [Read more...]

CHART CRITICS, GRAPHICS CURMUDGEONS, COME ONE COME ALL Once upon a time there was this graph (graph 1). Andrew Gelman went all graphics curmudgeon on it, calling it an “ugly, sloppy bit of data graphics“, so it became this graph (graph 2). Now the question is, which is better: graph 2 or graph 3? ... [Read more...]

[Read more...]

Today, I am packing up the car and heading south to my old home, Washington, DC, for the useR! 2010 conference, which is being held at the National Institute of Standards and Technology. Incidentally, where I was an intern in the Information Technology Lab during college. If you are not able ... [Read more...]

COUNTERINTUITIVE PROBLEM, INTUITIVE REPRESENTATION Blog posts about counterintuitive probability problems generate lots of opinions with a high probability. Andrew Gelman and readers have been having a lot of fun with the following probability problem: I have two children. One is a boy born on a Tuesday. What is the probability ... [Read more...]

Having worked with Unix (BSD, HPUX, IRIX, Linux and OSX), Windows (NT4, 2000, XP, Vista and 7) for quite a while I have seen a lot of different software tools. I would like to quickly exhibit my “must have” list. These are the packages that I find to be the single “must ... [Read more...]