Twitter is a favorite source of text data for analysis: it’s popular (there is a huge volume of variety on all topics) and easily accessible using Twitter’s free, open APIs which are easily consumable in JSON and ATOM formats. Some … Continue reading → [Read more...]

The following fully-automatic Bash script downloads, compiles, and initializes compcache version 0.6.2 on Ubuntu Karmic Koala (9.10). This script creates two swaps with a maximum of 4GB uncompressed size each. Two swaps are used to take advantage of 2 CPUs (or CPU cores in a multicore CPU). Compcache is a fascinating memory compression ... [Read more...]

Major credit card issuing networks (including Visa, MasterCard, Discover, and American Express) allow simple credit card number validation using the Luhn Algorithm (also called the “modulus 10″ or “mod 10″ algorithm). The following code demonstrates an implementation in SAS. The code also validates the credit card number by length and by checking ... [Read more...]

Conditional inference trees (ctree) in package party allows weighting which is useful when one classification outcome is more important than another. Useful examples are not difficult to imagine: in a marketing direct mailing, a false positive (non-res... [Read more...]

In our department and various places on the Intertubes, SAS programmers set the HTML title tag (which sets the title in web browsers and on search engines) in ODS using the headtext option: ods html headtext="My great report" /* wrong! */ file="foo.html"; This may work in some situations, but ... [Read more...]

Do you need a way to find out which individual variables in R consume the most memory? # create dummy variables for demonstration x [Read more...]

After installing Weka/RWeka in R, you may get this error if you try to load RWeka in the same session: require(RWeka) Cannot create Java virtual machine (-4) Error : .onLoad failed in 'loadNamespace' for 'RWeka' Solution: Just close R and re-open it. Cause: Apparently the installation requires some initialization. ... [Read more...]

This tutorial demonstrates to the R novice how to create five machine learning models for classification and compare the performance graphically with ROC curves in one chart. For a simpler introduction, start with Plot ROC curve and lift chart in R. # ... [Read more...]

This tutorial with real R code demonstrates how to create a predictive model using cforest (Breiman’s random forests) from the package party, evaluate the predictive model on a separate set of data, and then plot the performance using ROC curves ... [Read more...]

Recently Microsoft Office Outlook 2007 started giving me the vague error message Outlook cannot open this item. The item may be damaged. The message would appear randomly throughout the day. Sometimes five error message boxes would be stacked up on top of each other. OK, but which item? What kind of ... [Read more...]

Deleting rows from a data frame in R is easy by combining simple operations. Let’s say you are working with the built-in data set airquality and need to remove rows where the ozona is NA (also called null, blank or missing). The method is a conce... [Read more...]

Coming from a background writing SQL code directly for “real” RDBMS (Microsoft SQL Server, MySQL, and SQLite), I was initially confused when SAS would give me the following ‘note’ for a simple summary PROC SQL query: 429 proc sql; 430 create table undel_monthly as 431 select 432 year(date) as year, 433 month(date) ... [Read more...]