The next version of the htmltab package has just been released on CRAN and GitHub. The goal behind htmltab is to make the collection of structured information from HTML tables as easy and painless as possible (read about the package here and here). The most recent update got rid of many smaller bug fixes, inconsistencies and brings significant internal optimization of the code to increase not only the robustness of the function but also the level of verbosity in case something goes wrong. A complete list of the changes can be checked up here.

install.packages("htmltab")
#or
devtools::install_github("crubba/htmltab")

Header information that appear in the body

With v.0.6.0 a new features has been introduced that will allow users to process header information that appear somewhere in the table body. This is not an uncommon design choice and the question how such tables can be processed with R has been debated on stackoverflow. I illustrate this problem with a table from the American National Weather Service. Below is a crop of this table which should give you the basic idea:

The task is to assemble a table where the the model type information that appear in the body (global models, regional models, …) populate a seperate column in the final table. To this end, htmltab() has been extended to accept in its header argument a formula-like expression to signify the different dimensions of header information. The basic format of this formula interface is level1 + level2 + level3 + … , where you can express the position of each element either numerically or with a character vector for an XPath expression that identifies the respective element. So, for the table above, we pass 1 + “//tr/td[@colspan = '7']” which expresses that the first level header appears in row 1 and the second level headers appear in cells which have a colspan attribute of 7:

nomads <- htmltab(doc = "http://christianrubba.com/htmltab/ex/nomads.html", which = 1, header = 1 + "//tr/td[@colspan = '7']")

## Warning: The code for the HTML table you provided contains invalid table tags ('//trbody'). The following transformations were applied:
## 
##               //trbody -> //tbody 
## 
## If you specified an XPath that makes a reference to this tag, this may have caused problems with their identification.

## Warning: The code for the HTML table you provided is malformed. Not all
## cells are nested in row tags (<tr>). htmltab tried to normalize the table
## and ensure that all cells are within row tags. If you specified an XPath
## for body or header elements, this may have caused problems with their
## identification.

head(nomads, 22)

##           Header_1                                     Data Set     freq
## 3    Global Models                                         GDAS  6 hours
## 4    Global Models                              GFS 0.25 Degree  6 hours
## 5    Global Models            GFS 0.25 Degree (Secondary Parms)  6 hours
## 6    Global Models                              GFS 0.50 Degree  6 hours
## 7    Global Models                              GFS 1.00 Degree  6 hours
## 8    Global Models                 GFS Ensemble high resolution  6 hours
## 9    Global Models           GFS Ensemble Precip Bias-Corrected    daily
## 10   Global Models  GFS Ensemble high-resolution Bias-Corrected  6 hours
## 11   Global Models  GFS Ensemble NDGD resolution Bias-Corrected  6 hours
## 12   Global Models         NAEFS high resolution Bias-Corrected  6 hours
## 13   Global Models         NAEFS NDGD resolution Bias-Corrected  6 hours
## 14   Global Models                             NGAC 2D Products    daily
## 15   Global Models                             NGAC 3D Products    daily
## 16   Global Models          NGAC Aerosol Optical Depth Products    daily
## 17   Global Models        Climate Forecast System Flux Products  6 hours
## 18   Global Models Climate Forecast System 3D Pressure Products  6 hours
## 20 Regional Models                            AQM Daily Maximum 06Z, 12Z
## 21 Regional Models                     AQM Hourly Surface Ozone 06Z, 12Z
## 22 Regional Models                                 HIRES Alaska    daily
## 23 Regional Models                                  HIRES CONUS 12 hours
## 24 Regional Models                                   HIRES Guam 12 hours
## 25 Regional Models                                 HIRES Hawaii 12 hours
##    grib filter http     gds-alt
## 3  grib filter http OpenDAP-alt
## 4  grib filter http OpenDAP-alt
## 5  grib filter http        <NA>
## 6  grib filter http OpenDAP-alt
## 7  grib filter http OpenDAP-alt
## 8  grib filter http OpenDAP-alt
## 9  grib filter http OpenDAP-alt
## 10 grib filter http OpenDAP-alt
## 11 grib filter http OpenDAP-alt
## 12 grib filter http OpenDAP-alt
## 13 grib filter http OpenDAP-alt
## 14 grib filter http           -
## 15 grib filter http           -
## 16 grib filter http           -
## 17 grib filter http           -
## 18 grib filter http           -
## 20 grib filter http OpenDAP-alt
## 21 grib filter http OpenDAP-alt
## 22 grib filter http OpenDAP-alt
## 23 grib filter http OpenDAP-alt
## 24 grib filter http OpenDAP-alt
## 25 grib filter http OpenDAP-alt

In the data frame that was produced, we see that the model labels that appeared throughout the body now populate a seperate column. Such a format is almost always more useful for further exploration of the data frame either through summary statistics or visualizations. More generally, there is no limit with respect to the level of nestedness that the table exhibits. The only requirement for this feature to work is that the header levels must be strictly nested and you can specify the exact position of the elements through a numeric vector or an XPath.

For more information have a look at the package vignette. And as always, I am happy to hear about any problems you experience with the package.