Building R 4.2 for Windows with OpenBLAS

This post outlines the steps needed for building R 4.2 for Windows with OpenBLAS using the the new Rtools42 build system. With the soon-to-be released new UCRT toolchain for building R on Windows, the build process changes once again.

First and foremost, we Windows users owe a deep debt of gratitude first to Duncan Murdoch who supported the Windows builds through R3, then Jeroen Oomes who updated and re-platformed the toolchain for R4, and now Tomas Kalibera who has updated the toolchain to comply with R-core’s requirement that the default build for windows be UCRT-compliant and which will have native UTF8 support. Thankfully, the process is not much different from how Tomas explains it in his “how-to guide“. Therefore, I am going to leave the explanations to him, who knows this process much better and much more deeply than I do, and simply lay out the steps which are needed. Everything other than the specific steps to adjust the code to use OpenBLAS and machine-specific architecture is taken from his how-to guide and is better explained there as well!

Also, this post will not explain how to build OpenBLAS under the new toolchain, but will—for now—rely on the the library built by Tomas as part of the toolchain.

Step 1: Install Rtools42

Download the most recent version of Rtools42 from here and install it into its default directory (c:/rtools42).

Step 2: Update Rtools42

Step 2a: Update the build system

Run the msys.exe which was installed with Rtools42 and update the build system using:

pacman -Syuu
pacman -Sy wget subversion

Step 2b: Update the specialized libraries

Once the build system is updated, check the vintage of the libraries by calling

cat /x86_64-w64-mingw32.static.posix/.version

and comparing it with the version listed as XXXX in the gcc10_ucrt3_full_XXXX.tar.zst file in the repository. If the installed version is not up-to-date, run the following, replacing XXXX with the proper identifier:

cd /
wget https://www.r-project.org/nosvn/winutf8/ucrt3/gcc10_ucrt3_full_XXXX.tar.zst
rm -rf /x86_64-w64-mingw32.static.posix
tar xf gcc10_ucrt3_full_XXXX.tar.zst
rm gcc10_ucrt3_full_XXXX.tar.zst

Step 3: Install necessary support software

As per prior builds, the complete installation procedure requires MiKTeX, Inno Setup, and qpdf. Install the the first two programs into their default locations and unzip the most recent qpdf-XX.X.X-bin-mingw64.zip wherever you want.

Step 4: Download and extract the R source  code

In all of the following steps, a pattern of Xs are used to represent identifiers which are apt to change over time. Always check the repository to identify the most recent versions of files and substitute accordingly. Open msys.exe and change the directory to where you want the build to live. The default is your user directory, which on Windows is almost never a good choice. Once in the desired directory, run the following:

TCLBUNDLE=Tcl-XXXX-XXXX.zip
wget https://www.r-project.org/nosvn/winutf8/ucrt3/$TCLBUNDLE

svn checkout -r XXXXX https://svn.r-project.org/R/trunk

RDIFF=R-devel-XXXXX-XXXX.diff
wget https://www.r-project.org/nosvn/winutf8/ucrt3/$RDIFF

cd trunk
patch --binary -p0 < ../$RDIFF

unzip ../$TCLBUNDLE
cd src/gnuwin32

Step 5: Add a MkRules.local

In a similar reversion to the older build method, add a Mkrules.local for any options you may want. You must add one to get OpenBLAS to work, and that will be the first line of the file below. Add any other options, such as EOPTS or LTO as desired. Feel free to copy and adjust from below as needed. If your Inno Setup is not installed to the standard location, add a line ISDIR = /path/to/Inno. The last step above placed us in trunk/src/gnuwin32 which is where we need to be. If you want to manually write your own MkRules.local remember to do it in that specific subdirectory.

IMPORTANT: If want to build R 4.2 for Windows with OpenBLAS and specific machine architectures, you need to be careful. If you are going to pass march=native for a Haswell processor or later (for example, I have a skylake) you must pass -mno-fma. Otherwise, R will go into an infinite loop in the regression tests. According to Martin Maechler (offlist of r-devel), this appears to be a compiler bug in GCC, and I have confirmed it is present the 10.3 version used by Tomas. So fused-multiply-add needs to be turned off.

cat <<EOF >MkRules.local
USE_ATLAS = YES
EOPTS = -march=native -pipe -mno-rtm -mno-fma
LTO = -flto -ffat-lto-objects -fuse-linker-plugin
LTO_OPT = -flto -ffat-lto-objects -fuse-linker-plugin
LTO_FC_OPT = -flto -ffat-lto-objects -fuse-linker-plugin
QPDF = C:/R/qpdf-10.5.0
OPENMP = -fopenmp
EOF

Note there is no entry for (ATLAS_PATH)!

Step 6: Adjust src/extra/blas/Makefile.win

With Rtools40, the adjustment was made within the PKGBUILD process using patch files. While that can be done here as well (see the patch call above) I have found it easier to use Notepad++ to manually edit the appropriate Makefile.win. Similar to prior builds, the line that needs to be changed is -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lf77blas -latlas. Switch it to:

-L../../../$(IMPDIR) -lR -fopenmp -lopenblas

Yes, (ATLAS_PATH) was removed this time. The new toolchain has a pre-built OpenBLAS library which lives with all the other UCRT-compliant libraries in a default location which does not need an explicit address. It is built with the OpenMP flag set which is why -fopenmp must be passed.

Step 7: Add needed variables to the PATH

In the MSYS window, run the following. The MiKTeX line should not be needed if you have a default (user-only install). If you have an administrator install (as I do on my home computer), so miktex.exe lives in Program files, then keep the line in and adjust as needed.

export PATH=/c/rtools42/x86_64-w64-mingw32.static.posix/bin:/c/rtools42/usr/bin:$PATH
export PATH=/c/Program\ Files/MiKTeX/miktex/bin/x64:$PATH
export TAR="/usr/bin/tar"
export TAR_OPTIONS="--force-local"

Step 8: Check support tool availability

Run which make gcc pdflatex tar. You should get four locations. If you do not, at least one of your support files is not configured properly.

Step 9: Build R

Once everything is ready, then all that is left is to call make distribution in trunk/src/gnuwin32. After a long time, the build should complete and you should find the installer in trunk/src/gnuwin32/installer. You have now successfully completed building R 4.2 for Windows with OpenBLAS!

Step 10: Check

Normally, now is when you would call make check-devel. However, there is known issue with one of the regex tests which will cause utf8-regex to fail. Running make check-recommended and testing installed packages for base and recommended from inside the R GUI returned no other failures. As always, I’d appreciate thoughts and corrections in the comments; good luck!

The post Building R 4.2 for Windows with OpenBLAS appeared first on Strange Attractors.