#23: Debugging with Docker and Rocker – A Concrete Example helping on macOS
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Welcome to the 23nd post in the rationally reasonable R rants series, or R4 for short. Today’s post was motivated by an exchange on the r-devel list earlier in the day, and a few subsequent off-list emails.
Roger Koenker posted a question: how to best debug an issue arising only with gfortran-9 which is difficult to get hold off on his macOS development platform. Some people followed up, and I mentioned that I had good success using Docker, and particularly our Rocker containers—and outlined a quick mini-tutorial (which had one mini-typo lacking the imporant slash in -w /work). Roger and I followed up over a few more off-list emails, and by and large this worked for him.
So what follows below is a jointly written / edited ‘mini HOWTO’ of how to deploy Docker on macOS for debugging under particular toolchains more easily available on Linux. Windows and Linux use should be very similar, albeit differ in the initial install. In fact, I frequently debug or test in Docker sessions when I do not want to install on my Linux host system. Roger sent one version (I had also edited) back to the list. What follows is my final version.
Debugging with Docker: Getting Hold of Particular Compilers
Context: The quantreg package was seen exhibiting errors when compiled with gfortran-9. The following shows how to use gfortran-9 on macOS by virtue of Docker. It is written in Roger Koenker’s voice, but authored by Roger and myself.
With extensive help from Dirk Eddelbuettel I have installed docker on my mac mini from
https://hub.docker.com/editions/community/docker-ce-desktop-mac
which installs from a dmg in quite standard fashion. This has allowed me to simulate running R in a Debian environment with gfortran-9 and begin the process of debugging my ancient rqbr.f code.
Some further details:
Step 0: Install Docker and Test
Install Docker for macOS following this Docker guide. Do some initial testing, e.g.
docker --version docker run hello-world
Step 1: Download r-base and test OS
We use the plainest Rocker container rocker/r-base, in the aliased form of the official Docker container for, i.e. r-base. We first ‘pull’, then test the version and drop into bash as second test.
docker pull r-base # downloads r-base for us docker run --rm -ti r-base R --version # to check we have the R we want docker run --rm -ti r-base bash # now in shell, Ctrl-d to exit
Step 2: Setup the working directory
We tell Docker to run from the current directory and access the files therein. For the work on quantreg package this is projects/rq for RogerL
cd projects/rq
docker run --rm -ti -v ${PWD}:/work -w /work r-base bash
This put the contents of projects/rq into the /work directory, and starts the session in /work (as can be seen from the prompt).
Next, we update the package information inside the container:
root@90521904fa86:/work# apt-get update Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB] Get:2 http://cdn-fastly.deb.debian.org/debian testing InRelease [117 kB] Get:3 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,385 kB] Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 Packages [7,916 kB] Fetched 16.6 MB in 4s (4,411 kB/s) Reading package lists... Done
Step 3: Install gcc-9 and gfortran-9
root@90521904fa86:/work# apt-get install gcc-9 gfortran-9 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: cpp-9 gcc-9-base libasan5 libatomic1 libcc1-0 libgcc-9-dev libgcc1 libgfortran-9-dev libgfortran5 libgomp1 libitm1 liblsan0 libquadmath0 libstdc++6 libtsan0 libubsan1 Suggested packages: gcc-9-locales gcc-9-multilib gcc-9-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libquadmath0-dbg gfortran-9-multilib gfortran-9-doc libgfortran5-dbg libcoarrays-dev The following NEW packages will be installed: cpp-9 gcc-9 gfortran-9 libgcc-9-dev libgfortran-9-dev The following packages will be upgraded: gcc-9-base libasan5 libatomic1 libcc1-0 libgcc1 libgfortran5 libgomp1 libitm1 liblsan0 libquadmath0 libstdc++6 libtsan0 libubsan1 13 upgraded, 5 newly installed, 0 to remove and 71 not upgraded. Need to get 35.6 MB of archives. After this operation, 107 MB of additional disk space will be used. Do you want to continue? [Y/n] Y Get:1 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libasan5 amd64 9.1.0-10 [390 kB] Get:2 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libubsan1 amd64 9.1.0-10 [128 kB] Get:3 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libtsan0 amd64 9.1.0-10 [295 kB] Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9-base amd64 9.1.0-10 [190 kB] Get:5 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libstdc++6 amd64 9.1.0-10 [500 kB] Get:6 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libquadmath0 amd64 9.1.0-10 [145 kB] Get:7 http://cdn-fastly.deb.debian.org/debian testing/main amd64 liblsan0 amd64 9.1.0-10 [137 kB] Get:8 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libitm1 amd64 9.1.0-10 [27.6 kB] Get:9 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgomp1 amd64 9.1.0-10 [88.1 kB] Get:10 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran5 amd64 9.1.0-10 [633 kB] Get:11 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libcc1-0 amd64 9.1.0-10 [47.7 kB] Get:12 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libatomic1 amd64 9.1.0-10 [9,012 B] Get:13 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc1 amd64 1:9.1.0-10 [40.5 kB] Get:14 http://cdn-fastly.deb.debian.org/debian testing/main amd64 cpp-9 amd64 9.1.0-10 [9,667 kB] Get:15 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc-9-dev amd64 9.1.0-10 [2,346 kB] Get:16 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9 amd64 9.1.0-10 [9,945 kB] Get:17 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran-9-dev amd64 9.1.0-10 [676 kB] Get:18 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gfortran-9 amd64 9.1.0-10 [10.4 MB] Fetched 35.6 MB in 6s (6,216 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 17787 files and directories currently installed.) Preparing to unpack .../libasan5_9.1.0-10_amd64.deb ... Unpacking libasan5:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../libubsan1_9.1.0-10_amd64.deb ... Unpacking libubsan1:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../libtsan0_9.1.0-10_amd64.deb ... Unpacking libtsan0:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../gcc-9-base_9.1.0-10_amd64.deb ... Unpacking gcc-9-base:amd64 (9.1.0-10) over (9.1.0-8) ... Setting up gcc-9-base:amd64 (9.1.0-10) ... (Reading database ... 17787 files and directories currently installed.) Preparing to unpack .../libstdc++6_9.1.0-10_amd64.deb ... Unpacking libstdc++6:amd64 (9.1.0-10) over (9.1.0-8) ... Setting up libstdc++6:amd64 (9.1.0-10) ... (Reading database ... 17787 files and directories currently installed.) Preparing to unpack .../0-libquadmath0_9.1.0-10_amd64.deb ... Unpacking libquadmath0:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../1-liblsan0_9.1.0-10_amd64.deb ... Unpacking liblsan0:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../2-libitm1_9.1.0-10_amd64.deb ... Unpacking libitm1:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../3-libgomp1_9.1.0-10_amd64.deb ... Unpacking libgomp1:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../4-libgfortran5_9.1.0-10_amd64.deb ... Unpacking libgfortran5:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../5-libcc1-0_9.1.0-10_amd64.deb ... Unpacking libcc1-0:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../6-libatomic1_9.1.0-10_amd64.deb ... Unpacking libatomic1:amd64 (9.1.0-10) over (9.1.0-8) ... Preparing to unpack .../7-libgcc1_1%3a9.1.0-10_amd64.deb ... Unpacking libgcc1:amd64 (1:9.1.0-10) over (1:9.1.0-8) ... Setting up libgcc1:amd64 (1:9.1.0-10) ... Selecting previously unselected package cpp-9. (Reading database ... 17787 files and directories currently installed.) Preparing to unpack .../cpp-9_9.1.0-10_amd64.deb ... Unpacking cpp-9 (9.1.0-10) ... Selecting previously unselected package libgcc-9-dev:amd64. Preparing to unpack .../libgcc-9-dev_9.1.0-10_amd64.deb ... Unpacking libgcc-9-dev:amd64 (9.1.0-10) ... Selecting previously unselected package gcc-9. Preparing to unpack .../gcc-9_9.1.0-10_amd64.deb ... Unpacking gcc-9 (9.1.0-10) ... Selecting previously unselected package libgfortran-9-dev:amd64. Preparing to unpack .../libgfortran-9-dev_9.1.0-10_amd64.deb ... Unpacking libgfortran-9-dev:amd64 (9.1.0-10) ... Selecting previously unselected package gfortran-9. Preparing to unpack .../gfortran-9_9.1.0-10_amd64.deb ... Unpacking gfortran-9 (9.1.0-10) ... Setting up libgomp1:amd64 (9.1.0-10) ... Setting up libasan5:amd64 (9.1.0-10) ... Setting up libquadmath0:amd64 (9.1.0-10) ... Setting up libatomic1:amd64 (9.1.0-10) ... Setting up libgfortran5:amd64 (9.1.0-10) ... Setting up libubsan1:amd64 (9.1.0-10) ... Setting up cpp-9 (9.1.0-10) ... Setting up libcc1-0:amd64 (9.1.0-10) ... Setting up liblsan0:amd64 (9.1.0-10) ... Setting up libitm1:amd64 (9.1.0-10) ... Setting up libtsan0:amd64 (9.1.0-10) ... Setting up libgcc-9-dev:amd64 (9.1.0-10) ... Setting up gcc-9 (9.1.0-10) ... Setting up libgfortran-9-dev:amd64 (9.1.0-10) ... Setting up gfortran-9 (9.1.0-10) ... Processing triggers for libc-bin (2.28-10) ... root@90521904fa86:/work# pwd
Here filenames and versions reflect the Debian repositories as of today, August 5, 2019. While minor details may change at a future point in time, the key fact is that we get the components we desire via a single call as the Debian system has a well-honed package system
Step 4: Prepare Package
At this point Roger removed some dependencies from the package quantreg that he knew were not relevant to the debugging problem at hand.
Step 5: Set Compiler Flags
Next, set compiler flags as follows:
root@90521904fa86:/work# mkdir ~/.R; vi ~/.R/Makevars
adding the values
CC=gcc-9 FC=gfortran-9 F77=gfortran-9
to the file. Alternatively, one can find the settings of CC, FC, CXX, … in /etc/R/Makeconf (which for the Debian package is a softlink to R’s actual Makeconf) and alter them there.
Step 6: Install the Source Package
Now run
R CMD INSTALL quantreg_5.43.tar.gz
which uses the gfortran-9 compiler, and this version did reproduce the error initially reported by the CRAN maintainers.
Step 7: Debug!
With the tools in place, and the bug reproduces, it is (just!) a matter of finding the bug and fixing it.
And that concludes the tutorial.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.