Outcome

The specification provided by Copilot workspace diagnosed the issue requirements, mentioning that the NAMESPACE file, which stores information on functions offered by a package and functions imported from other packages, would have to be updated for the R package to export this currently unexported function. The proposed solution also addressed the desired changes, stating that the function was to be exported and the keyword internal would be removed. The automatically generated plan proposed to edit the file documentation and NAMESPACE, but we edited this to more specifically replace the @keyword internal {roxygen} tag with @export.

Our thoughts

The implementation offered by GitHub copilot workspace was correct and replaced @keywords internal with @export in the correct function documentation epinow2_stan_model().

However, it also changed the formatting of one of the function arguments (@param model) by moving some text to the next line and added a comma to the last but one item listed in the sentence. (It seems AI has taken a side on the Oxford comma debate).

Overall, this test case of GitHub copilot workspace was a partial success. It correctly recognised the links between inline R function documentation, Roxygen tags, and the R-specific documentation files in man/. However, it (naturally) does not have a mechanistic understanding of package development and how changing tags in documentation is used to auto-update doc files using development tools. It correctly modified the part of the file needed to export an internal function. This showcases an example of where this technology can be beneficial: those new to a language or paradigm that have not yet picked up all of the development details that are often second nature to experienced developers. In this example, the time taken to make this change in GitHub copilot workspace was likely ~10x longer than what our RSEs would take. However, if someone was used to developing in say, Python, and was new to the conventions of exporting functions in R using {roxygen} documentation, this test case shows how copilot workspace can reliably assist. On the other hand, because these changes make no reference to standard development tools, they would not help new developers learn how or where to use them.

The changes made will still require the user to manually run the documentation (using either devtools::document() or roxygen2::roxygenize()) to update the package NAMESPACE for the function to be exported. Our recommendation would be for AI tools to integrate with and call development tools used by developers (analogous to pre-commit hooks running tools before committing) to make sure this is taken care of all in a single generative step.

Outcome

GitHub copilot workspace outlined the existing functionality in the package and proposed changes including adding a new file called R/model_sir.R, which follows the model_*.R naming convention used in {epidemics}. As requested, copilot also proposed that the new SIR model would support the use of interventions that are already been implemented in the package.

Our thoughts

The code generated was an R function that constructed a basic SIR model, and this was well documented using a {roxygen2} documentation block. However, several aspects of the code generated did not match the proposal generated by copilot workspace, and the code contained inadvisable coding practices in R (i.e. code smells). Firstly the SIR model itself. The model implemented follows the standard set of differential equations that are solved using {deSolve}, as requested in our prompt. However, the model does not have any options to input interventions, which the proposed solution by GitHub copilot workspace suggested it would. The second downside is the use of require(deSolve) in the function body of the generated code. This is bad practice in R package development as functions should namespace any functions that are being called from other packages (e.g. packagename::func_name()).

This task required copilot workspace to understand the contents and conventions of the {epidemics} R package and generate a simple compartmental model which would integrate with the infrastructure in the package for defining interventions in models. Although it correctly summarised the package contents, its attempt to answer the specific request from our prompt had many inaccuracies. The generated code is likely influenced by the many examples of SIR models written in R on the internet and on GitHub. These might be R scripts or gists and thus coding practises used there are not always appropriate for writing code to be distributed in an R package, resulting in decisions like having require(deSolve) in the function.

AI-generated code like that seen in this example showcases where users need to be diligent and not assume that the text descriptions from copilot match the code. In the end, this was a test example and was unlikely to be merged into {epidemics} even if the code was correct. However, with the code generated, it presents almost no use; it would require as much time for an RSE to refactor this function as it would to write one from scratch following the design principles and demands of the {epidemics} package.

Outcome

Here, Copilot provided no analysis of the code base. It regurgitated the changes described in the package review pull request, likely from a mix of the pull request description and the changelog (NEWS.md file), and did not analyse any of the code. Therefore, it provided no additional use than if the reviewer had just read the NEWS.