This vignette steps through the process of creating an R package that
is intended to be validated, by extending usethis and
devtools functionality to provide a seamless validation
process.
There is a combination of explaining how to use {valtools} and tutorial steps for the reader to follow along in creating a simple R package that could undergo validation.
Starting a new R package using {valtools} is a one line command, where the only information required is the path to the location the new package structure is to be built, with the last folder name in the path being the new name of the package.
The package is constructed using the
usethis::create_package() function, this function acts as a
wrapper and adds the necessary validation infrastructure required for
{valtools}.
Importantly, there is now a vignettes/validation folder,
which is the working directory for the validation. This is where almost
all the content for validation will be created.
Inside this folder, there is the validation.yml file,
which will be referred as the validation config file going forward. This
YML file informs {valtools} how to interact with the various pieces of
validation that will be created, and information that needs to be shared
across multiple sessions/users. The user does not need to interact with
this file directly, the functions inside {valtools} will update this
file as necessary.
Run the chunk of code below to create a package in a temporary directory to follow along with the tutorial. The new package will be created in a new session. Run all subsequent code in that new session.
To examine the folder structure of the new package, run the following function:
Requirements document the goals of the package - what it does to
solve whatever problem or task - and should be documented before any
code is written. Requirements are recorded within the
vignettes/validation/requirements folder by default. The
collection of requirements may be called specifications.
To make adding validation content easy, {valtools} extended the
usethis approach to package contents creation through a
family of “vt_use_*” functions.
vt_use_req() creates a new requirement in the
vignettes/validation/requirements folder, with the main
argument being the name of the requirement, and an optional argument
username to record the name of the person writing the
requirement.
If the username argument is not passed, {valtools} will
automatically get the computer username of the user creating the
requirement and attempt to put in their full name. If the user has not
created any validation contents before, it will ask the user some
questions (Name, Title, and Role) and record them in the validation
config file for documentation in the validation report.
Run the command above and in the newly opened requirements file, on
line 5, Replace REQUIREMENTS with 1.1, and
ASSESSMENT with 1, Low Risk, Small Impact to
indicate requirement 1.1 has a risk assessment that determined it has a
low risk and small impact when it is wrong.
Add a new line underneath the line above (at line 6) line that
contains: #' 1.2: 5, Low risk, Medium Impact
Copy the following content:
## Say Hello
+ 1.1 Say Hello to users
+ 1.2 Say Hello to multiple users at onceNow that requirements are set, the normal package development process
can be followed. {valtools} expects the developers to be using
usethis and devtools to create the rest of the
package contents, but it is not required.
A critical piece to validation during development is tracking which
programmers wrote which functions. {valtools} supports this by
augmenting the {roxygen2} framework of function documentation with two
custom roxygen tags: @editor and
@editDate.
@editor is for tracking the last editor of the function,
and @editDate is for recording whenever a function is
modified.
Run the command below to create an R file in the package and copy the following code into the file.
#' Hello World
#' @param name name to say hello to
#' @returns character string saying hello
#' @editor Ellis Hughes
#' @editDate 2021-03-12
#' @export
hello_world <- function(name){
paste("Hello,",name)
}
#' Hello World - internal
#' @param name name to say hello to
#' @returns character string saying hello
#' @editor Ellis Hughes
#' @editDate 2021-03-12
hello_world_internal <- function(name){
paste("hello,",name)
}Similar to a news file, {valtools} suggests the use of a change log that is directly tied to validation. The purpose of this is to separate package update and information that is useful for developers from information that is important to capture in validation.
To create this change log file, {valtools} has the function
vt_use_change_log(). It will create the change log file
inside the working directory and open it up for editing.
The header information tracks the version of validation (usually tied to the package version when developing a package), and the date of the release of validation. This is a markdown file, so normal markdown can be used to document the changes. However, critically here, only bullets marked with [validation] will be recorded in the validation report.
Run the command above to create a change log.
Testing is done to ensure that the package meets the requirements that were set out for the project. Testing is done in two major steps: the firsts consists of writing out a series of cases that would prove that the requirements have been met, the second is the application of these cases.
The addition and writing of test cases is handled by the
vt_use_test_case() function. Similarly to
vt_use_req(), a username can be passed, or it will look to
determine which user is calling the function and input their
information.
This function creates the test case file in the
vignettes/validation/test_cases folder of the package and
opens it for editing.
Run the code above and in the newly opened test case file, replace
TESTCASE with 1.1, and
REQUIREMENT with 1.1 to indicate test case 1.1
shows that requirement 1.1 is being met.
Add a new line underneath the line above (at line 6) line that
contains: #' 1.2: 1.1, 1.2
This is to indicate test case 1.2 shows requirements 1.1 and 1.2 are being met.
Copy the following test case into file where test cases are to be documented:
## Test Case 1: hello_world
+ 1.1 test that the software can say "hello, Johnny" by passing a user "Johnny"
to the hello_world function
+ 1.2 test that the software can say hello to multiple people by passing a vector of users - "Johnny", "Beth", "Andy","Elisabeth"
to the hello_world function and getting back a vector of: c("Hello, Johnny", "Hello, Beth", "Hello, Andy","Hello, Elisabeth")Test code is the implementation of the test cases as code. The goal is that the code is completely reproducible and able to be run without human interaction. Additionally, test code is written by a third party - someone that was not involved with writing the actual code or the test case. This helps ensure the integrity of the testing as well as providing valuable review of the documentation of the test cases and package code.
Similarly to vt_use_req() for requirements and
vt_use_test_case for test cases, {valtools} provide a
function for creating test code files and recording which user created
the file.
Add “Val A Dashun” to the validation config file:
valtools::vt_add_user_to_config(
username = "user_b",
name = "Val A Dashun",
title = "Programmer II",
role = "tester"
)Now that this persons information is recorded, construct the test code file that they will use to record the test code through the code below.
In the newly opened test code file. Update TESTNUMBER to
1.1 in the new test code file and copy the code below into
the body of the test:
add a new test with the following beneath the test. Replace “TODAYS DATE” with today’s date.
Now that there is a validation report as an Rmarkdown, validation is only a compiling of the report away. However, different organizations might follow different approaches to when the validation documentation might be necessary. {valtools} can support several different methods of validation at this moment:
| Validation Type | Description | {valtools} Function |
|---|---|---|
| Validation on Source | Validation of the source code, where individuals might install from a controlled source. | valtools::vt_validate_source() |
| Validation on Build | Validation of the source code and then compiles it into a usable tar so that the source code cannot change after validation, so users can install from a controlled source without having to build/compile as well. | valtools::vt_validate_build() |
| Validation on Install | Validation of the source code, and then the validation report is also re-built on install to the machine it will ultimately be running on. | valtools::vt_validate_install() |
| Validation After Install | This assumes the package was built and installed using the prior methods, but in this method, the validation report can be re-run at any point to re-validated based on updates to the environment. | valtools::vt_validate_installed_package() |
Run each method of validation and observe the differences