The REDCap Cypress Developer Toolkit provides an automated framework for validating REDCap’s core functionality. Automated testing helps ensure system integrity, reduces manual testing burden, and supports institutions in meeting regulatory compliance requirements (e.g., 21 CFR Part 11).
The toolkit was the brain child of Adam De Fouw, who deserves most of the credit for getting it off the ground. This and all related projects were originally developed by him and the RSVC Automated Testing Subcommittee (ATS). They are now maintained by Vanderbilt University Medical Center (VUMC). RSVC continues to contribute to updates and actively uses the toolkit for REDCap validation efforts.
This repository includes scripts to download all the necessary components for a developer to begin developing automated feature tests on their developer machine. This toolkit is based on the industry-standard Cypress Testing Framework.
If you'd like to participate in our ongoing automation efforts, see our Contribution Guidelines.
Automated testing involves executing pre-scripted tests against a REDCap instance to validate that core functionality works as expected. These tests:
- Ensure system stability after software updates
- Reduce redundant manual testing for common workflows
- Support regulatory compliance efforts by providing structured validation
Tests are executed in a controlled test environment and should never be run on a production system, as they modify project data.
Not all REDCap features are suitable for automation. When determining which features should be tested automatically, the RSVC considers:
- Reproducibility – Can the test be executed consistently across institutions?
- Regulatory Impact – Does the feature support compliance with 21 CFR Part 11?
- Complexity – Is automation feasible, or does the feature require manual review?
- Test Stability – Does the test produce reliable results across different REDCap versions?
For each REDCap release, RSVC reviews system changes and determines which features should be tested manually and which can be automated.
- Software Prerequisites
- Create SSH Key
- Developer Toolkit Installation Instructions
- Changing REDCap Versions
- Start REDCap Test Environment
- Configure & Start Cypress
- Contribute to Feature Tests
- Update Repositories
- Rebuild Docker Containers
The following steps are intended to help set up a testing environment on a personal computer in order to aid with development and contribute changes back to this project. If instead you're hoping to re-run all automated tests at your institution, see Creating An Automated Testing Environment.
A developer needs the following software on their machine before installing this Developer Toolkit.
-
Git (version control)
- For Windows
- For macOS options (choose one)
- For Linux
-
Rancher Desktop - This is a drop-in replacement for Docker Desktop, which is still supported but not allowed at some organizations (like VUMC).
- You may want to set Rancher to
Automatically start on loginandStart in the backgroundunderPreferences -> Application -> Behavior - If you've previously had Docker Desktop installed, you may have to run
docker context use defaultbefore Rancher will work as expected.
- You may want to set Rancher to
-
Node.js - available for Windows, macOS, Linux
-
VS Code - This is the recommended IDE, but others may be used as well (e.g. PhpStorm). Be mindful that Visual Studio Code is most often referred to as VS Code in part to distinguish it from a different application simply called Visual Studio which we do not use.
You will need to place your public key on GitHub for this process to work correctly.
To generate a key on your local machine, most of time the command is:
ssh-keygen
Please consult GitHub's SSH documentation for more information: GitHub SSH Key Instructions
Specifically, you will need to
-
Clone this repository to your machine. 1
git clone [email protected]:vanderbilt-redcap/redcap_cypress_docker.git
To test against a different version of REDCap, simply update the REDCAP_VERSION variable in the config.yml file in the /redcap_cypress/.circleci/ folder to your target REDCap version.
REDCAP_VERSION: "15.5.7"
Run the following from a bash compatible command line to start the test environment. Git Bash is supported on Windows, and standard terminals are supported on Mac & Linux. You will be prompted for REDCap Community site credentials in order to download REDCap the first time you run this and after certain updates.
./run.sh
You should perform the following steps periodically to ensure your local environment includes the latest changes:
- Close cypress (if it's open)
- Run
./update.sh - Run
./run.shto open cypress again
-
The single most useful tool is
Continue Last Run.feature. The REDCap database is normally cleared between feature runs. However, executing this special feature file immediately runs whatever steps you place in it against the current state of the database. Rather than repeatedly running any features files you're working on from the beginning, you should use this file to test a small number of steps at a time to greatly reduce iteration time and significantly speed up your workflow. -
Execution can be paused in the middle of a feature to check details or perform manual actions by adding the following step:
And I want to pause. To continue execution, press the play button in the Cypress UI.
-
Create your own fork of redcap_rsvc that is based upon https://github.com/vanderbilt-redcap/redcap_rsvc
-
Configure the cloned redcap_rsvc repository as needed to match your own Fork.
cd redcap_cypress/redcap_rsvc
git remote rename origin upstream
git remote add origin <your_fork_url_here>
Having your own fork enables you to issue pull requests to vanderbilt-redcap/redcap_rsvc after you complete a feature.
For more information about the innerworkings, see the REDCap Cypress Test Suite Docs
Footnotes
-
Git Clone Fail: If the message says you do not have permissions or mentions a public key, you might need to setup a SSH key with Github. ↩