Unit and Regression Tests of Scientific Software: A Study on SWMM
Testing helps assure software quality by executing programs and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons such as inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM), we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 1,458 SWMM tests have a 54.0% code coverage and an 82.4% User’s Manual coverage. Meanwhile, an examination of eight regression tests from a test set shows a 79.5% code coverage and a near 100% User’s Manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests and suggest a diversified way of designing or adopting regression tests.
This dataset is associated with the following publication:
Peng, Z., X. Lin, M. Simon, and N. Niu. Unit and Regression Tests of Scientific Software: A Study on SWMM. Journal of Computational Science. Elsevier B.V., Amsterdam, NETHERLANDS, 53: 101347, (2021).
Complete Metadata
| accessLevel | public |
|---|---|
| bureauCode |
[
"020:00"
]
|
| contactPoint |
{
"fn": "Michelle Simon",
"hasEmail": "mailto:simon.michelle@epa.gov"
}
|
| describedBy | https://pasteur.epa.gov/uploads/10.23719/1519338/documents/Unit%20and%20Regression%20Tests%20of%20Scientific%20Softwaredatadictionary.docx |
| describedByType | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| description | Testing helps assure software quality by executing programs and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons such as inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM), we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 1,458 SWMM tests have a 54.0% code coverage and an 82.4% User’s Manual coverage. Meanwhile, an examination of eight regression tests from a test set shows a 79.5% code coverage and a near 100% User’s Manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests and suggest a diversified way of designing or adopting regression tests. This dataset is associated with the following publication: Peng, Z., X. Lin, M. Simon, and N. Niu. Unit and Regression Tests of Scientific Software: A Study on SWMM. Journal of Computational Science. Elsevier B.V., Amsterdam, NETHERLANDS, 53: 101347, (2021). |
| distribution |
[
{
"title": "Mappings_of_SWMM_unit_tests.xlsx",
"mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/Mappings_of_SWMM_unit_tests.xlsx"
},
{
"title": "regression test data.xlsx",
"mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/regression%20test%20data.xlsx"
},
{
"title": "test study on swmm(data table).xlsx",
"mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/test%20study%20on%20swmm%28data%20table%29.xlsx"
}
]
|
| identifier | https://doi.org/10.23719/1519338 |
| keyword |
[
"Storm Water Management Model (SWMM)",
"User’s Manual",
"regression testing",
"scientific software",
"test-coverage",
"unit testing"
]
|
| license | https://pasteur.epa.gov/license/sciencehub-license.html |
| modified | 2020-09-08 |
| programCode |
[
"020:096"
]
|
| publisher |
{
"name": "U.S. EPA Office of Research and Development (ORD)",
"subOrganizationOf": {
"name": "U.S. Environmental Protection Agency",
"subOrganizationOf": {
"name": "U.S. Government"
}
}
}
|
| references |
[
"https://doi.org/10.1016/j.jocs.2021.101347",
"https://pasteur.epa.gov/uploads/10.23719/1519338/documents/SWMM%20OpenCppCoverage%20Step.pdf"
]
|
| rights |
null
|
| title | Unit and Regression Tests of Scientific Software: A Study on SWMM |
