Active Evaluation Software for Selection of Ground Truth Labels
This software repository contains a python package Aegis (Active Evaluator Germane Interactive Selector) package that allows us to evaluate machine learning systems's performance (according to a metric such as accuracy) by adaptively sampling trials to label from an unlabeled test set to minimize the number of labels needed. This includes sample (public) data as well as a simulation script that tests different label-selecting strategies on already labelled test sets. This software is configured so that users can add their own data and system outputs to test evaluation.
Complete Metadata
| @type | dcat:Dataset |
|---|---|
| accessLevel | public |
| bureauCode |
[
"006:55"
]
|
| contactPoint |
{
"fn": "Peter Fontana",
"hasEmail": "mailto:peter.fontana@nist.gov"
}
|
| description | This software repository contains a python package Aegis (Active Evaluator Germane Interactive Selector) package that allows us to evaluate machine learning systems's performance (according to a metric such as accuracy) by adaptively sampling trials to label from an unlabeled test set to minimize the number of labels needed. This includes sample (public) data as well as a simulation script that tests different label-selecting strategies on already labelled test sets. This software is configured so that users can add their own data and system outputs to test evaluation. |
| distribution |
[
{
"title": "DOI Access for Active Evaluation Software for Selection of Ground Truth Labels",
"accessURL": "https://doi.org/10.18434/M32227"
}
]
|
| identifier | ark:/88434/mds2-2227 |
| issued | 2020-07-09 |
| keyword |
[
"active evaluation",
"ar",
"machine learning"
]
|
| landingPage | https://github.com/usnistgov/active-evaluation |
| language |
[
"en"
]
|
| license | https://www.nist.gov/open/license |
| modified | 2020-04-28 00:00:00 |
| programCode |
[
"006:045"
]
|
| publisher |
{
"name": "National Institute of Standards and Technology",
"@type": "org:Organization"
}
|
| theme |
[
"Information Technology:Data and informatics"
]
|
| title | Active Evaluation Software for Selection of Ground Truth Labels |
