Skip to main content
U.S. flag

An official website of the United States government

This site is currently in beta, and your feedback is helping shape its ongoing development.

Return to search results

Challenging Medically-Relevant Genes Benchmark Set

Published by National Institute of Standards and Technology | National Institute of Standards and Technology | Metadata Last Checked: August 02, 2025 | Last Modified: 2021-09-29 00:00:00
CMRG v1.00 of a small variant benchmark and structural variant benchmark focused on 273 challenging medically relevant genes for the Genome in a Bottle (GIAB) sample HG002 (aka Ashkenazi son). These benchmarks were generated from a trio-based hifiasm v0.11 (https://doi.org/10.1038/s41592-020-01056-5) diploid assembly of HG002 using PacBio HiFi reads for HG002 for assembly and partitioning into phased haplotypes using Illumina reads for the parents, HG003 and HG004. This benchmark contains vcfs for small and structural variants along with corresponding benchmark bed files indicating regions that are homozygous reference if they do not have a variant in the vcf. We extensively curated the variant calls, excluding any found to be questionable or errors. This benchmark helps measure performance in important challenging regions, including challenging segmental duplications, regions with complex variants, regions with structural variants, and regions affected by false duplications in GRCh37 or GRCh38. This benchmark is described in https://doi.org/10.1101/2021.06.07.444885.

Resources

65 resources available

Find Related Datasets

Click any tag below to search for similar datasets

Complete Metadata
@type dcat:Dataset
accessLevel public
bureauCode 
                                            [
  "006:55"
]
contactPoint 
                                            {
  "fn": "Nathanael David Olson",
  "hasEmail": "mailto:nathanael.olson@nist.gov"
}
description CMRG v1.00 of a small variant benchmark and structural variant benchmark focused on 273 challenging medically relevant genes for the Genome in a Bottle (GIAB) sample HG002 (aka Ashkenazi son). These benchmarks were generated from a trio-based hifiasm v0.11 (https://doi.org/10.1038/s41592-020-01056-5) diploid assembly of HG002 using PacBio HiFi reads for HG002 for assembly and partitioning into phased haplotypes using Illumina reads for the parents, HG003 and HG004. This benchmark contains vcfs for small and structural variants along with corresponding benchmark bed files indicating regions that are homozygous reference if they do not have a variant in the vcf. We extensively curated the variant calls, excluding any found to be questionable or errors. This benchmark helps measure performance in important challenging regions, including challenging segmental duplications, regions with complex variants, regions with structural variants, and regions affected by false duplications in GRCh37 or GRCh38. This benchmark is described in https://doi.org/10.1101/2021.06.07.444885.
distribution 
                                            [
  {
    "title": "DOI Access for Challenging Medically-Relevant Genes Benchmark Set",
    "accessURL": "https://doi.org/10.18434/mds2-2475"
  },
  {
    "title": "GIAB FTP Site",
    "accessURL": "https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_NA24385_son/CMRG_v1.00/",
    "description": "NCBI Hosted Genome In A Bottle FTP Site"
  },
  {
    "title": "Code for Manuscript Analysis Repository",
    "accessURL": "https://github.com/usnistgov/cmrg-benchmarkset-manuscript",
    "description": "Github repository with code used to generate figures and perform analysis for manuscript."
  },
  {
    "title": "Code Repository",
    "accessURL": "https://github.com/usnistgov/giab-cmrg-benchmarkset",
    "description": "Github repository with code used to generate benchmark sets."
  },
  {
    "mediaType": "text/plain",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/README.md"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002_CHM13_CMRG_smallvar_v1.00_GRCh38-equiv-regions_draft.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/GRCh37_CMRG_benchmark_gene_coordinates.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/GRCh38_CMRG_benchmark_gene_coordinates.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/chksum.md5"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_MRG_GAPs.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_curation_medicalgene_SV_errorsorunsure.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_curation_medicalgene_smallvar_complexrepeat_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_hifiasm_error.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_mrg_full_gene.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_CD4_gaps.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_CD4_gaps_slop50.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_MRG_GAPs.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_curation_medicalgene_SV_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_curation_medicalgene_smallvar_complexrepeat_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_hifiasm_error.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_mrg_full_gene.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh37_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_FPs_repeatexpanded_slop50.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh38_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_FPs_repeatexpanded_slop50.bed"
  },
  {
    "mediaType": "text/plain",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh38_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_subtract_FPs_repeatexpanded_slop50_manual_curation_sites.tsv_manual_curation_sites.txt"
  },
  {
    "mediaType": "text/tab-separated-values",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/combined%20curation%20responses%20from%20benchmarking%20with%20sm%20variant%20v0.02.03%20-%20GRCh37andGRCh38.tsv"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.mat.fa.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.mat.gff.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.pat.fa.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.pat.gff.gz"
  }
]
identifier ark:/88434/mds2-2475
keyword 
                                            [
  "Bioinformatics",
  "Bioinformatics",
  "DNA sequencing",
  "Human genomics",
  "Medical genomics",
  "Reference materials"
]
landingPage https://data.nist.gov/od/id/mds2-2475
language 
                                            [
  "en"
]
license https://www.nist.gov/open/license
modified 2021-09-29 00:00:00
programCode 
                                            [
  "006:045"
]
publisher 
                                            {
  "name": "National Institute of Standards and Technology",
  "@type": "org:Organization"
}
references 
                                            [
  "https://doi.org/10.1038/s41592-020-01056-5",
  "https://doi.org/10.1101/2021.06.07.444885"
]
theme 
                                            [
  "Bioscience:Genomics"
]
title Challenging Medically-Relevant Genes Benchmark Set

data.gov

An official website of the GSA's Technology Transformation Services

Looking for U.S. government information and services?
Visit USA.gov