Class CrosscheckFingerprints

java.lang.Object
picard.cmdline.CommandLineProgram
picard.fingerprint.CrosscheckFingerprints
Direct Known Subclasses:
CrosscheckReadGroupFingerprints

@DocumentedFeature public class CrosscheckFingerprints extends CommandLineProgram
Checks that all data in the set of input files appear to come from the same individual. Can be used to compare according to readgroups, libraries, samples, or files. Operates on bams/sams and vcfs (including gvcfs).

Summary

Checks if all the genetic data within a set of files appear to come from the same individual. It quickly determines whether a "group's" genotype matches that of an input SAM/BAM/VCF by selective sampling, and has been designed to work well even for low-depth SAM/BAMs.
The tool collects "fingerprints" (essentially genotype information from different parts of the genome) at the finest level available in the data (readgroup for SAM files and sample for VCF files) and then optionally aggregates it by library, sample or file, to increase power and provide results at the desired resolution. Output is in a "Moltenized" format, one row per comparison. The results will be emitted into a metric file for the class CrosscheckMetric. In this format, the output will include both the LOD score and the tumor-aware LOD score; the latter can help assess identity even in the presence of a severe loss of heterozygosity with high purity (the tool could otherwise fail to notice that samples are from the same individual.) A matrix output is also available to facilitate visual inspection of crosscheck results.
Since there can be many rows of output in the metric file, we recommend the use of ClusterCrosscheckMetrics as a follow-up step to running CrosscheckFingerprints.
There are cases where one would like to identify a few groups out of a collection of many possible groups (say to link a bam to its correct sample in a multi-sample vcf. In this case, one would not care for the cross-checking of the various samples in the VCF against each other, but only in checking the identity of the bam against the various samples in the vcf. The SECOND_INPUT is provided for this use-case. With SECOND_INPUT provided, CrosscheckFingerprints does the following:
  • aggregation of data happens independently for the input files in INPUT and SECOND_INPUT.
  • aggregation of data happens at the SAMPLE level.
  • each sample from INPUT will only be compared to that same sample in INPUT.
  • MATRIX_OUTPUT is disabled.

  • In some cases, the groups collected may not have any observations (calls for a vcf, reads for a bam) at fingerprinting sites, or a sample in INPUT may be missing from the SECOND_INPUT. These cases are handled as follows: If running in CHECK_SAME_SAMPLES mode with INPUT and SECOND_INPUT, and either INPUT or SECOND_INPUT includes a sample not found in the other, or contains a sample with no observations at any fingerprinting sites, an error will be logged and the tool will return EXIT_CODE_WHEN_MISMATCH. In all other running modes, when any group which is being crosschecked does not have any observations at fingerprinting sites, a warning is logged. As long as there is at least one comparison where both sides have observations at fingerprinting sites, the tool will return zero. However, if all comparisons have at least one side with no observations at fingerprinting sites, an error will be logged and the tool will return EXIT_CODE_WHEN_NO_VALID_CHECKS.

    Examples

    Check that all the readgroups from a sample match each other:

         java -jar picard.jar CrosscheckFingerprints \
              INPUT=sample.with.many.readgroups.bam \
              HAPLOTYPE_MAP=fingerprinting_haplotype_database.txt \
              LOD_THRESHOLD=-5 \
              OUTPUT=sample.crosscheck_metrics
     

    Check that all the readgroups match as expected when providing reads from two samples from the same individual:

         java -jar picard.jar CrosscheckFingerprints \
              INPUT=sample.one.with.many.readgroups.bam \
              INPUT=sample.two.with.many.readgroups.bam \
              HAPLOTYPE_MAP=fingerprinting_haplotype_database.txt \
              LOD_THRESHOLD=-5 \
              EXPECT_ALL_GROUPS_TO_MATCH=true \
              OUTPUT=sample.crosscheck_metrics
     

    Detailed Explanation

    This tool calculates the LOD score for identity check between "groups" of data in the INPUT files as defined by the CROSSCHECK_BY argument. A positive value indicates that the data seems to have come from the same individual or, in other words, the identity checks out. The scale is logarithmic (base 10), so a LOD of 6 indicates that it is 1,000,000 times more likely that the data matches the genotypes than not. A negative value indicates that the data do not match. A score that is near zero is inconclusive and can result from low coverage or non-informative genotypes. Each group is assigned a sample identifier (for SAM this is taken from the SM tag in the appropriate readgroup header line, for VCF this is taken from the column label in the file-header. After combining all the data from the same "group" together, an all-against-all comparison is performed. Results are categorized as a CrosscheckMetric.FingerprintResult enum: EXPECTED_MATCH, EXPECTED_MISMATCH, UNEXPECTED_MATCH, UNEXPECTED_MISMATCH, or AMBIGUOUS depending on the LOD score and on whether the sample identifiers of the groups agree: LOD scores that are less than LOD_THRESHOLD are considered mismatches, and those greater than -LOD_THRESHOLD are matches (between is ambiguous). If the sample identifiers are equal, the groups are expected to match. They are expected to mismatch otherwise.
    The identity check makes use of haplotype blocks defined in the HAPLOTYPE_MAP file to enable it to have higher statistical power for detecting identity or swap by aggregating data from several SNPs in the haplotype block. This enables an identity check of samples with very low coverage (e.g. ~1x mean coverage).
    When provided a VCF, the identity check looks at the PL, GL and GT fields (in that order) and uses the first one that it finds.
    • Field Details

      • INPUT

        @Argument(shortName="I", doc="One or more input files (or lists of files) with which to compare fingerprints.", minElements=1) public List<String> INPUT
      • INPUT_INDEX_MAP

        @Argument(doc="A tsv with two columns and no header which maps the input files to corresponding indices; to be used when index files are not located next to input files. First column must match the list of inputs. ", optional=true) public File INPUT_INDEX_MAP
      • REQUIRE_INDEX_FILES

        @Argument(doc="A boolean value to determine whether input files should only be parsed if index files are available. Without turning this option on, the tool will need to read through the entirety of input files without index files either provided via the INPUT_INDEX_MAP or locally accessible relative to the input, which significantly increases runtime. If set to true and no index is found for a file, an exception will be thrown. This applies for both the INPUT and SECOND_INPUT files.") public boolean REQUIRE_INDEX_FILES
      • INPUT_SAMPLE_MAP

        @Argument(doc="A tsv with two columns representing the sample as it appears in the INPUT data (in column 1) and the sample as it should be used for comparisons to SECOND_INPUT (in the second column). Need only include the samples that change. Values in column 1 should be unique. Values in column 2 should be unique even in union with the remaining unmapped samples. Should only be used with SECOND_INPUT. ", optional=true, mutex="INPUT_SAMPLE_FILE_MAP") public File INPUT_SAMPLE_MAP
      • INPUT_SAMPLE_FILE_MAP

        @Argument(doc="A tsv with two columns representing the sample as it should be used for comparisons to SECOND_INPUT (in the first column) and the source file (in INPUT) for the fingerprint (in the second column). Need only to include the samples that change. Values in column 1 should be unique even in union with the remaining unmapped samples. Values in column 2 should be unique in the file. Will error if more than one sample is found in a file (multi-sample VCF) pointed to in column 2. Should only be used in the presence of SECOND_INPUT. ", optional=true, mutex="INPUT_SAMPLE_MAP") public File INPUT_SAMPLE_FILE_MAP
      • SECOND_INPUT

        @Argument(shortName="SI", optional=true, mutex="MATRIX_OUTPUT", doc="A second set of input files (or lists of files) with which to compare fingerprints. If this option is provided the tool compares each sample in INPUT with the sample from SECOND_INPUT that has the same sample ID. In addition, data will be grouped by SAMPLE regardless of the value of CROSSCHECK_BY. When operating in this mode, each sample in INPUT must also have a corresponding sample in SECOND_INPUT. If this is violated, the tool will proceed to check the matching samples, but report the missing samples and return a non-zero error-code.") public List<String> SECOND_INPUT
      • SECOND_INPUT_INDEX_MAP

        @Argument(doc="A tsv with two columns and no header which maps the second input files to corresponding indices; to be used when index files are not located next to second input files. First column must match the list of second inputs. ", optional=true) public File SECOND_INPUT_INDEX_MAP
      • SECOND_INPUT_SAMPLE_MAP

        @Argument(doc="A tsv with two columns representing the sample as it appears in the SECOND_INPUT data (in column 1) and the sample as it should be used for comparisons to INPUT (in the second column). Note that in case of unrolling files (file-of-filenames) one would need to reference the final file, i.e. the file that contains the genomic data. Need only include the samples that change. Values in column 1 should be unique. Values in column 2 should be unique even in union with the remaining unmapped samples. Should only be used with SECOND_INPUT. ", optional=true) public File SECOND_INPUT_SAMPLE_MAP
      • SAMPLE_INDIVIDUAL_MAP

        @Argument(doc="A tsv with two columns representing the individual with which each sample is associated. The first column is the sample id, and the second column is the associated individual id. Values in the first column must be unique. If INPUT_SAMPLE_MAP or SECOND_INPUT_SAMPLE_MAP is also specified, then the values in the first column of this file should be the sample aliases specified in the second columns of INPUT_SAMPLE_MAP and SECOND_INPUT_SAMPLE_MAP, respectively. When this input is specified, expectations for matches will be based on the equality or inequality of the individual ids associated with two samples, as opposed to the sample ids themselves. Samples which are not listed in this file will have their sample id used as their individual id, for the purposes of match expectations. This means that one sample id could be used as the individual id for another sample, but not included in the map itself, and these two samples would be considered to have come from the same individual. Note that use of this parameter only affects labelling of matches and mismatches as EXPECTED or UNEXPECTED. It has no affect on how data is grouped for crosschecking.", optional=true) public File SAMPLE_INDIVIDUAL_MAP
      • CROSSCHECK_MODE

        @Argument(doc="An argument that controls how crosschecking with both INPUT and SECOND_INPUT should occur. ") public picard.fingerprint.Fingerprint.CrosscheckMode CROSSCHECK_MODE
      • OUTPUT

        @Argument(shortName="O", optional=true, doc="Optional output file to write metrics to. Default is to write to stdout.") public File OUTPUT
      • MATRIX_OUTPUT

        @Argument(shortName="MO", optional=true, doc="Optional output file to write matrix of LOD scores to. This is less informative than the metrics output and only contains Normal-Normal LOD score (i.e. doesn\'t account for Loss of Heterozygosity). It is however sometimes easier to use visually.", mutex="SECOND_INPUT") public File MATRIX_OUTPUT
      • HAPLOTYPE_MAP

        @Argument(shortName="H", doc="The file lists a set of SNPs, optionally arranged in high-LD blocks, to be used for fingerprinting. See https://software.broadinstitute.org/gatk/documentation/article?id=9526 for details.") public File HAPLOTYPE_MAP
      • LOD_THRESHOLD

        @Argument(shortName="LOD", doc="If any two groups (with the same sample name) match with a LOD score lower than the threshold the tool will exit with a non-zero code to indicate error. Program will also exit with an error if it finds two groups with different sample name that match with a LOD score greater than -LOD_THRESHOLD.\n\nLOD score 0 means equal likelihood that the groups match vs. come from different individuals, negative LOD score -N, mean 10^N time more likely that the groups are from different individuals, and +N means 10^N times more likely that the groups are from the same individual. ") public double LOD_THRESHOLD
      • CROSSCHECK_BY

        @Argument(doc="Specifies which data-type should be used as the basic comparison unit. Fingerprints from readgroups can be \"rolled-up\" to the LIBRARY, SAMPLE, or FILE level before being compared. Fingerprints from VCF can be be compared by SAMPLE or FILE.") public CrosscheckMetric.DataType CROSSCHECK_BY
      • NUM_THREADS

        @Argument(doc="The number of threads to use to process files and generate fingerprints.") public int NUM_THREADS
      • CALCULATE_TUMOR_AWARE_RESULTS

        @Argument(doc="Specifies whether the Tumor-aware result should be calculated. These are time consuming and can roughly double the runtime of the tool. When crosschecking many groups not calculating the tumor-aware results can result in a significant speedup.") public boolean CALCULATE_TUMOR_AWARE_RESULTS
      • ALLOW_DUPLICATE_READS

        @Argument(doc="Allow the use of duplicate reads in performing the comparison. Can be useful when duplicate marking has been overly aggressive and coverage is low.") public boolean ALLOW_DUPLICATE_READS
      • GENOTYPING_ERROR_RATE

        @Argument(doc="Assumed genotyping error rate that provides a floor on the probability that a genotype comes from the expected sample. Must be greater than zero. ") public double GENOTYPING_ERROR_RATE
      • OUTPUT_ERRORS_ONLY

        @Argument(doc="If true, then only groups that do not relate to each other as expected will have their LODs reported.") public boolean OUTPUT_ERRORS_ONLY
      • LOSS_OF_HET_RATE

        @Argument(doc="The rate at which a heterozygous genotype in a normal sample turns into a homozygous (via loss of heterozygosity) in the tumor (model assumes independent events, so this needs to be larger than reality).", optional=true) public double LOSS_OF_HET_RATE
      • EXPECT_ALL_GROUPS_TO_MATCH

        @Argument(doc="Expect all groups\' fingerprints to match, irrespective of their sample names. By default (with this value set to false), groups (readgroups, libraries, files, or samples) with different sample names are expected to mismatch, and those with the same sample name are expected to match. ") public boolean EXPECT_ALL_GROUPS_TO_MATCH
      • EXIT_CODE_WHEN_MISMATCH

        @Argument(doc="When one or more mismatches between groups is detected, exit with this value instead of 0.") public int EXIT_CODE_WHEN_MISMATCH
      • EXIT_CODE_WHEN_NO_VALID_CHECKS

        @Argument(doc="When all LOD scores are zero, exit with this value.") public int EXIT_CODE_WHEN_NO_VALID_CHECKS
      • MAX_EFFECT_OF_EACH_HAPLOTYPE_BLOCK

        @Argument(doc="Maximal effect of any single haplotype block on outcome (-log10 of maximal likelihood difference between the different values for the three possible genotypes).", minValue=0.0) public double MAX_EFFECT_OF_EACH_HAPLOTYPE_BLOCK
      • TEST_INPUT_READABILITY

        @Hidden @Argument(doc="When true code will check for readability on input files (this can be slow on cloud access)") public boolean TEST_INPUT_READABILITY
    • Constructor Details

      • CrosscheckFingerprints

        public CrosscheckFingerprints()
    • Method Details

      • customCommandLineValidation

        protected String[] customCommandLineValidation()
        Description copied from class: CommandLineProgram
        Put any custom command-line validation in an override of this method. clp is initialized at this point and can be used to print usage and access argv. Any options set by command-line parser can be validated.
        Overrides:
        customCommandLineValidation in class CommandLineProgram
        Returns:
        null if command line is valid. If command line is invalid, returns an array of error message to be written to the appropriate place.
      • doWork

        protected int doWork()
        Description copied from class: CommandLineProgram
        Do the work after command line has been parsed. RuntimeException may be thrown by this method, and are reported appropriately.
        Specified by:
        doWork in class CommandLineProgram
        Returns:
        program exit status.