Class PairedCorrectedTTester

  • All Implemented Interfaces:
    java.io.Serializable, OptionHandler, RevisionHandler, TechnicalInformationHandler, Tester

    public class PairedCorrectedTTester
    extends PairedTTester
    implements TechnicalInformationHandler
    Behaves the same as PairedTTester, only it uses the corrected resampled t-test statistic.

    For more information see:

    Claude Nadeau, Yoshua Bengio (2001). Inference for the Generalization Error. Machine Learning..

    BibTeX:

     @article{Nadeau2001,
        author = {Claude Nadeau and Yoshua Bengio},
        journal = {Machine Learning},
        title = {Inference for the Generalization Error},
        year = {2001},
        PDF = {http://www.iro.umontreal.ca/\~lisa/bib/pub_subject/comparative/pointeurs/nadeau_MLJ1597.pdf}
     }
     

    Valid options are:

     -D <index,index2-index4,...>
      Specify list of columns that specify a unique
      dataset.
      First and last are valid indexes. (default none)
     -R <index>
      Set the index of the column containing the run number
     -F <index>
      Set the index of the column containing the fold number
     -G <index1,index2-index4,...>
      Specify list of columns that specify a unique
      'result generator' (eg: classifier name and options).
      First and last are valid indexes. (default none)
     -S <significance level>
      Set the significance level for comparisons (default 0.05)
     -V
      Show standard deviations
     -L
      Produce table comparisons in Latex table format
     -csv
      Produce table comparisons in CSV table format
     -html
      Produce table comparisons in HTML table format
     -significance
      Produce table comparisons with only the significance values
     -gnuplot
      Produce table comparisons output suitable for GNUPlot
    Version:
    $Revision: 1.13 $
    Author:
    Richard Kirkby (rkirkby@cs.waikato.ac.nz)
    See Also:
    Serialized Form
    • Constructor Detail

      • PairedCorrectedTTester

        public PairedCorrectedTTester()
    • Method Detail

      • getTechnicalInformation

        public TechnicalInformation getTechnicalInformation()
        Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
        Specified by:
        getTechnicalInformation in interface TechnicalInformationHandler
        Returns:
        the technical information about this class
      • calculateStatistics

        public PairedStats calculateStatistics​(Instance datasetSpecifier,
                                               int resultset1Index,
                                               int resultset2Index,
                                               int comparisonColumn)
                                        throws java.lang.Exception
        Computes a paired t-test comparison for a specified dataset between two resultsets.
        Specified by:
        calculateStatistics in interface Tester
        Overrides:
        calculateStatistics in class PairedTTester
        Parameters:
        datasetSpecifier - the dataset specifier
        resultset1Index - the index of the first resultset
        resultset2Index - the index of the second resultset
        comparisonColumn - the column containing values to compare
        Returns:
        the results of the paired comparison
        Throws:
        java.lang.Exception - if an error occurs
      • main

        public static void main​(java.lang.String[] args)
        Test the class from the command line.
        Parameters:
        args - contains options for the instance ttests
      • getToolTipText

        public java.lang.String getToolTipText()
        returns a string that is displayed as tooltip on the "perform test" button in the experimenter
        Specified by:
        getToolTipText in interface Tester
        Overrides:
        getToolTipText in class PairedTTester
        Returns:
        the string for the tool tip