Using PairedCorrectedTTester from the command line

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Using PairedCorrectedTTester from the command line

annettes
I am running machine learning algorithms for survival analysis, so can't use the Weka GUI, but I would like to use the Weka PairedCorrectedTTester to analyse my results if possible.

I followed the advice in this post: https://weka.8497.n7.nabble.com/T-test-td41515.html

I have the following results from a 5 x 5-fold cross-validation run of two different algorithms. The numbers are the concordance index, which is equivalent in behaviour to the ROC:
Algorithm 1         Algorithm 2
0.507760532       0.77383592
0.636890391       0.827136649
0.580253432       0.779831045
0.671664943       0.715098242
0.699468085       0.842021277
0.625181598       0.827118644
0.764672365       0.838176638
0.583056478       0.633444075
0.542886492       0.783769902
0.550796813       0.892430279
0.678899083       0.850662589
0.613187954       0.818795431
0.524119241       0.76802168
0.627700831       0.789473684
0.679423459       0.80666004
0.638481013       0.814177215
0.625249501       0.756487026
0.626065163       0.788972431
0.607303371       0.769101124
0.592332613       0.75161987
0.675237563       0.85410844
0.595964822       0.806001035
0.546071775       0.803588749
0.599462366       0.796236559
0.50129199         0.696124031

I ran this code:
cat corr_ttest.csv | java -cp /usr/share/java/weka.jar weka.experiment.PairedStatsCorrected -V -significance

And I got the following results:

Analysis for 25 points:
                         Column 1         Column 2       Difference
Minimums                   0.5013           0.6334          -0.3416
Maximums                   0.7647           0.8924          -0.0434
Sums                            15.2934          19.7829          -4.4895
SumSquares                 9.4516          15.7256           0.9114
Means                          0.6117           0.7913          -0.1796
SDs                               0.0633           0.0544           0.0662
Prob(differences)      0 (sigflag -1)
Correlation                 0.3747

But how do I interpret this output? I am looking for a p-value that will tell me whether the means of the two sets of results are statistically different.
Thanks.
Annette


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html