Model performance evaluation in Weka(Experimenter)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Model performance evaluation in Weka(Experimenter)

pkc533
Hello,

I am new to machine learning analysis. This is for the first time I am using
Weka_Experimenter for regression model performance comparison.

The result I got is:-
Tester:     weka.experiment.PairedCorrectedTTester -G 3,4,5 -D 1 -R 2 -S
0.05 -V -result-matrix "weka.experiment.ResultMatrixPlainText -mean-prec 2
-stddev-prec 2 -col-name-width 0 -row-name-width 25 -mean-width 2
-stddev-width 2 -sig-width 1 -count-width 5 -show-stddev -print-col-names
-print-row-names -enum-col-names"
Analysing:  Correlation_coefficient
Datasets:   4
Resultsets: 5
Confidence: 0.05 (two tailed)
Sorted by:  -
Date:       6/16/20 9:24 PM


Dataset                   (1) functions.Li | (2) functions (3) functions (4)
trees.Ran (5) meta.Bagg
----------------------------------------------------------------------------------------------------
data1 (1)   0.81(NaN) |   0.84(NaN) v   0.75(NaN) *   0.84(NaN) v  
0.82(NaN) v
Data2  (1)   0.93(NaN) |   0.67(NaN) *   0.60(NaN) *   0.90(NaN) *  
0.87(NaN) *
Data3  (1)   0.89(NaN) |   0.87(NaN) *   0.81(NaN) *   0.87(NaN) *  
0.86(NaN) *
Data4  (1)   0.93(NaN) |   0.94(NaN) v   0.81(NaN) *   0.84(NaN) *  
0.78(NaN) *
----------------------------------------------------------------------------------------------------
                                   (v/ /*) |       (2/0/2)       (0/0/4)      
(1/0/3)       (1/0/3)


Key:
(1) functions.LinearRegression '-S 0 -R 1.0E-8 -num-decimal-places 4'
-3364580862046573747
(2) functions.MultilayerPerceptron '-L 0.3 -M 0.2 -N 500 -V 0 -S 0 -E 20 -H
a' -5990607817048210779
(3) functions.SMOreg '-C 1.0 -N 0 -I
\"functions.supportVector.RegSMOImproved -T 0.001 -V -P 1.0E-12 -L 0.001 -W
1\" -K \"functions.supportVector.RBFKernel -C 250007 -G 0.01\"'
-7149606251113102827
(4) trees.RandomForest '-P 100 -I 1000 -num-slots 1 -K 0 -M 1.0 -V 0.001 -S
1' 1116839470751428698
(5) meta.Bagging '-P 100 -S 1 -num-slots 1 -I 100 -W trees.REPTree -- -M 2
-V 0.001 -N 3 -S 1 -L -1 -I 0.0' -115879962237199703

Could you please explain what are these values:- (v/ /*) |       (2/0/2)      
(0/0/4)       (1/0/3)       (1/0/3) ?
 and What does this (Nan) represents?


Thanks/





--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Model performance evaluation in Weka(Experimenter)

Peter Reutemann
> I am new to machine learning analysis. This is for the first time I am using
> Weka_Experimenter for regression model performance comparison.
>
> The result I got is:-
> Tester:     weka.experiment.PairedCorrectedTTester -G 3,4,5 -D 1 -R 2 -S
> 0.05 -V -result-matrix "weka.experiment.ResultMatrixPlainText -mean-prec 2
> -stddev-prec 2 -col-name-width 0 -row-name-width 25 -mean-width 2
> -stddev-width 2 -sig-width 1 -count-width 5 -show-stddev -print-col-names
> -print-row-names -enum-col-names"
> Analysing:  Correlation_coefficient
> Datasets:   4
> Resultsets: 5
> Confidence: 0.05 (two tailed)
> Sorted by:  -
> Date:       6/16/20 9:24 PM
>
>
> Dataset                   (1) functions.Li | (2) functions (3) functions (4)
> trees.Ran (5) meta.Bagg
> ----------------------------------------------------------------------------------------------------
> data1 (1)   0.81(NaN) |   0.84(NaN) v   0.75(NaN) *   0.84(NaN) v
> 0.82(NaN) v
> Data2  (1)   0.93(NaN) |   0.67(NaN) *   0.60(NaN) *   0.90(NaN) *
> 0.87(NaN) *
> Data3  (1)   0.89(NaN) |   0.87(NaN) *   0.81(NaN) *   0.87(NaN) *
> 0.86(NaN) *
> Data4  (1)   0.93(NaN) |   0.94(NaN) v   0.81(NaN) *   0.84(NaN) *
> 0.78(NaN) *
> ----------------------------------------------------------------------------------------------------
>                                    (v/ /*) |       (2/0/2)       (0/0/4)
> (1/0/3)       (1/0/3)

[...]

> Could you please explain what are these values:- (v/ /*) |       (2/0/2)
> (0/0/4)       (1/0/3)       (1/0/3) ?
>  and What does this (Nan) represents?

With the default settings of the Experimenter, you'd perform 10 runs
of 10-fold cross-validation. This will give you a "(100)" after the
dataset name, as there are 100 results for a classifier/dataset
combination.
Your output above, for some reason, lists "(1)". That looks like you
only have one data point per dataset/classifier combination. You
cannot compute a standard deviation from one data points, hence the
"NaN" in the parentheses.

If you want to know what " (v/ /*)" means, then please read the
section on the Experimenter in the Weka manual that came with your
Weka installation.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html