ROC Curves Comparision

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ROC Curves Comparision

Praveen Boinee
Dear WEKA People

1) How to plot multiple ROC 's on single plot ... I see nice graphics
roc plot in WEKA for individual classifiers
is thery any way to put all the ROC's in one plot

2) I want to compare MLP with SVM and random forests
   my test set has 6400 rows ...i am using the explorer in WEKA
I can see the False positive rate[FPR]  and true pos.rate[TPR] for all
6400 rows only in MLP !! With SVM there  only 2 rows  of FPR and TPR and
with random forests only 30 rows  of FPR and TPR  ..

       is it common to get like this ..!!

_______________________________________________
Wekalist mailing list
[hidden email]
https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
Reply | Threaded
Open this post in threaded view
|

Re: ROC Curves Comparision

Mark Hall-11
It is possible to get multiple ROC curves on a single plot by using
the KnowledgeFlow. Set up a flow as follows:

ArffLoader ---dataSet---> ClassAssigner ---dataSet--->
ClassValuePicker ---dataSet---> CrossValidationFoldMaker ---
trainingSet/testSet (i.e. BOTH connections) ---> Classifier of your
choice --- batchClassifier ---> ClassifierPerformanceEvaluator --->
ModelPerformanceChart

Next configure the ArffLoader with a data set, and then configure the
ClassValuePicker by selecting which class value you want to treat as
the positive class. Then you can start the flow running by selecting
the "start loading" action from the ArffLoader. The model performance
chart will show the ROC curve.

Subsequent ROC curves for different algorithms can be displayed on the
same plot by either deleting your first classifier from the existing
flow and inserting a new one and then running the flow again, or,
setting up a new flow identical to the first one (but with a different
classifier) and connecting this one's  ClassifierPerformanceEvaluator
to the first flow's ModelPerformanceChart.

For subsequent curves to appear on the same plot the dataset, class
attribute, positive class value and evaluation method (eg 10 fold
Xval) must be the same as for the first curve.

Hope this helps.

Cheers,
Mark.




>
> Dear WEKA People
>
> 1) How to plot multiple ROC 's on single plot ... I see nice graphics
> roc plot in WEKA for individual classifiers
> is thery any way to put all the ROC's in one plot
>
> 2) I want to compare MLP with SVM and random forests
>     my test set has 6400 rows ...i am using the explorer in WEKA
> I can see the False positive rate[FPR]  and true pos.rate[TPR] for all
> 6400 rows only in MLP !! With SVM there  only 2 rows  of FPR and TPR and
> with random forests only 30 rows  of FPR and TPR  ..
>
>         is it common to get like this ..!!
>

_______________________________________________
Wekalist mailing list
[hidden email]
https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist