Output prediction from Experimenter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Output prediction from Experimenter

Saleh Shahinfar-2

Dear Eibe, 


I wonder if there is a way that can output prediction( actual, predicted, error) from Experimenter? similar to what we can get from Explorer?


Sincerely, 

saleh


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Output prediction from Experimenter

Mark Hall
I'm afraid that instance-level predictions are not available for output from the Experimenter.

Cheers,
Mark.

On 8/05/17, 6:03 PM, "Saleh Shahinfar" <[hidden email] on behalf of [hidden email]> wrote:

    Dear Eibe,
   
   
   
    I wonder if there is a way that can output prediction( actual, predicted, error) from Experimenter? similar to what we can get from Explorer?
   
   
   
    Sincerely,
   
    saleh
   
   
   
    _______________________________________________
    Wekalist mailing list
    Send posts to: [hidden email]
    List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
    List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
   


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Output prediction from Experimenter

Saleh Shahinfar-2
Thanks Mark!

Is there any hope that in near future we can have this option in Experimenter?

Sincerely,
Saleh
________________________________________
From: [hidden email] <[hidden email]> on behalf of Mark Hall <[hidden email]>
Sent: Tuesday, May 9, 2017 7:07 AM
To: Weka machine learning workbench list.
Subject: Re: [Wekalist] Output prediction from Experimenter

I'm afraid that instance-level predictions are not available for output from the Experimenter.

Cheers,
Mark.

On 8/05/17, 6:03 PM, "Saleh Shahinfar" <[hidden email] on behalf of [hidden email]> wrote:

    Dear Eibe,



    I wonder if there is a way that can output prediction( actual, predicted, error) from Experimenter? similar to what we can get from Explorer?



    Sincerely,

    saleh



    _______________________________________________
    Wekalist mailing list
    Send posts to: [hidden email]
    List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
    List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html



_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Output prediction from Experimenter

Eibe Frank-2
Administrator
If you are interested in classifications (and not class probability estimates), there is already a way to get the predictions and ground truth in the Experimenter. You need to switch to the advanced mode of the Experimenter and configure the ClassifierSplitEvaluator that’s used inside the result producer to output predictions and actual targets. The ClassifierSplitEvaluator has a “predTargetColumn” parameter, which is “False” by default. Set it to true and you’ll get two additional columns in the results file: “Targets” and “Predictions”.

The individual class  values (predicted and actual ones) are separated by “|” characters.

Unfortunately, the RegressionSplitEvaluator, etc., don’t have that option. Also, it would be nice to also output class probability estimates with the ClassifierSplitEvaluator.

Another option may be to run your experiments through the OpenML Experimenter interface for WEKA. I may be wrong, but I think OpenML stores all the predictions.

Cheers,
Eibe

An example instance from some results for the kr-vs-kp data, which has class values “won” and “nowin":

kr-vs-kp,1,1,weka.classifiers.rules.ZeroR,'',48055541465867954,20170508.2303,2876,320,167,153,0,52.1875,47.8125,0,0,0.499027,0.499521,100,100,319.558155,319.558155,0,0.998619,0.998619,0,0,0,0,1,167,1,153,0,0,0,0,0.521875,1,0.685832,0,0.5,0.521875,0.521875,0.521875,0.478125,0.478125,0.272354,0.521875,0.357918,0,0.5,0.500957,0.342916,0.521875,0.009,0.01,0.002202,0.002551,2.202,2.551,886,940552,109852,100,100,won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin|nowin,won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won|won,?,0.5,0


> On 9/05/2017, at 10:24 AM, Saleh Shahinfar <[hidden email]> wrote:
>
> Thanks Mark!
>
> Is there any hope that in near future we can have this option in Experimenter?
>
> Sincerely,
> Saleh
> ________________________________________
> From: [hidden email] <[hidden email]> on behalf of Mark Hall <[hidden email]>
> Sent: Tuesday, May 9, 2017 7:07 AM
> To: Weka machine learning workbench list.
> Subject: Re: [Wekalist] Output prediction from Experimenter
>
> I'm afraid that instance-level predictions are not available for output from the Experimenter.
>
> Cheers,
> Mark.
>
> On 8/05/17, 6:03 PM, "Saleh Shahinfar" <[hidden email] on behalf of [hidden email]> wrote:
>
>    Dear Eibe,
>
>
>
>    I wonder if there is a way that can output prediction( actual, predicted, error) from Experimenter? similar to what we can get from Explorer?
>
>
>
>    Sincerely,
>
>    saleh
>
>
>
>    _______________________________________________
>    Wekalist mailing list
>    Send posts to: [hidden email]
>    List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
>    List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
>
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Loading...