Weight every attribute using Java API for WEKA

classic Classic list List threaded Threaded
9 messages Options
JC
Reply | Threaded
Open this post in threaded view
|

Weight every attribute using Java API for WEKA

JC
Hi everyone!

I am continuing the development of a powerfull project to try to know the
weight of an attribute in the development of a disease.

For example, I have 10 attributes taken from a pacient. I would like to
check the prediction giving differents weights to the attribute, for
example:

- Attribute 1: 0.5
- Attribute 2: 0.1
- Attribute 3: 0.3
- Attribute 4: 0.2
- Attribute 5: 0.2
- Attribute 6: 0.1
- Attribute 7: 0.3
- Attribute 8: 0.1

And then change the weight of every attribute in order:

- Attribute 1: 0.1
- Attribute 2: 0.5
- Attribute 3: 0.1
- Attribute 4: 0.3
- Attribute 5: 0.2
- Attribute 6: 0.2
- Attribute 7: 0.1
- Attribute 8: 0.3

And in the final just get *the best predicction*.

/So my question is/, is there a way to do this in WEKA? I know that in
others workstations for Machine Learning are something similar to this but I
want to work with WEKA because I like it and has potential.

Thank you for read me and try to help me!





--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

Eibe Frank-2
Administrator

AFAIK, currently, the only classifier in WEKA for which changing the attribute weights at prediction time will have any effect is NaiveBayes (and its updateable version).

 

NaiveBayes implements WeightedAttributeHandler and you can see in its distributionForInstance(Instance) method that the attribute weights are used to adjust the probability estimates that are computed.

 

Cheers,

Eibe

 

From: [hidden email]
Sent: Monday, 1 July 2019 9:52 PM
To: [hidden email]
Subject: [Wekalist] Weight every attribute using Java API for WEKA

 

Hi everyone!

 

I am continuing the development of a powerfull project to try to know the

weight of an attribute in the development of a disease.

 

For example, I have 10 attributes taken from a pacient. I would like to

check the prediction giving differents weights to the attribute, for

example:

 

- Attribute 1: 0.5

- Attribute 2: 0.1

- Attribute 3: 0.3

- Attribute 4: 0.2

- Attribute 5: 0.2

- Attribute 6: 0.1

- Attribute 7: 0.3

- Attribute 8: 0.1

 

And then change the weight of every attribute in order:

 

- Attribute 1: 0.1

- Attribute 2: 0.5

- Attribute 3: 0.1

- Attribute 4: 0.3

- Attribute 5: 0.2

- Attribute 6: 0.2

- Attribute 7: 0.1

- Attribute 8: 0.3

 

And in the final just get *the best predicction*.

 

/So my question is/, is there a way to do this in WEKA? I know that in

others workstations for Machine Learning are something similar to this but I

want to work with WEKA because I like it and has potential.

 

Thank you for read me and try to help me!

 

 

 

 

 

--

Sent from: http://weka.8497.n7.nabble.com/

_______________________________________________

Wekalist mailing list

Send posts to: [hidden email]

To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist

List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

JC
Hi Eibe.

I am working with WEKA using his Java API. Is there a method to give a
weight for every attribute using Naive Bayes?

My objetive is take the best prediction giving weights to every attribute
one by one (for example 5) and know what attribute is more important in the
final prediction.

/For example/:

- Attribute 1: 0.5

- Attribute 2: 0.1

- Attribute 3: 0.1

/Then/:

- Attribute 1: 0.1

- Attribute 2: 0.5

- Attribute 3: 0.1

Regards.



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

Eibe Frank-2
Administrator

Yes, the distributionForInstance() method in NaiveBayes adjusts the probability estimates based on the attribute weights:

 

temp = Math.max(1e-75, Math.pow(m_Distributions[attIndex][j]

            .getProbability(instance.value(attribute)),

            m_Instances.attribute(attIndex).weight()));

 

Cheers,

Eibe

 

From: [hidden email]
Sent: Monday, 22 July 2019 11:27 PM
To: [hidden email]
Subject: Re: [Wekalist] Weight every attribute using Java API for WEKA

 

Hi Eibe.

 

I am working with WEKA using his Java API. Is there a method to give a

weight for every attribute using Naive Bayes?

 

My objetive is take the best prediction giving weights to every attribute

one by one (for example 5) and know what attribute is more important in the

final prediction.

 

/For example/:

 

- Attribute 1: 0.5

 

- Attribute 2: 0.1

 

- Attribute 3: 0.1

 

/Then/:

 

- Attribute 1: 0.1

 

- Attribute 2: 0.5

 

- Attribute 3: 0.1

 

Regards.

 

 

 

--

Sent from: http://weka.8497.n7.nabble.com/

_______________________________________________

Wekalist mailing list

Send posts to: [hidden email]

To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist

List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

JC
Thank you Eibe!

So with this... Can I evaluate an instance with differents weights and get
what attribute has more importance in the final result?

*Temp* what type is?

And a final question, can I get differents results like /*Minimum Mean
Square Error*/ in a variable?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

Eibe Frank-3
Yes, you could change the weights, rebuild the NaiveBayes model using the Instances object with the new weights settings (so that m_Instances will have the new weights), reclassify your validation set based on the new weights using a WEKA Evaluation object, and obtain the RMSE from the Evaluation object.

You could write a Java program, a Groovy program, or a Jython program containing a loop iterating over the different weight vectors that you want to evaluate. The latter two programming options are available in consoles that can be installed using the kfGroovy and tigerJython packages respectively.

The temp variable is of type double. Note that WEKA is open-source (the source code is in weka-src.jar and is also available on GitHub or via our SVN server).

Cheers,
Eibe

On Tue, Aug 6, 2019 at 10:21 PM JC <[hidden email]> wrote:
Thank you Eibe!

So with this... Can I evaluate an instance with differents weights and get
what attribute has more importance in the final result?

*Temp* what type is?

And a final question, can I get differents results like /*Minimum Mean
Square Error*/ in a variable?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

JC
Thank you Eibe for your help again!

Is there any example to make this? sounds good but don't know well how to
use this code



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

JC
By the way, this is my actually code:

public static void trainRandomCommittee(Instances trainingSet, String
patharff) throws Exception {

                trainingSet.setClassIndex(trainingSet.numAttributes() - 1);
                System.out.println("TrainingSet contiene este nº de instancias: " +
trainingSet.numInstances());
                BufferedReader breader = new BufferedReader(new FileReader(patharff));
                Instances originalTrain = null;
                originalTrain = new Instances(breader);

                originalTrain.setClassIndex(originalTrain.numAttributes() - 1);

                // originalTrain.set
                System.out.println("Muestro lo de numAttributes para ver qué es:" +
originalTrain.attribute(13));

                Classifier cls = new NaiveBayes();

                // CHANGE WEIGHTS
                System.out.println("-*-*-*-*-*-*-**-*-*-* Se modifican los pesos ");
                double factor = 0.5 / (double) trainingSet.numInstances();
                int iter = 0; // iterador para avanzar el atributo sobre el q modificar el
peso

                        for (int i = 0; i < trainingSet.numInstances(); i++) {
                                trainingSet.instance(i).attribute(1 + iter).setWeight(10 + factor * i);
// 10.5
                        }

                cls.buildClassifier(trainingSet);

                Evaluation eval = new Evaluation(trainingSet);

                eval.evaluateModel(cls, trainingSet);
                /** Print the algorithm summary */
                System.out.println("** Naive Bayes Evaluation with Datasets **");
                System.out.println(eval.toSummaryString());
                System.out.print(" the expression for the input data as per alogorithm is
");
                System.out.println(cls);
                for (int i = 0; i < originalTrain.numInstances(); i++) {
                        System.out.println("Instance prueba " + "s" + i + ": " +
originalTrain.instance(i));
                        double index = cls.classifyInstance(originalTrain.instance(i));
                }

                int s1 = 0;
                double value = cls.classifyInstance(originalTrain.instance(s1));

                // get the prediction percentage or distribution
                double[] percentage =
cls.distributionForInstance(originalTrain.instance(s1));
                System.out.println("Percentage: " + percentage);


                System.out.println("The predicted value of instance " +
Integer.toString(s1) + ": " + prediction);
                // Format the distribution
                String distribution = "";
                for (int i = 0; i < percentage.length; i = i + 1) {
                        if (i == value) {
                                distribution = distribution + "*" + Double.toString(percentage[i]) +
",";
                        } else {
                                distribution = distribution + Double.toString(percentage[i]) + ",";
                        }

                }

                // System.out.println("Distribution: " + distribution);

                String finalpred = "The predicted value of instance " +
Integer.toString(s1) + ": " + prediction + "\n"
                                + "Distribution:" + finalDistribution;
                originalTrain.instance(s1).setClassValue(value);

               
                System.out.println("El error absoluto medio es: " +
eval.meanAbsoluteError());
                System.out.println("El root mean squared error es: " +
eval.rootMeanSquaredError());
                System.out.println("Instancias no clasificadas: " + eval.unclassified());
                System.out.println("Kappa: " + eval.kappa());
        }



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weight every attribute using Java API for WEKA

Eibe Frank-2
Administrator
The bit of code that sets the weights seems strange:

        int iter = 0; // iterador para avanzar el atributo sobre el q modificar el peso

                        for (int i = 0; i < trainingSet.numInstances(); i++) {
                                trainingSet.instance(i).attribute(1 + iter).setWeight(10 + factor * i);
// 10.5
                        }


I would have expected something like this:

double[] attributeWeights = **array holding the weight of each attribute**;
for (int i = 0; i trainingSet.numAttributes(): i++) {
  trainingSet.attribute(i).setWeight(attributeWeights[i]);
}

Note that setWeight() will change the weight of the attribute in all Instances objects that the attribute is associated with.

Cheers,
Eibe

> On 12/08/2019, at 10:35 PM, JC <[hidden email]> wrote:
>
> By the way, this is my actually code:
>
> public static void trainRandomCommittee(Instances trainingSet, String
> patharff) throws Exception {
>
> trainingSet.setClassIndex(trainingSet.numAttributes() - 1);
> System.out.println("TrainingSet contiene este nº de instancias: " +
> trainingSet.numInstances());
> BufferedReader breader = new BufferedReader(new FileReader(patharff));
> Instances originalTrain = null;
> originalTrain = new Instances(breader);
>
> originalTrain.setClassIndex(originalTrain.numAttributes() - 1);
>
> // originalTrain.set
> System.out.println("Muestro lo de numAttributes para ver qué es:" +
> originalTrain.attribute(13));
>
> Classifier cls = new NaiveBayes();
>
> // CHANGE WEIGHTS
> System.out.println("-*-*-*-*-*-*-**-*-*-* Se modifican los pesos ");
> double factor = 0.5 / (double) trainingSet.numInstances();
> int iter = 0; // iterador para avanzar el atributo sobre el q modificar el
> peso
>
> for (int i = 0; i < trainingSet.numInstances(); i++) {
> trainingSet.instance(i).attribute(1 + iter).setWeight(10 + factor * i);
> // 10.5
> }
>
> cls.buildClassifier(trainingSet);
>
> Evaluation eval = new Evaluation(trainingSet);
>
> eval.evaluateModel(cls, trainingSet);
> /** Print the algorithm summary */
> System.out.println("** Naive Bayes Evaluation with Datasets **");
> System.out.println(eval.toSummaryString());
> System.out.print(" the expression for the input data as per alogorithm is
> ");
> System.out.println(cls);
> for (int i = 0; i < originalTrain.numInstances(); i++) {
> System.out.println("Instance prueba " + "s" + i + ": " +
> originalTrain.instance(i));
> double index = cls.classifyInstance(originalTrain.instance(i));
> }
>
> int s1 = 0;
> double value = cls.classifyInstance(originalTrain.instance(s1));
>
> // get the prediction percentage or distribution
> double[] percentage =
> cls.distributionForInstance(originalTrain.instance(s1));
> System.out.println("Percentage: " + percentage);
>
>
> System.out.println("The predicted value of instance " +
> Integer.toString(s1) + ": " + prediction);
> // Format the distribution
> String distribution = "";
> for (int i = 0; i < percentage.length; i = i + 1) {
> if (i == value) {
> distribution = distribution + "*" + Double.toString(percentage[i]) +
> ",";
> } else {
> distribution = distribution + Double.toString(percentage[i]) + ",";
> }
>
> }
>
> // System.out.println("Distribution: " + distribution);
>
> String finalpred = "The predicted value of instance " +
> Integer.toString(s1) + ": " + prediction + "\n"
> + "Distribution:" + finalDistribution;
> originalTrain.instance(s1).setClassValue(value);
>
>
> System.out.println("El error absoluto medio es: " +
> eval.meanAbsoluteError());
> System.out.println("El root mean squared error es: " +
> eval.rootMeanSquaredError());
> System.out.println("Instancias no clasificadas: " + eval.unclassified());
> System.out.println("Kappa: " + eval.kappa());
> }
>
>
>
> --
> Sent from: https://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html