I have dataset with in 229 instances and 20 attributes the class has 4 values (Normal, early, definite, severe) . the distribution of instances are (Normal: 116 instance, early: 93 instance , definite: 13 instance , severe: 4 instance) , my question is do I need to re sample the dataset or its normal ?
Looks like J48 is doing really well: it classifies all instances of the smallest class correctly, and most of the others as well. Perhaps you should also try RandomForest without OrdinalClassClassifier.
Whether the result is good enough really depends on your application. Is this a medical classification problem? In that case, you probably want to introduce different misclassification costs to penalise some errors more than others. For example, classifying an “early” case as “normal” is probably much worse than classifying a “normal” case as “early”.
You can use misclassification costs by applying the CostSensitiveClassifier (perhaps configuring it to minimise expected cost rather than using it in default mode). You will need to specify an appropriate cost matrix when you apply it.
RandomForest applies WEKA’s Bagging with default parameters to build a random forest with RandomTree. Perhaps your smallest class is too small to be processed effectively by Bagging. You could also try building a random forest without bagging (i.e., subsampling) by using RandomCommittee with RandomTree as the base learner.