I am balancing the training set by for instance using SMOTE, i.e.,
"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P 100.0 -S 1" -S 1 -W
weka.classifiers.trees.J48 -- -C 0.25 -M 2
Since sampling requires a random selection of data in the training set, then
it is a common procedure to repeat the (balanced) classification multiple
times (e.g., 100 times). Is there a way to do it in the GUI?
Unfortunately, SMOTE does not implement the Randomizable interface, otherwise you could just use RandomCommittee with the FilteredClassifier as the base learner. If a filter (or the base classifier) is Randomizable, FilteredClassifier will automatically modify the seed of the random number generator in the base scheme.
The alternative is to shuffle the data before it is passed to SMOTE. One possibility is something like this: