FixedDictionaryStringToWordVector usage?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

FixedDictionaryStringToWordVector usage?

mcbenly
Hi,
So I am trying to use *FixedDictionaryStringToWordVector *filter for my
pre-built dictionary. When I run with this below configuration, I can see
all the words. But don't see distribution of each of the word from dataset.

weka.filters.unsupervised.attribute.FixedDictionaryStringToWordVector -I
false -T false -R first-last -dictionary
C:\Users\irobot2\Documents\riz_words.txt -C -stemmer
weka.core.stemmers.NullStemmer -stopwords-handler "weka.core.stopwords.Null
" -tokenizer "weka.core.tokenizers.WordTokenizer -delimiters \"
\\r\\n\\t.,;:\\\'\\\"()?!\""


How can I use this filter as attribute selector from my dictionary to train
a classifier?


Thanks, Ben




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: FixedDictionaryStringToWordVector usage?

Eibe Frank-3
The configuration looks fine to me. What exactly is the problem? For each input document, from each row of the input dataset, the filter will produce a vector (i.e., a row in the new dataset produced by the filter) that contains the frequency of each of the words from the dictionary in the document. It will output the frequency because you have set the -C flag.

Cheers,
Eibe

On Thu, Sep 5, 2019 at 4:36 AM mcbenly <[hidden email]> wrote:
Hi,
So I am trying to use *FixedDictionaryStringToWordVector *filter for my
pre-built dictionary. When I run with this below configuration, I can see
all the words. But don't see distribution of each of the word from dataset.

weka.filters.unsupervised.attribute.FixedDictionaryStringToWordVector -I
false -T false -R first-last -dictionary
C:\Users\irobot2\Documents\riz_words.txt -C -stemmer
weka.core.stemmers.NullStemmer -stopwords-handler "weka.core.stopwords.Null
" -tokenizer "weka.core.tokenizers.WordTokenizer -delimiters \"
\\r\\n\\t.,;:\\\'\\\"()?!\""


How can I use this filter as attribute selector from my dictionary to train
a classifier?


Thanks, Ben




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: FixedDictionaryStringToWordVector usage?

mcbenly
Thanks Eibe, apparently this configuration worked fine in filteredclass. But
wasn't producing any result in preprocess tab.

Thanks a lot.


Ben



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html