tag:blogger.com,1999:blog-36589303.post6652486140098698861..comments2024-01-22T09:48:10.802+01:00Comments on Nihil Obstat: Text Mining in WEKA Revisited: Selecting Attributes by Chaining FiltersJose Maria Gomez Hidalgohttp://www.blogger.com/profile/17053588779560658723noreply@blogger.comBlogger29125tag:blogger.com,1999:blog-36589303.post-30086466558780327652016-11-23T08:51:49.454+01:002016-11-23T08:51:49.454+01:00Very informative posts to get one started with Wek...Very informative posts to get one started with Weka based ML. I had a very basic question - Assume that I have multiple TEXT attributes (bug report, summary , description, related conversation etc) , and the class can have multiple values (instead of a yes/no or spam/ not a spam) .. <br />let's say my ARFF looks like - <br /><br />@attribute Summary string<br />@attribute Desc string<br />@attribute @@PROBLEM-CLASS@@ {Severe,Moderate,Regular,NoFixNeeded}<br /><br />@data<br /><br />...<br />...<br /><br />Now the questions - <br />Can I handle such cases in Weka.<br />I had been trying with taking one String attribute at a time and apply StringToWord (with TF-IDF) transformation followed by SMO (had to do SMOTE on the class as there were too few SEVERE records in the sample). My confusion comes from the fact that the class attribute is not 1/0 type.. there are multiple values.<br />If you could kindly guide me with this.<br /><br />Regards,<br /><br /> Rahulnoreply@blogger.comtag:blogger.com,1999:blog-36589303.post-67576382834695971832015-10-28T12:15:52.466+01:002015-10-28T12:15:52.466+01:00Hi Jose!
Usually in literature, LSA is a method re...Hi Jose!<br />Usually in literature, LSA is a method related to unstructured data. I used LSA as an attribute selector for my structured data, increasing the accuracy of the classifier. It's correct to use it to structured data? Thanks!Anonymoushttps://www.blogger.com/profile/13133321656481008619noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-21117773565869506612015-04-30T08:47:30.231+02:002015-04-30T08:47:30.231+02:00Hi, Abhishek
I am currently very busy and not abl...Hi, Abhishek<br /><br />I am currently very busy and not able to perform self-supported research, and very limited freelancing. I am afraid I cannot help you, sorry for that.<br /><br />Good luckJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-59840455693612196262015-04-29T22:50:43.174+02:002015-04-29T22:50:43.174+02:00Hello! I found your post very useful and informati...Hello! I found your post very useful and informative. I have a .arff file and by using the same approach as yours, I have achieved an efficiency of 45% on a 10 fold cross validation using rule based PART classifier. Is there a way where I could send you my results? Would it be possible on your part to give me suggestions to improve the efficiency to about 85%?<br /><br />ThanksAnonymoushttps://www.blogger.com/profile/09372659073848518032noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-23561526077882772052015-04-29T22:50:42.864+02:002015-04-29T22:50:42.864+02:00Hello! I found your post very useful and informati...Hello! I found your post very useful and informative. I have a .arff file and by using the same approach as yours, I have achieved an efficiency of 45% on a 10 fold cross validation using rule based PART classifier. Is there a way where I could send you my results? Would it be possible on your part to give me suggestions to improve the efficiency to about 85%?<br /><br />ThanksAnonymoushttps://www.blogger.com/profile/09372659073848518032noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-18201800914550522882015-01-09T11:48:16.971+01:002015-01-09T11:48:16.971+01:00With reference to my previous message, I forgot to...With reference to my previous message, I forgot to mention that I had the following instance variables defined in addition to those already defined in the MyFilteredLearner class: <br /><br />MultiFilter multiFilter;<br />Filter[] filters;<br />StringToWordVector filter1;<br />AttributeSelection filter2;<br /><br />Thanks<br />Ranahttps://www.blogger.com/profile/13151774099237935292noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-14085598529453465472015-01-09T11:39:33.297+01:002015-01-09T11:39:33.297+01:00Dear Jose,
Inspired by your post :), I have used ...Dear Jose,<br /><br />Inspired by your post :), I have used it to updat your MyFilteredLearner.java such that a MultiFilter is used to chain the STWV and the Attribute selection filters as shown below, but I’m having a null pointer exception at the statement: filters[0] = filter1;<br /><br />Here is the code in the try block of evaluate() method:<br /> trainData.setClassIndex(0);<br />multiFilter = new MultiFilter();<br /> <br />filter1 = new StringToWordVector();<br />filter1.setAttributeIndices("last");<br />filters[0] = filter1;<br /> <br />filter2 = new AttributeSelection();<br />ASEvaluation attEvaluator = new InfoGainAttributeEval();<br /> filter2.setEvaluator(attEvaluator);<br />Ranker ranker = new Ranker();<br />ranker.setThreshold(0); //<0 ignored<br />ASSearch asSearch = ranker;<br /> filter2.setSearch(asSearch);<br />filters[1] = filter2; <br /> <br />//Apply the chained filters<br />classifier = new FilteredClassifier();<br /> multiFilter.setFilters(filters);<br />classifier.setFilter(multiFilter);<br />classifier.setClassifier(new NaiveBayes());<br /> <br />Evaluation eval = new Evaluation(trainData);<br />eval.crossValidateModel(classifier, trainData, 4, new Random(1));<br /> <br />The rest is the same as yours. <br />Also, do I have to repeat the same setup in the learn() method? especially that applying the evaluation and search is expensive. Can't I just use the same multiFilter defined in evalute()?<br /><br />I would appreciate your soonest reply<br /><br />RegardsRanahttps://www.blogger.com/profile/13151774099237935292noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-87399320378831247282015-01-09T09:17:59.771+01:002015-01-09T09:17:59.771+01:00Hi, Sir_Kay
I guess there is a problem in the scr...Hi, Sir_Kay<br /><br />I guess there is a problem in the script or the config at the Explorer, most likely related with setting up the class attribute. However, unless I see an example or more details, I cannot be sure :(<br /><br />Regards,<br /><br />JMJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-83515023654138570312015-01-09T09:16:02.579+01:002015-01-09T09:16:02.579+01:00Hi, Rana
Well, it depends on your PC and Operatin...Hi, Rana<br /><br />Well, it depends on your PC and Operating System; for datasets as yours, I usually make use of the Explorer on a very small subset (e.g. 100 tweets) in order to set up the config I want (filters, classifiers, etc.) and then I write them in a script (bat or bash) and run the script on the full dataset.<br /><br />There is some guidance on this in the WEKA page: http://www.cs.waikato.ac.nz/ml/weka/bigdata.html<br /><br />Best regards,<br /><br />JMJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-37576975648454712632015-01-08T13:53:44.049+01:002015-01-08T13:53:44.049+01:00Hello,
Thank you for this post.
I'm facing p...Hello,<br /><br />Thank you for this post. <br />I'm facing problems with removing attributes from my sparse arff file. But first I would appreciate your help with a problem in the Explorer when I try using it with my sparse arff file. It hangs if I try to set my class attribute; the 'Edit' tab takes a long time to get the list of attributes; then if I right click the attribute to set it as my class attribute it just hangs and stops responding. <br />Can the Explorer handle a sparse file of around 22800 attributes (mostly unigrams generated out of around 6000 tweets).<br /><br />Thank you in advance<br /><br />RanaRanahttps://www.blogger.com/profile/13151774099237935292noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-82532973557784182532014-11-28T17:48:53.452+01:002014-11-28T17:48:53.452+01:00Dear Jose,
Your discussion is really marvelous. I...Dear Jose,<br /><br />Your discussion is really marvelous. It has helped me in getting better accuracy for my classification task.<br />But I had an error message when I applied the MulfFilter concept. The error is"Problem evaluating classifier:weka.attributeSelection.InfoGainAttributeEval: Cannot handle string attribute<br /><br />Please, kindly help out.<br />ThanksSir_Kayhttps://www.blogger.com/profile/06483758463198275189noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-61997589204033413012014-04-07T22:36:01.380+02:002014-04-07T22:36:01.380+02:00For applying LSA to a train/test splitted dataset ...For applying LSA to a train/test splitted dataset with WEKA, you only need to make use of the <i>LatentSemanticAnalysis</i> evaluator with <i>Ranker</i> search in the <i>AttributeSelection</i> (AS) filter. Something like:<br /><br /><i>java weka.filters.supervised.attribute.AttributeSelection -E "weka.attributeSelection.LatentSemanticAnalysis -R 0.95 -A 5" -S "weka.attributeSelection.Ranker -T -1.7976931348623157E308 -N -1"</i><br /><br />As AS is a filter, you can apply it in batch mode (<i>-b</i> option) to a training & a test subset at once, thus representing the test dataset according to the dimensions defined in the training set.Jose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-31253903888076478392014-04-07T21:23:12.459+02:002014-04-07T21:23:12.459+02:00Hi!
First of all I would like to say that your blo...Hi!<br />First of all I would like to say that your blog posts are amazing! They really helped me resolve some issues for my research. However, now I wish to apply LSA on my dataset which is divided into a train and test set. Could you please give me some directions in how I can apply LSA on both train as test set?<br /><br />thanks in advance!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-36589303.post-40882673559840587102014-04-02T10:32:59.401+02:002014-04-02T10:32:59.401+02:00Very nice question. One of the techniques I have s...Very nice question. One of the techniques I have suggested in the article is unsupervised: Latent Semantic Indexing. This technique does not make use of the class information, and it is intended to reduce the dimmensionality of the (otherwise sparse) text representation, along with capturing the semantic dimmensions of the texts. It does fit clustering perfectly.<br /><br />Other alternatives are the utilization of lexical databases or thesauri to map related terms or synonyms into single classes of semantically related words. For instance, using Roget's Thesaurus, we could map all occurrences of ball, bat, pitcher, etc. into a single class [baseball]. This reduces dimmensionality as well, but it has the problem of Word Sense Disambiguation (which still has limmited effectiveness). Thesauri can be constructed manually or automatically (see classic books by Salton or van Rijsbergen).<br /><br />Most unsupervised dimmensionality reduction techniques come from Information retrieval, as text retrieval is an usupervised task (in general - some subtasks are supervised: Relevance Feedback, Learning to Rank, etc.).<br /><br />From these alternatives, WEKA only supports LSI by SVD. It is not trivial, but you can use Babelnet or Wordnet as thesauri in Java, they both have APIs. Babelnet provides a very simple WSD algorithm as well, implemented in Java.Jose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-29193032457473298772014-04-02T10:12:46.420+02:002014-04-02T10:12:46.420+02:00Dear Jose,
Thanks for your informative sharing.You...Dear Jose,<br />Thanks for your informative sharing.Your posts focused on feature selection/reduction in text classification, which was an supervised problem. But when it comes to unsupervised problems,such as text clustering, what's your solutions for feature selection?Thank you for your reply in advance!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-36589303.post-10337428884126838402014-01-20T09:44:25.230+01:002014-01-20T09:44:25.230+01:00Dear Jose,
you blog is very informative and helpf...Dear Jose,<br /><br />you blog is very informative and helpful. I commend you for your efforts. We have developped a machine learning based method for information retrieval. For this, different attributes like TF.IDF, length, etc are used to classify textual documents. Now I want to know what attributes are the most useful in process of classification. I think you use AttributeSelection with InfoGainAttributeEval as attribute evaluator and Ranker as search method. Using these classes in java code, I would like to know how to get the different attributes with their corresponding information gain. Sample code would be appreciated.<br />Thanks in advance for any help.<br />Best regards,<br /><br />khadimAnonymoushttps://www.blogger.com/profile/09867724863759888684noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-58457065904325415372014-01-02T17:22:39.337+01:002014-01-02T17:22:39.337+01:00Hi
Yes, it will be applied to training and test d...Hi<br /><br />Yes, it will be applied to training and test datasets in CV, but correctly. I mean, for instance in 10-fold CV, WEKA auto atically generates 10 subsets and it uses 9 for training and 1 for testing, round-robin. If you make use of FilteredClassifier with MultiFilter and AttributeSelection, WEKA will apply AttributeSelection on the training folders as input and map it over the testing folder. This way:<br /><br />- Only infirmation from the training folders will be used for selecting the attributes.<br />- The testing folder will be represented according to the attributes selected using the training folders.<br /><br />This is equivalent to having separate trainin and test datasets, and using AttributeSelection (and/or StringToWordVector) in <b>batch mode</b> as I demonstrate, for instance, in my following posts:<br /><br />http://jmgomezhidalgo.blogspot.com.es/2013/05/mapping-vocabulary-from-train-to-test.html<br />http://jmgomezhidalgo.blogspot.com.es/2013/04/command-line-functions-for-text-mining.html<br /><br />Regards and thanks for reading!Jose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-52574735139336475712014-01-02T16:39:00.407+01:002014-01-02T16:39:00.407+01:00Dear Jose Maria,
I have a question concerning thi...Dear Jose Maria,<br /><br />I have a question concerning this approach. When combining the STW filter and IG filter through the MultiFilter in a FilteredClassifier for cross-validation, will the IG filter be applied to both the test and training sets?<br /><br />I would think you want to apply the IG filter only to the training sets, since the IG filter is supervised and shouldn't be applied to the test data.<br /><br />Best regardsAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-36589303.post-90734522454597417292014-01-02T14:27:45.937+01:002014-01-02T14:27:45.937+01:00Dear Jose Maria,
ok great, perhaps you can help m...Dear Jose Maria, <br />ok great, perhaps you can help me with a thought(it is right or not). <br /><br />I am thinking to apply both in text classification<br />i) feature selection via information Gain<br />ii) Feature reduction/transformation via <br />LSI or PCA<br /><br />When i apply the feature selections filter. It work very well however the feature reductions filter is too slow for performance. <br /><br />Do we need to apply both of them or only one of them is enough for text classification? <br /><br />Reason for asking this question is i found different views i.e. some prefer to apply both while some only end up with feature selection (IG or ChiSqr)<br /><br />Thanx for valued information<br /><br />Regard, Kashifkashifhttps://www.blogger.com/profile/11813184161695496997noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-38775317147028059302014-01-02T11:44:50.988+01:002014-01-02T11:44:50.988+01:00Dear Kashif
Do not worry, perhaps I missunderstoo...Dear Kashif<br /><br />Do not worry, perhaps I missunderstood the question.<br /><br />WEKA supports PCA via another filter, which is <i>weka.filters.unsupervised.attribute.PrincipalComponents</i>. I will write another post on the topic in the near future, because it is a very interesting technique for Text Mining, but for the moment, I just shoot this example:<br /><br /><i>java weka.filters.unsupervised.attribute.PrincipalComponents -i weather.numeric.arff</i><br /><br />If you apply Naive Bayes to this dataset without PCA, you get an accuracy of 64.2857 %, however if you apply it after PCA, you get 85.7143 %.<br /><br />RegardsJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-73005051781339470552014-01-02T11:17:51.502+01:002014-01-02T11:17:51.502+01:00Dear Jose Maria,
aah thanx but i realized i misspe...Dear Jose Maria,<br />aah thanx but i realized i misspelled my question above and missed what i really wanted to ask.<br /><br />I want to ask about feature reduction like PCA or LSI(latent semantic indexing). I searched in weka but could'nt find any filter which can be applied like the way we applied in feature selection via information gain....!! Do you know of any filter or way with which i can apply LSI for text classification ?<br /><br />Thanx alot for valued information..kashifhttps://www.blogger.com/profile/11813184161695496997noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-23418592238871684442014-01-02T10:49:04.754+01:002014-01-02T10:49:04.754+01:00Dear Kashif
Regarding other posts dealing with di...Dear Kashif<br /><br />Regarding other posts dealing with dimmensionality reduction, there are several of them; check the label WEKA in the blog: http://jmgomezhidalgo.blogspot.com.es/search/label/WEKA. Most of them are application-oriented.<br /><br />Regarding your second question, the answer is yes; the same technique can be applied to multiclass problems. Entropy, and thus, Information Gain, applies to multiclass problems as well, so the same commands should work in a multiclass problem with no-overlapping classes.<br /><br />In case you have N overlapping classes, you may think about dealing with it by using N binary classifiers (one class against the rest), or using the library MULAN: http://mulan.sourceforge.net/starting.html.<br /><br />RegardsJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-63655040236849413672014-01-01T16:09:24.119+01:002014-01-01T16:09:24.119+01:00Hi, First this post is very informative and defini...Hi, First this post is very informative and definitely helped alot. I just want to ask that you mentioned above to share the dimensionality reduction way in another post, can you redirect me to that post (if its written)?<br /><br /><br />Secondly if i have more than two classes let's say 20, would the way of applying feature selection technique same as discussed in this post ?<br /><br />Thanx alot in advance for your information..kashifhttps://www.blogger.com/profile/11813184161695496997noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-7946825118681103132013-09-05T09:06:02.595+02:002013-09-05T09:06:02.595+02:00Dear Harold
I do not know exactly which kind of n...Dear Harold<br /><br />I do not know exactly which kind of number you want to generate from the bug description text. Specifically, I do not understand why it should depende on the class of a bug.<br /><br />So just guessing: if you need to transform the text in order to predict the class (which I do not know), the best strategy is to decompose the text into words, then selecting predictive words. So you should apply the StringToWordVector filter, then the AttributeSelection filter. You can do it programmatically as explained here: http://jmgomezhidalgo.blogspot.com.es/2013/04/a-simple-text-classifier-in-java-with.html, or you can do it in the command line like here: http://jmgomezhidalgo.blogspot.com.es/2013/04/command-line-functions-for-text-mining.html<br /><br />If you provide me with more details, I may give you better advice...<br /><br />RegardsJose Maria Gomez Hidalgohttps://www.blogger.com/profile/17053588779560658723noreply@blogger.comtag:blogger.com,1999:blog-36589303.post-31751817907716658462013-09-05T04:42:35.170+02:002013-09-05T04:42:35.170+02:00Hi Jose.
Your post is really, really helpful. I&#...Hi Jose.<br /><br />Your post is really, really helpful. I've been reading your posts all night.<br /><br />I am working with Soft Bug Repositories (e.g. Bugzilla, Jira) and one of my features is the bug's description. <br /><br />I was looking for some method for convert or transform this feature into a numerical value. I thought PCA or LSA could help me, but they don't take into account the class info.<br /><br />Do you know if there are other methods that use the class information?.<br />Harold Valdivia Garciahttps://www.blogger.com/profile/04923485024578705507noreply@blogger.com