MATLAB Answers

0

how to find most common words in text by matlab

Asked by bita hallajian on 28 Oct 2017
Latest activity Commented on by Charmaine Tan on 26 Nov 2018
how to tag POS on nouns and verbs in MATLAB, Is it related to regular expressions? I know that regular expressions find a pattern in a text, but I want to find the most common words in texts and tag POS on them( I mean the words are nouns or verbs) and then exchange that POS and make an unfamiliar pair of words. how can I find the most common words in texts by MATLAB?is there any solution for that or I should use another software?

  0 Comments

Sign in to comment.

3 Answers

Answer by Christopher Creutzig on 2 Nov 2017
Edited by Christopher Creutzig on 26 Nov 2018
 Accepted Answer

Finding the most common words is easy with Text Analytics Toolbox:
>> sonnets = extractFileText("sonnets.txt");
>> sonnets = erasePunctuation(sonnets);
>> tokenizedSonnets = tokenizedDocument(lower(sonnets));
>> bag = bagOfWords(tokenizedSonnets);
>> topkwords(bag, 10)
ans =
10×2 table
Word Count
______ _____
"and" 490
"the" 436
"to" 409
"my" 371
"of" 370
"i" 344
"in" 321
"that" 320
"thy" 281
"thou" 234
You probably want to remove some words (check out removeWords and stopWords). POS tagging is supported in release R2018b and later, see addPartOfSpeechDetails.

  2 Comments

With great thanks for your help I'll try the helpful points you directed me I created a text document with the name "sonnets.txt" and loaded it in command window but there is an error : "Number of columns on line 2 of ASCII file sonnets.txt must be the same as previous lines." Can you give me useful advice to eliminate this error?
What command(s) did you try to read that file? The error message looks like you tried to read it as a table; try using the commands listed above instead.

Sign in to comment.


Answer by Sarah Palfreyman on 30 Apr 2018
Edited by Sarah Palfreyman on 30 Apr 2018

  2 Comments

Which version of matlab is the least that supports the Text analytic toolbox?
R2017b

Sign in to comment.


Answer by Charmaine Tan on 26 Nov 2018

Hi, after finding my topkwords (most frequent words), how can I plot a histogram of these?

  2 Comments

txt = extractFileText('sonnets.txt');
td = tokenizedDocument(lower(txt));
td = erasePunctuation(td);
bow = bagOfWords(td);
top = topkwords(bow,20);
bar(top.Count)
set(gca,'XTick',1:size(top,1),'XTickLabel',top.Word,'XTickLabelRotation',45)
Screen Shot 2018-11-26 at 09.23.48.png
(In general, it's a good idea not to ask a new question as an “answer,” but to open a new question instead. It helps other people searching MATLAB Answers in the future.)
Noted, I'll do that. Thanks a lot!

Sign in to comment.