Homework 5 – Using the NLTK to investigate corpora and word frequency

In this homework you will practice loading and extracting information from various corpora, and calculating word frequency and conditional word frequency. There will be questions about conditionals, loops, and list expressions. Please put your answers in an executable python script named hmwk5.py, and commit it to the subversion repository.
It is due Oct. 2nd and covers material up to Sep. 24th.

1. Create a list called my_ints, with the following values: (10, 15, 24, 67, 1098, 500, 700) (2 points)

2. Print the maximum value in my_ints (3 points)
3. Use a for loop and a conditional to print out whether the value is an odd or even number. For example, for 10, your program should print “10 is even”. (5 points)
4. Now create a new list called new_ints and fill it with values from my_ints which are divisible by 3. In addition, double each of the new values. For example, the new list should contain 30 (15*2). Use a for loop and a conditional to accomplish this task. (5 points)
5. Now do the same thing as in the last question, but use a list expression to accomplish the task. (5 points)
6. Import the Reuters corpus from the NLTK. How many documents contain stories about coffee? (4 points)
7. Print the number of words in the Reuters corpus which belong to the barley category. (5 points)
8. Create a conditional frequency distribution of word lengths from the Reuters corpus for the categories barley, corn, and rye. (8 points)
9. Using the cfd you just created, print out a table which lists cumulative counts of word lengths (up to nine letters long) for each category. (5 points)
10. Load the devilsDictionary.txt file from the ling5200 svn repository in resources/texts into the NLTK as a plaintext corpus (3 points)
11. Store a list of all the words from the Devil’s Dictionary into a variable called devil_words (4 points)
12. Now create a list of words which does not include punctuation, and store it in devil_words_nopunc. Import the string module to get a handy list of punctuation marks, stored in string.punctuation. (5 points)
13. Create a frequency distribution for each of the two lists of words from the Devil’s dictionary, one which includes punctuation, and one which doesn’t. Find the most frequently occuring word in each list. (6 points)