Introduction to Python Text Processing Text processing directly applies to natural language processing (NLP). NLP aims to process language communication between humans, whether spoken or written. This differs from communication between computers and humans, which occurs either through human-written programs…
Python WordNet Interface WordNet is an English dictionary, similar to a traditional thesaurus. NLTK includes WordNet for English. We can use it as a reference to find word meanings, usage examples, and definitions. Groups of similar words are called word…
Counting Tokens in a Paragraph with Python When reading text from a source, we sometimes need to know some statistical information about the types of vocabulary used. This makes it necessary to count the number of words of a specific…
Removing Stop Words in Python Stop words are words that don’t have much meaning in an English sentence. They can be safely ignored without losing the meaning of the sentence. For example, words like the, he, have, etc. These words…
Python Text Processing Tutorial Python programming can be used to process a variety of text data needs, particularly in the area of natural language processing (NLP). NLP is used in search engines, newsfeed analysis, and more recently, voice applications like…
Python Synonyms and Antonyms Synonyms and antonyms are available as part of the English lexical database Wordnet. It can be accessed through the NLTK corpus. In Wordnet, synonyms are words that represent the same concept and can be used interchangeably…
Python Biagrams Some English words appear together more frequently than others. For example, “Sky High,” “Do or Die,” “Best Performance,” “Heavy Rain,” and so on. Therefore, in text documents, we may need to identify such word pairs to aid in…
Reformatting Paragraphs in Python When processing large amounts of text and presenting them in an acceptable format, we need to format the paragraphs. We might want to print each line at a specific width, or add indents to each line…
Filtering Duplicate Words in Python Many times, we only need to analyze unique words in a document. Therefore, we need to eliminate duplicate words from the text. This can be achieved by using word tokenization and set functions provided by…
Python Strings as Files When reading a file, it is read as a dictionary with multiple elements. Therefore, we can access each line of the file using the index of the element. In the following example, we have a file…