Reading RSS Feeds with Python

RSS (Rich Site Summary) is a format for delivering regularly changing web content. Many news websites, blogs, and other online publishers make their content available as RSS feeds. In Python, we use the following packages to read and process these feeds.

pip install feedparser

Feed Structure

In the following example, we retrieve the structure of a feed so we can further analyze which parts of the feed we want to process.

import feedparser
NewsFeed = feedparser.parse("https://timesofindia.indiatimes.com/rssfeedstopstories.cms")
entry = NewsFeed.entries[1]

print entry.keys()

When we run the above program, we will get the following output −

['summary_detail', 'published_parsed', 'links', 'title', 'summary', 'guidislink', 'title_detail', 'link', 'published', 'id']

Feed Title and Articles

In the following example, we read the title and header of a feed.

import feedparser

NewsFeed = feedparser.parse("https://timesofindia.indiatimes.com/rssfeedstopstories.cms")

print 'Number of RSS posts :', len(NewsFeed.entries)

entry = NewsFeed.entries[1]
print 'Post Title :',entry.title

When we run the above program, we get the following output –

Number of RSS posts : 5
Post Title : Cong-JD(S) in SC over choice of pro tem speaker

Feed Details

Based on the above entry structure, we can use Python program to get the required details from the feed. Since entry is a dictionary, we utilize its keys to produce the required values.

import feedparser

NewsFeed = feedparser.parse("https://timesofindia.indiatimes.com/rssfeedstopstories.cms")

entry = NewsFeed.entries[1]

print entry.published
print "******"
print entry.summary
print "------News Link--------"
print entry.link

When we run the above program, we get the following output –

Fri, 18 May 2018 20:13:13 GMT
****
Controversy erupted on Friday over the appointment of BJP MLA K G Bopaiah as pro tem speaker for the assembly, with Congress and JD(S) claiming the move went against convention that the post should go to the most senior member of the House. The combine approached the SC to challenge the appointment. Hearing is scheduled for 10:30 am today.
------News Link--------
https://timesofindia.indiatimes.com/india/congress-jds-in-sc-over-bjp-mla-made-pro-tem-speaker-hearing-at-1030-am/articleshow/64228740.cms