I was putting some data together about previous catalogs for students for projects in my Applied Databases course and realized that I was missing something. I had course info (subject, number, title and url) for the last 4 catalog years at the College of Idaho, but I didn’t have course descriptions! What a great chance to do some simple web scraping in python.
Data Import and Cleaning Since I have a csv file for each catalog year with a link to each course, I just needed to read the urls, extract the description from the page, and save the results.