#!/usr/bin/python -u"""Program: pcdl - a PodCast DownLoader.Author: Ben WestDate: Wed Apr 5 18:42:37 EDT 2006Purpose: Amazon.com wanted a code sample. The task was to create a podcastdownloader. A Podcast is an extension on RSS 2.0 by apple to describe availablemedia.Requirements: Make a podcast downloader that will download the media from apodcast (given a url) into subdirectories named for the "post date".Description:------------ I essentially ran out of time on this. There are a lot of features I would have liked to add to make this easier to use, and more robust. There is no error checking on inputs, which is usually very bad. pcdl asks for a url on stdin but it doesn't check to make sure it's a valid url. If it's an invalid url, the program will barf. Moreover, if the url does not contain valid xml it will barf. Moreover, If the xml does not contain <enclosure> and <pubDate> nodes, probably nothing will happen: no notifying the user at all. Moreover, if there are inadequate permissions, Bad Things will happen. That said, it _does_ work. It will happily download a podcast, attempt to find all the media, and download it according to its pubDate in a directory called "<pubYear>/<pubMonth>/<basename(media_url)". This also assumes that there is only one <pubDate> and <enclosure> per <item>-tree. It will grab the 0th one if there is more than one. The spec by apple (at http://www.apple.com/itunes/podcasts/techspecs.html ) allows for a moved podcast by use of the <itunes:new-feed-url> tag. This tag is not supported in this version. Also, another big minus is that this downloader will ALWAYS download media... It won't check to see if the media at the url is newer than the file that potentially already exists, it'll just download. Anyway, there are lots of ways to improve this. Hopefully by documenting the known limitations and assumptions, it'll make up for what it lacks in features and "robustitude".Features:------------Here are some features I would have liked to have added, but didn't have time(I'll probably go back and add these in for the fun of it.)Commandline switches: help, version, media masks, url, config_file, authorization, proxy, interface, verbosity, pre_path, download_format_string help - Contains useage information. version - maybe not... would have contained version info. media masks - I really wanted to do this one. There are several "supported" media types by Apple. This would have allowed you to positively or negatively mask these types. These would have been registered with the MediaDownloader class as a dictionary of allowed types. It would have worked like: +mp3 to allow mp3's or -mp3 to disallow mp3's. The default is to get anything specified by <enclosure>. url - this would have been nice as well. Allow a user to specify the url on the commandline. This would have made the program pipeable, suitable for use by other scripts (eg, cron). config_file - It also would have been nice to allow a config file at perhaps ~/.pcdl_config which would specify authorization and proxy settings, and perhaps even allow a library of podcasts urls. authorization - This would have allowed a user to specify http auth username and password. proxy - Might allow a user to set proxy settings. interface - Would have liked to created an ncurses interface in addition to a plain-text CLI one. verbosity - This would allow a user to exercise some control over how much output the program generates. pre_path - Allow the user to specify the root where all downloads will begin. The default is "./". download_format_string - The requirement was that the media be downloded into subdirectories named for the publish date. This was a bit ambiguous. I decided to give a sensible default of year/month/ as ints. However, it would have been nice to let the user intepret this requirement and specify it as a date format string.Again, none of these features are implemented."""import xml.dom.minidom, urllib, sys, time, os, mathclass MediaDownloader: """ Class: MediaDownloader Purpose: Provide a simple interface for client code to download a media file specified in a podcast xml document. This provides an apropriate scope for things such as the progress of the current download, and the allowed media files. The only method currently implemented that is publicly useful is download_object, which is described below. Additional methods would have been [un]mask_media, register_allowed_media, {set,get}_pre_path, and {set,get}_download_format. """ def __init__(self, allowed_media = None, pre_path = None): """ We could have instantiated this object using an allowed_media dictionary of allowed file types, configured from either the config file or the commandline. Didn't have time to implement it though. Also would have liked to set the download path, and maybe even use a date format string to create the subdirectories the media is saved. Oh well... we'll set some reasonable defaults below. """ self._pre_path = './' if pre_path: self._pre_path = pre_path self._allowed_media = {} if allowed_media: self._allowed_media = allowed_media self.cur_blocks = 0 def download_object(self, date, url): """ download_object: Takes tuple-time, and the url to the media. Try to download it. """ # Figure out where to download the media. # Default is already described above. path = self._pre_path + '%s/%s' % (date.tm_year, date.tm_mon) dest_path = path + '/' + os.path.basename(url) print 'Downloading %s into %s' % (url, dest_path) # Don't try to create directories that already exist. # If they don't exist, then go ahead and create them. if not os.path.exists(path): os.makedirs(path) # Make sure our property cur_blocks is re-initialized at 0. self.cur_blocks = 0 # Retrieve the media, and show progress as we download. media = urllib.urlretrieve(url, dest_path, self.show_progress) print ' Saved.' def show_progress(self, block_count, block_size, file_size): """ show_progress: Since files can take a long time to download, let's try to show some progress on the way. """ # unfortunately, if the connection doesn't support telling us how big the # file is, we won't give any progress report at all. if(file_size > 0): # figure out how many blocks would be needed. total_blocks = math.ceil(file_size / block_size) # if the difference between this time and last time is greater or equal # than 10 percent, let the user know about it. if math.floor((block_count / total_blocks) * 100) - math.floor((self.cur_blocks / total_blocks) * 100) >= 10: sys.stdout.write(' %d%%' % (math.floor( (block_count / total_blocks) * 100) ) ) sys.stdout.flush() self.cur_blocks = block_count# Command line parsing would have gone here.# Main flow of execution starts here.downloader = MediaDownloader()#gah... trust the user to give use a decent url# TODO: validated input, eg non-empty string, looks like a url.url = raw_input('Enter podcast url: ')print('Fetching %s' % url)#gah... fetch the url and parse it. Would have liked to make this more robust.# TODO: handle errors in opening the url and parsing the xml.pcast = urllib.urlopen(url)doc = xml.dom.minidom.parse(pcast)# This bit is rather easy:# Get all the items, looping through them to get the <pubDate> and <enclosure>items = doc.getElementsByTagName('item')for item in items: pubDate = item.getElementsByTagName('pubDate')[0].firstChild.nodeValue # If the timezone is other than GMT or the date doesn't follow the RFC, this # will fail. pubDate = time.strptime(pubDate, '%a, %d %b %Y %H:%M:%S GMT') media = item.getElementsByTagName('enclosure')[0] # Attempt to download the media. downloader.download_object(pubDate, media.getAttribute('url'))