Documentation for the ISOCREATOR Vimeo Software
###############################################################################
# OUTLINE
###############################################################################
1: Introduction to isocreator for Vimeo
1.1: Requirements
1.2: Installation
1.3: Integrating isocreator with Vimeo
1.3.1: Using XML
1.3.2: Using SQL
2: Overview
2.1: options.xml
2.2: text_menu_options.xml
2.3: thumbnail_menu_options.xml
2.4: isocreator.sql
2.5: requests.py
2.6: isocreator.py
2.7: generate_xml.py
3: Development
3.1: extending isocreator
4: Unsorted Notes
4.1: Aspect Ratios and Widescreen Clips
4.2: Fonts and Color for Menus
4.3: Acceptable File Types
4.4: Animated Thumbnail Menus
4.5: Custom thumbnails
###############################################################################
# 1: Introduction to isocreator
###############################################################################
Isocreator is a tool developed for Vimeo that generates DVD ISO files. The software
can get information about what ISOs to create from either XML documents, or an SQL
database. Isocreator is meant to be run on a server - it is not a client-side
application. The software's purpose is to allow users of Vimeo to create DVD videos
from their favorite clips on Vimeo.
A small overview of isocreator's program flow would look like this:
read in options
get list of requests
for each request:
download each requested clip
encode each requested clip
create an ISO file of the clips
email the user a link to the ISO file
check for more requests
Isocreator does not come with a frontend. The Vimeo staff will have to provide a GUI
which sends requests to the isocreator script to process. We have tried to make it as
easy as possible to submit a request to the isocreator software. Documentation follows.
###############################################################################
#1.1: Requirements
###############################################################################
Linux-based operating system
Python 2.5 (or greater, should work fine with 2.6)
MySQLdb (may be included in your distro's python - on Ubuntu, apt-get install
python-mysqldb)
On some systems, you may need to install the gsfonts package
tovid - http://tovid.sourceforge.net
You can also not use MySQLdb if you never plan on using the SQL interface -
just comment out the import line in isocreator.py
###############################################################################
#1.2: Installation
###############################################################################
Unzip the files into a folder on your server.
>>Install and patch tovid (http://tovid.sourceforge.net), and then copy the new
>>version of todisc provided with isocreator to wherever you installed tovid to -
>>/usr/local/bin by default
Customize the options files. The default options should be fine.
You will have to set the following in options.xml...
scheme: xml or sql, respectively
smtp_server: if you are using a separate server for sending email, or if the server
you're installing it on can't send email.
smtp_sender: set this to the email address you want emails to appear from
smtp_subject: set these to what you want the subject to be in emails
smtp_message: set these to what you want the message to be in emails
db_server
db_user
db_pass: if you're using SQL
db
requests_table
clips_table: if you're using SQL, and not using the provided isocreator.sql file for
creating the database
url_base: the url to the directory on your server this is extracted to, e.g
http://yoursite.com/isocreator/
You will also want to set some options for DVDs with menus. See sections
2.2 and 2.3 for more information on customization of the DVD menus.
###############################################################################
#1.3: Using isocreator
###############################################################################
We recommend starting isocreator with the command "nohup python isocreator.py &"
This allows you to start through ssh, logout, and have the script continue to
run. There is also a lot of output generated by the script, and you probably
don't want this scrolling all over the screen.
You will probably want to set up a cron job to remove files from the working
directory that are more than a few days old, unless you want to fill up your
server with completed iso files. An example:
0 0 * * * find /path/to/isocreator/working/ -mtime +2 -exec rm {} ;
This would check every day to see if there were any files under the
/path/to/isocreator/working/ directory older than 2 days, and then delete them.
Note that we might keep these files around, or move them to your Amazon S2 account,
for the digital locker features in a future version of this DVD ISO software.
If you want background images for your DVDs, put the image in isocreator's
directory and set the appropriate options in text_menu_options.xml and
thumbnail_menu_options.xml
See sections 2.2 and 2.3 for more information on customization of the DVD menus.
###############################################################################
#1.3.1: Using XML
###############################################################################
There is a directory named requests. XML files should be written to this
directory. One XML file per request. The structure should look like this:
<?xml version="1.0"?>
<request>
<email>user@example.com</email>
<dvd_title>text menu dvd</dvd_title>
<menu_type>thumbnail</menu_type>
<encoding>ntsc</encoding>
<clip>
<title>test clip one</title>
<url>http://example.com/video.avi</url>
<thumbnail>http://example.com/video.jpg</thumbnail>
<description>bllah blah</description>
</clip>
<clip>
<title>test clip two</title>
<url>http://example.com/video2.avi</url>
<thumbnail>http://exaple.com/video2.jpg</thumbnail>
<description>blah blah blah</description>
</clip>
</request>
We believe this is a very simple configuration.
Write the XML to the requests directory. When isocreator is done
processing a request, it will move its XML file into the finished directory.
###############################################################################
#1.3.2: Using SQL
###############################################################################
If you do not want to send XML requests to the script, you can also request DVD
ISO files to be made using SQL.
Assuming you are using the provided isocreator.sql scheme, you would do
something like this to add requests (in php):
mysql_connect($server,$username,$password)
mysql_select_db($db)
$query = "INSERT INTO requests VALUES('','$email','$dvd_title', '$menu_type',
'$encoding');";
mysql_query($query);
$id = mysql_insert_id();
mysql_query("COMMIT;");
//for each clip:
$query = "INSERT INTO clips VALUES('','$id','$clip_title', '$clip_url',
'$thumbnail', '$description');" ;
mysql_query($query);
mysql_query("COMMIT;");
There is a more detailed description of the database in section 2.4
###############################################################################
#2: Overview
###############################################################################
This section is a technical overview of all the important files included in
isocreator. This section should clear up some implementation questions you may have.
###############################################################################
#2.1: options.xml
###############################################################################
This file contains over-arcing options for the script. Most of these options
can be left standard.
Available options include,
scheme: this can be either sql or xml, and needs to be set to whichever way you
are going to be sending requests to isocreator
db_server: if you're using sql, the database server. "none" otherwise
db_user: if you're using sql, the user to connect to the database as
db_pass: if you're using sql, the user's password to use
db: if you're using sql, and aren't using the provided isocreator.sql file, set
the name of the database here. Leave it alone otherwise
requests_table: if you're using sql, and aren't using the provided isocreator.sql file, set
the name of the table to use for requests here. Leave it alone otherwise
clips_table: if you're using sql, and aren't using the provided isocreator.sql file, set
the name of the table to use for clips here. Leave it alone otherwise
requests_dir: if you're using xml, and don't want to use the default place for
storing unprocessed requests, set it here. Leave it alone otherwise.
finished_dir: if you're using xml, and don't want to use the default place for
storing finished requests, set it here. Leave it alone otherwise. (You could
set this to /dev/null/ if you don't want to keep finished requests)
errors_log: if you want to use a different name for logging errors while
making requests, set it here.
encoded_prefix: if you want to change the prefix used for encoded file names,
set it here
smtp_server: your smtp_server. change it if it's not localhost
smtp_sender: the address you want mail to appear from
smtp_subject: subject for emails
smtp_message: what you want to say in the emails.
working_dir: if you want to change the directory that isocreator does it's work in,
change it here
url_base: the url to the directory on your server this is extracted to, e.g
http://vimeo.com/isocreator/
>>use_thumb: whether or not to enable the use of custom thumbnails for clips in dvd
menus.
>>thumb_length: when use_thumb is true, the thumbnail specified for a clip is downloaded
and turned into an mpeg stream to be used in the menu. This is, unfortunately, necessary
(see notes section at the end for a more thorough explanation). Note that this clip is only
used for making thumbnails - it will not appear during dvd play. This option is how long, in
seconds, to make the clip from the thumbnail image. It's provided mainly for speed tweaking -
be aware that values under 3 may cause this to be a bit unstable though.
>>default_thumb: if you want to provide a default thumbnail to use if one isn't provided in
a request, specify it here. Image should ideally be stored in the same directory as isocreator.
###############################################################################
#2.2: text_menu_options.xml
###############################################################################
These are options isocreator.py will use when creating DVDs with text menus
background_image: image to use as the background for DVDs. This should scale well to
720x480 or 720x576 and should be placed in the same directory as isocreator.py
All standard image types, including JPG, GIF, and PNG, are acceptable.
crop_or_scale: whether to crop or scale the background image. We recommend scale
font: font to use for menu text. Can use any font available on your server. You may
need to install additional fonts in your Linux environment.
fontsize: fontsize to use for menu text
align: how to align the text on the screen
text_color: the color to use for text
select_color: the color to use for selected text (when someone hits play on a clip)
highlight_color: the color to use for highlighted text
###############################################################################
#2.3: thumbnail_menu_options.xml
###############################################################################
These are the options isocreator.py will use when creating DVDs with thumbnail
menus
background_image: image to use as the background for dvds. This should scale well to
720x480 or 720x576 and should be placed in the same directory as isocreator.py
All standard image types, including JPG, GIF, and PNG, are acceptable.
menu_font: font to use for menu titles. Can use any font available on your server.
You may need to install additional fonts in your Linux environment.
menu_fontsize: fontsize for menu titles
thumb_font: font to use for thumbnail titles
thumb_fontsize: fontsize for menu titles
title_color: color to use for the menu title
use_fading: whether or not to fade the menu in and out (note, this will make it
take a lot longer to create the menu)
thumbnail_shape: the shape to use for thumbnails. Options are normal (square),
oval, plectrum (guitar pick), and egg.
use_3d: whether to use pseudo-3d thumbnails
playall: whether to add a "playall" button to the menu
###############################################################################
#2.4: isocreator.sql
###############################################################################
This is the dump for the reccomended database structure if you're using the
sql scheme. It creates a database called "isocreator" with two tables on it:
requests:
id email dvd_title menu_type encoding processed
clips:
id request_id title url thumbnail description
clips->request_id is a foriegn key that corresponds with requests->id
The tables use the innoDB storage engine - your host may not support this, and
you should check with them before deciding to use the sql theme
###############################################################################
#2.5: requests.py
###############################################################################
This file, included by isocreator.py defines a few classes used by the script.
Clip - a representation of an individual clip in a request
Request - A base class, representation of one request
XmlRequest - Inherited from Request, knows how to get its info from xml files
SqlRequest - Inherited from Request, knows how to get its info from an sql
database
###############################################################################
#2.6: isocreator.py
###############################################################################
The main script. This reads in options files, creates appropriate objects
depending on what scheme is being used, downloads and encodes clips, creates
ISO files, and emails the user.
###############################################################################
#2.7: generate_xml.py
###############################################################################
This can be used to make a bunch of xml files if you want to test the script,
or just see example xml files.
Open the file and near the top there's a line that says movie_urls = []
fill this with the clips you'd like to use to test with. e.g.:
movie_urls = ['http://www.example.com/vid1.avi',
'http://www.example.com/vid2.wmv']
at the bottom of the script is says:
for i in range(0,10)
change 10 to whatever number of xml files you want to create
Note that the XML files created by this are very rough and for development testing
only.
###############################################################################
#3: Development
###############################################################################
###############################################################################
#3.1: Extending isocreator
###############################################################################
New schemes can be made by inheriting from the Request class in requests.py
and implementing the functions get_request() and get_clips(). For instance,
let's say we want to read from a CSV file.
class CsvRequest(Request):
def __init__(self,get_from):
Request.__init__(self,get_from):
def get_request(self):
#read in the user's information from the csv file
#which should be self.get_from
def get_clips(self):
#populate the self.clips list with the clip informatio
After creating the new class, you need to add appropriate code to the
get_requests() method in isocreator.py and also pretty much anywhere
in the code you see "if options['scheme'] - this is only three or four places,
implementing new ways of retrieving requests should be relatively painless.
###############################################################################
#4: Notes
###############################################################################
###############################################################################
#4.1: Aspect Ratios and Widescreen Clips
###############################################################################
In options.xml - aspect - this can be set to an aspect ratio (e.g. 4:3) or "auto"
if set to an aspect ratio, the script will encode clips to said aspect ratio. if
set to auto, it will keep the clips original ratio and letterbox them in the dvd.
We have successfully made a few dvds with the "auto" feature, and had widescreen
and odd-aspected clips letterboxed correctly. The dvd is forced 4:3 right now
(but the clips aren't), so things get letterboxed if they are widescreen.
We can add an aspect ratio chooser option for DVDs as well, but I don't know of
many tvs that aren't 4:3. And I don't think it's an option that would be used
very often. (and it would have to be on a per-request basis, which makes it a bit
odd for automation)
For now I think this meets Jakob's requirement that "Every video should be as
large as possible, but cannot be cropped or stretched."
###############################################################################
#4.2: Fonts and Color for Menus
###############################################################################
Don't be surprised if your results for "convert -list type" are slim. Most web
servers will not have any fonts installed (no graphical environment). Let us know
if you need help installing fonts on your server. The list of available colors in
"convert -list color" should be huge.
###############################################################################
#4.3: Acceptable File Types
###############################################################################
We have tested making DVDs with wmv, avi, mov, mpg, and rm files.
###############################################################################
#4.4: Animated Thumbnail Menus
###############################################################################
When making a thumbnail dvd, the script seeks to two seconds into the movie,
and makes a small movie clip to use as a thumbnail.
In thumbnail_menu_options.xml, thumb_type can be set to static or animated.
using static thumbnail menus increases the speed of creating DVDs dramatically.
Running a simple test with static thumbnails, the ISO creation process took ~ 1:13.
With movie clip thumbnails in the DVD menu, the ISO creation process took ~ 23:45.
Please note that we cannot substitute the Vimeo thumbnail GIF or JPG for the DVD
menu thumbnail. The thumbnail has to be generated during the DVD creation process.
###############################################################################
#4.5: Using your own thumbnails
###############################################################################
There is an option now in options.xml - use_thumb, which can be set to true or
false. When enabled, this option will do a few things:
For thumbnail menus:
Use the thumbnail specified for a clip as the thumbnail in the menu. This
thumbnail will be shown before the clip when watching the movie.
For other types:
Insert the thumbnail for a clip before the clip
NOTES FOR INTEGRATION-
Clip length needs to be present for all of your source files.
>>
####################################################
# On using custom thumbnails
#####################################################
Alright, as of now, thumbnails can be specified without adding an extra clip to the dvd itself.
An extra clip still does get generated though - just an mpeg stream of the still frame, which gets
swapped for the original clip in the part of todisc that searches through a clip to create thumbnails
for it.
It should be possible to modify todisc to accept just an alternate thumbnail image and not a mpeg
stream created from an image - this would be a pretty big speed up, minute per few clips or so. If
you know BASH scripting, open up todisc, and search for #XXXXXX to see the relevant modifications made
to todisc that allowed swapping of videos searched through for clips.