use grub for dos (http://grub.blogeden.cn/post/23308/2412 - it's in chinese though)

If you just want to do something like in linux live cd, i.e., the iso file is used to host the systems files, it's easy and straightforward to use iso.

If you want to boot from a iso file on a harddisk, do something in menu.lst like

title Boot from iso on a harddisk
map (hdX,Y)/your.iso (hdZ)
map --rehook
chainloader (hdZ)+1
rootnoverify (hdZ)
boot

The iso file must be in one chunk - contig from sysinternal http://www.sysinternals.com/Utilities/Contig.html may come handy. It's a good idea to use fat32 to host the iso file.

Also check out
http://www.911cd.net/forums//index.phpshowtopic=15617&st=20&p=101928&#entry101928
and
http://www.911cd.net/forums//index.php?showtopic=18045