Fake Video with iTunes Chapter Markers

January 7th, 2006

The other day as I walked home from the gym, listening to my iPod Nano, I started to feel jealous of the iPod Video owners. I love my Nano, but it would be cool to be able to watch video on it from time to time. In spite of having a pretty good color screen, I have no access to such a feature (except perhaps with some Linux-on-iPod trick).

Then I got a sick idea. What if I could convert the frames of a video into chapter markers in an iTunes AAC file?

You may be familiar with chapter markers: they allow authors of multi-part AAC audio files to put marker points in the file so that you can skip between different sections of the audio, even though it’s all part of one file. When you play a song or audio file on your iPod that includes these chapter markers, you can also see an associated image for each chapter.

So, I set out to create an AAC file with lots of chapters. Of course the mechanics of this are a bit harder than it sounds. I had to master Apple’s weird “ChapterTool” (the only official Apple software I know of distributed through a .Mac homepage!), came up with a funny perl script to generate the XML input file that ChapterTool requires, and relied heavily on a modified version of David Van Brink’s qt_tools and Pixture Studio’s awesome QuickImageCM.

I hacked on this for several hours today and finally came up with a pretty interesting demo (10MB). The linked file contains a 3.5 minute song with 800+ chapter markers. The result, as viewed in iTunes, is “somewhat video-like!™” I can’t figure out yet whether the slight jerkiness is because iTunes is not used to flipping chapter art so frequently, or because I screwed something up in the time code generation for the XML file.

Though the file behaves fairly well in iTunes, all hell breaks loose when I try to play the file on my Nano. The Nano sort of freezes for a minute before giving up and skipping to the next track. I guess Nano just isn’t ready to deal with that many chapters in a song file! I tried scaling back my experiment quite a bit by reducing the number of chapters to just one per second. While Nano is now able to play the song, it is rather flaky about keeping the displayed image up to date with the chapters as they fly by. Another major drawback to this “solution” is that having all those chapters on the song essentially makes it impossible to skip to the end or beginning of the track while it’s playing.

So while my dream of hacking video onto my Nano via iTunes is not exactly fulfilled, I think too much went into this not to share the (probably good) news of my failure.

11 Responses to “Fake Video with iTunes Chapter Markers”

  1. Mark Grimes Says:

    The other day as I walked home from the gym, listening to my iPod Nano, I started to feel jealous of the iPod Video owners. I love my Nano, but it would be cool to be able to watch video on it from time to time. In spite of having a pretty good color screen, I have no access to such a feature (except perhaps with some Linux-on-iPod trick).

    How do you watch video while walking, that sounds dangerous? :) The only real reason I would have for getting a video ipod is to jack it into the tv at a friends house to watch a movie, otherwise the nano just seems more ideal in every regard… well until you can actually boot from a usb device that is ;)

  2. Beau Hartshorne Says:

    Check this out: http://engadget.com/2005/11/16/ipod-nano-now-does-video-too/

  3. gman Says:

    So lets see the perl script! That looks awesome enough for me.

  4. Daniel Jalkut Says:

    Mark: Heh – I wasn’t specifically thinking of watching video while I was walking :) Actually I was listening to a Podcast (I think it was This Week in Tech) where they were extoling the virtues of the iPod video.

    Beau: Yes, yes. The Linux solution… but unfortunately I want to keep running “iPod-OS”.

    Gman: Hah! The perl script was only part of the solution, and was in fact the easiest of them all. I used a multi-step process which I haven’t bothered to automate much:

    1. Use a hacked version of qt_export to create a directory full of PICT images for the movie.
    2. Use qt_export to produce an AIFF version of the audio of the movie.
    3. Use iTunes to produce an AAC of the AIFF (for some reason the AAC directly from QuickTime didn’t work as I expected).
    4. Use a Perl script to turn the directory of images into a “ChapterTool” compatible XML file.
    5. Use ChapterTool to put it all together.

    So at this point there isn’t anything I can really easily share… since the practical benefits of this solution seem rather iffy right now, I’m not sure how much I’ll work on refining the process. Just sort of a fun experiment :)

  5. blog.no-panic.at » Blog Archive » Fake Video with iTunes Chapter Markers Says:

    […] » read the whole articel for some more details […]

  6. Mark Says:

    iTunes for Windows doesn’t much like the song either — it’s very skippy. And not in a good peanut buttery sort of way.

  7. Red Sweater Links » Blog Archive » Qt_Tools 2.0 Says:

    […] David Van Brink’s “amazingly useful in a pinch” command-line suite for working with QuickTime movies has been updated to include a new tool for creating a proofsheet image from the frames of a movie. I have used these tools from time to time for batch processing or creative hacking. I’m really happy to see that David is still working on them! Link. […]

  8. dvb Says:

    a) Pardon the long lag time,
    b) Thanks for link and mention for qt_tools 2.0! And finally
    c) Now… why again did you need to modify qt_tools to produce a directory of PICT files? Can’t the grex exporter do that already? Don’t misunderstand: I’m all for hacking it up, that’s why the source code is included… Just trying to remember if there’s an opportunity for a feature there.

  9. Daniel Jalkut Says:

    Hi David! Back when we chatted about this before, you suggested I could use the grex component, but I never got it to work. How do you specify an output folder/file for that component? I tried this:

    qt_export –video=grex:appl –audio=0 MyMovie.mov TestDir/

    and

    qt_export –video=grex:appl –audio=0 MyMovie.mov TestDir/Test.mov

    I end up with errors like this:

    # progress: (012:000) 0.00% MyMovie.mov –> TestDir/Test.mov
    # progress: (012:001) 80.00% MyMovie.mov –> TestDir/Test.mov
    # progress: (012:002) 100.00% MyMovie.mov –> TestDir/Test.mov
    ### error: MovieExportToFile failed
    ### error is -50

    Perhaps it can only be accessed via the –dodialog option somehow? But I don’t see it in the list of video exporters…

  10. dvb Says:

    Arrrrgh my documentation sukkoth. Actually I guess my design does too. I’ll look into that. Meanwhile:

    qt_export –exporter=grex MyMovie.mov TestDir/MyMovie.jpg

    will secrete TestDir/MyMovie 001.jpg and friends. The –exporter option takes components visible from qt_thing –type=spit.

  11. Daniel Jalkut Says:

    dvb: Indeed, that is perfect! the -exporter option kicks butt! thanks for pointing this out.

Comments are Closed.

Follow the Conversation

Stay up-to-date by subscribing to the Comments RSS Feed for this entry.