Moving at the Speed of Creativity by Wesley Fryer

Lessons Learned About Enhanced or Multimedia EPUB eBook Validation

We are living in the midst of a “sea change” for books, literacy, reading and writing. The way in which official standards for EPUB eBooks are transitioning to support embedded video and audio files provide vivid a case in point. I spent most of this afternoon and evening debugging and fixing the enhanced (or “multimedia”) version of my eBook, “Playing with Media: simple ideas for powerful sharing,” so the EPUB formatted file can successfully pass “verification” tests and go for sale on the iTunes iBookStore. I learned quite a few new things, and in this post I’ll highlight some of them. This theme of EPUB eBook publishing has become a series here, so if you want to catch up on past, related posts please see:

'epub ebooks img' photo (c) 2009, Cristian Eslava - license: http://creativecommons.org/licenses/by-sa/2.0/

1. EPUB eBooks are Compressed Zip Files

An EPUB book may look like a single file on your computer’s hard drive or saved to the flash memory storage of your eReader, but it’s actually a collection of files. The “EPUB Overview” portion of the Sigil documentation website explains the different elements of an EPUB file in relatively plain language. In addition to individual HTML (specifically XHTML) pages for each chapter of an EPUB eBook, images and other media elements (including videos) are saved as separate files in their own folders or directories. “The OPF file” includes metadata for the book and the “manifest” which is a comprehensive list of all files comprising the eBook. “The NCX file” is the table of contents and is also included. A working knowledge of these files and directories proved helpful this evening as I sought to validate my EPUB to publish it on iTunes.

Although EPUB books are essentially compressed files using the ZIP file format, I learned it’s not advisable to unzip and zip EPUB files with “standard” file compression software. Instead, it’s best to use freely available EPUB editing software programs like Calibre and Sigil. I’ll discuss more about these later in the post.

2. Viewing an EPUB on iBooks on an iPad Doesn’t Guarantee Format Validity

Apple’s Pages ’09 software makes the process of creating an EPUB eBook relatively straightforward. From the FILE menu choose EXPORT and then EPUB. The resulting EPUB file can be transferred to an iPad for viewing in the iBooks application in several ways:

  1. Load the file in iTunes on your computer (in the iBooks application) and sync your iPad.
  2. Upload the EPUB eBook to a website like DropBox (free) and then download it to your iPad using the free DropBox app.
  3. Use the app AirSharing ($3) to wirelessly connect your computer directly to your iPad, and use a web browser (with the AirSharing provided local URL) to upload the EPUB to the iPad. Once transferred, select it and choose to open it in iBooks.

Just because an EPUB you create in Pages ’09 can be viewed without apparent errors or problems on an iPad, don’t assume your EPUB is ready for upload to the iTunes iBookstore or other websites. Strict coding rules have been created for EPUB eBooks to insure compatibility on multiple devices, and the rules for embedding video as well as audio are still being finalized. I found Apple Pages is very forgiving with filenames and file formats when exporting to the EPUB format, but it does NOT export multimedia or enhanced EPUBS (at least in the version I’m using now) which “validate” properly according to official EPUB standards.

3. Apple Pages ’09 is Insufficient to Create Valid Enhanced or Multimedia EPUB eBooks

The term “enhanced EPUB” is used to describe EPUB ebooks which include embedded video files. These files are saved WITHIN the EPUB file, so a reader does not need to be online with a “live” Internet connection to view them. The phrase “multimedia EPUB” is also used by some individuals to describe these special files. While it’s possible to create working, enhanced EPUB eBooks using Apple Pages which include embedded videos, I found it necessary to use several other programs to “tweak” my exported Pages EPUB so it validated properly with the iTunes Connect website and “iTunes Producer” software. iTunes Producer is provided to people and organizations who register with iTunes Connect.

I used FlightCrew EPUB validator software, Calibre software, QuickTime Pro 7 software, TextWrangler software, and Elizabeth Castro’s $5 eBook, “Audio and Video in EPUB” to tweak my multimedia EPUB today so it validated successfully. I wish I could say Apple Pages ’09 is all you need for this process, but it’s not. If you’re creating multimedia EPUBs for classroom use and will just share them with other iPad users, you might be OK skipping this more rigorous process of “validating” the EPUB. I’m sure Apple will improve its EPUB exporting features in future versions of Pages too, so the steps I went through tonight may be unnecessary down the road.

In its current version, Pages ’09 always converts embedded video files to QuickTime (.mov) videos when a user exports a document to EPUB format. According to Elizabeth Castro, the best video format to include in EPUB eBooks today for compatibility on the Barnes & Noble Nook eReader as well as the iPad is MPEG-4 (.m4v) video files. I followed Elizabeth’s QuickTime Pro export settings precisely and created .m4v versions of my 17 embedded videos. This required a bunch of file and code changes to make these videos play properly within my final EPUB file.

4. Name EPUB Image and Video Files Carefully

Generally I follow strict file naming conventions when sharing online. I use all lowercase characters in file names, avoid special characters and spaces, and use hyphens when needed in longer file names. Thankfully I automatically followed those conventions for most of the 133 images included in my eBook and the 17 referenced videos. I did NOT follow those conventions in all cases, however, and that turned out to be one cause of validation errors in my EPUB files tonight. Apple offers the following five possibilities for why an uploaded EPUB file might not properly validate:

  • Your EPUB contains files not listed in the manifest. This error is not generally caught by EPUBCheck. All files must be listed in the manifest, or there is no way to confirm that they are intended for distribution.
  • Your EPUB contains characters that are not UTF-8 or UTF-16. This is typically caused by including accented or other special characters in the EPUB. If they are not properly encoded, they will not appear as expected when the customer reads your book. iTunes Producer lists the exact files that have these issues in the delivery error report.
  • XHTML namespace is not properly declared in all HTML in your EPUB. This is a strict requirement for the EPUB standard. Check to ensure the proper namespace is declared in each HTML document. See the current spec at http://www.idpf.org/2007/ops/ops2.0/download/.
  • All date formats are not full and complete. There are strict requirements for dates listed in the EPUB OPF and NCX files. Check to see that they are complete and properly formatted.
  • Your filenames have spaces or encoded characters. If your EPUB has any spaces in filenames, be sure the spaces are properly encoded in the EPUB manifest by using “%20” in their place. Filenames may not contain periods (“.”) other than to separate the filename from its extension.

Avoid nightmarish EPUB validation errors like these

I wish I’d known, at the start of this writing project, that I needed to follow strict conventions for filenames. That could have saved several hours of work today. Also, it would have been nice to know about this clever workaround for providing alternative text for embedded videos when people use screen readers with your EPUB eBook.

5. Use Specific M4V (MPEG-4) Video Compression Settings

The “expert settings” in QuickTime Pro can be daunting, and until today I’d just left my screencasts and videos in their original formats when I embedded them in my EPUB. Elizabeth Castro painstakingly details the settings needed for EPUB video compatibility on eReaders in addition to the iPad in her $5 “Audio and Video in EPUB” eBook, however, and I followed her advice. I’m not sure if I’ll sell the enhanced / multimedia version of my eBook outside the iTunes iBookstore for purchase on devices like the Nook, but at some point I might. (Currently B&N’s PubIt website doesn’t accept enhanced EPUB books from independent publishers. They also limit the size of uploaded EPUBs to 20 MB, which is generally too small for enhanced EPUBs.)

6. Try Different EPUB Validation and Editing Tools

I learned today there are at least three different EPUB format validation standards, and it’s a good idea to use several if you’re trying to upload to Apple’s iBookStore. EpubCheck is the most well known validation tool, but I never managed to get it running locally on my Mac. I used the free, web-based version of EpubCheck on Threepress Consulting’s website to validate the “standard” (non-multimedia) version of my eBook, but it limits upload file size to just 10 MB. My standard EPUB is 7.5 MB, but my final enhanced EPUB is 225 MB.

Passed epubcheck

I used the FlightCrew EPUB validator, which is used by the Sigil eBook editor, to validate both my standard and multimedia EPUB versions.

No problems found!

FlightCrew validation errors for Multimedia eBook - 14 August 2011

Both EpubCheck and FlightCrew are validating EPUBs using the 2.0 standard, as of this writing. The 2.0 standard does not include videos and multimedia EPUBS. Those specifications are coming with version 3.0. For this reason, I had to disregard the errors FlightCrew turned up for my EPUB which were related to video tag elements, manifest entries, etc. Lulu.com also seems to validate EPUBs currently to the 2.0 standard, and does NOT accept enhanced/multimedia EPUBs.

EPUB error: Unmanifested files

Sigil lets you edit the contents of your eBook, but you cannot edit the “OPF file” which includes files on the manifest. For this reason, I found Calibre along with TextWrangler to be my software tools of choice for editing my EPUB files after exporting out of Pages.

After loading your EPUB into Calibre, right click (or control-click on a Mac) and choose “Tweak ePub” to make edits to your individual files.

Tweak EPUB in Calibre

Although it sounds violent, don’t be dissuaded by the name… Choose “Explode ePub” to view all the component parts within your file.

Explode EPUB in Calibre

You can then use a text editor (I love TextWrangler, it’s free and powerful) to make needed text changes to your documents. You can also rename and move files as needed from the “images” and “media” folders within your EPUB. When finished, back in Calibre choose to REBUILD your EPUB.

Rebuild EPUB

7. Multiple ISBNs Required for Different Book Formats on iTunes

iTunes Connect requires different, unique ISBNs for each eBook you list on the store. ePubBud is one site where I’ve seen discussion forum posts discouraging the use of unique ISBNs for eBooks, but that advice can’t go for iTunes. Your need for an ISBN depends on purpose and context. If you’re not selling on iTunes, you may not need or want to buy an ISBN. ePubBud will sell you an ISBN for $5, which is far cheaper than you can buy them directly (in small quantities) in the United States from the official ISBN agency. I purchased a block of ten ISBNs several months ago, and I’m using three of them currently for “Playing with Media: simple ideas for powerful sharing.”

  • One for the Amazon Kindle (.mobi & .azw) version
  • One for the “standard EPUB” version
  • One for the “enhanced EPUB” or “multimedia EPUB” version

Closing Thoughts

This process of getting an enhanced / multimedia EPUB eBook available for sale and download on the iTunes iBookstore has been a much more tedious and involved process than I anticipated. I knew I’d learn a great deal attempting this process myself, so on that front I haven’t been disappointed! I am hopeful the process will get a bit simpler, but I suspect companies serving as “Apple certified aggregators” for EPUB eBooks will enjoy this complexity since it will bring more business their way. Most people are NOT going to want to make the file conversion and coding changes which took me HOURS today to sell their own eBooks online. The price for having someone else do this “heavy lifting” is steep, however. These companies don’t just want a one-time fee, they require by contract (and it’s not negotiable, from what I’ve found) a PERMANENT PERCENTAGE of your eBook sales forever. Consider carefully if you want to give that up before signing a contract with an aggregator. As of late July 2011, only two companies (Ingram and Libre Digital) were certified “aggregators” by Apple to handle enhanced / multimedia EPUBs.

iBookstore Aggregators

Now that I THINK I have successfully “validated” my EPUB versions for the iBookStore, it’s time to play “the waiting game” again for awhile.

iTunes Connect - 2 Versions Uploaded

Hopefully in a week or two, both the “enhanced” and “standard” versions of “Playing with Media: simple ideas for powerful sharing” will be available for sale via iTunes. Until then and afterwards, you can purchase the standard version on Amazon and Barnes & Noble! The book has five reviews on Amazon.com so far, and I encourage you to submit your own review after reading it!

Please also consider “liking” the “Playing with Media” book page I’ve setup on Facebook. I hope to release a complimentary, interactive website within a few months which I eventually hope to make into a mobile application for media project sharing and commenting. I’m hoping it will be “4Square-ish” in some ways. Let the “playing with media” continue!

 

Technorati Tags:
, , , , , , , , , ,

 


Posted

in

, , , ,

by

Tags:

Comments

10 responses to “Lessons Learned About Enhanced or Multimedia EPUB eBook Validation”

  1. Kevin Ho Avatar

    I just published my Enhanced “The Golden Goose Book” at Lulu.com, but for iBookstore, they said “Lulu doesn’t accept Enhanced eBook for iBookstore”.

  2. Eric Freese Avatar
    Eric Freese

    Congratulations on your book. It’s a given that EPUB is the accepted standard for eBooks. Authors can always try their creative hand in converting their printed books to eBooks or making existing eBooks interactive. But with so many eBook readers, dedicated and otherwise, it’s never an easy thing to ensure that the final eBook is readable across all platforms (and initially it will be worse in EPUB 3). Like their printed counterparts, eBooks should be easily read no matter what mobile device is used to open them. 
    Eric FreeseAptara, Inc.http://www.aptaracorp.com

  3. Stanislav Fritz Avatar
    Stanislav Fritz

    Sigil DOES allow editing of OPF and any of the files in the epub.  At least it has for the past 11 months.  I have not used it prior to that.

  4. Ankit Kumar Ojha Avatar
    Ankit Kumar Ojha

    How can we make images moving in ePub??

  5. Janus Avatar
    Janus

    Quite an interesting article, but what I would like to know is this…

    All those multimedia epubs seem to be designed/created for iPads and other “Apple” products or programs, but what other solutions (in general and for the Windows/Linux/… users) are there that can open and display such epubs without some sort of content or format loss? (i.e. I have a few multimedia epubs, but whatever program I try, none are capable of showing the videos/pictures/… integrated into the text as intended and the process of extracting said videos/pictures is somewhat time-consuming and counterproductive.)
    The usual suspects (Calibre, Firefox epub plugin, Adobe D.E., …) are apparentely not yet capable of processing multimedia epubs…
    I would appreciate any ideas, tips and/or solutions for this problem, thank you.

  6. Wesley Fryer Avatar

    The first cross-platform solution I’ve found to read enhanced eBooks on any platform is the Readium Plugin for Chrome (free):

    https://chrome.google.com/webstore/detail/readium/fepbnnnkkadjhjahcafoaglimekefifl?hl=en

    It’s part of the Readium project which is a standards-based effort to bring ePUB3 readability to the open web:

    http://readium.org

  7. Wesley Fryer Avatar

    I don’t think animated GIFs will work, I think you need to embed a video.

  8. Wesley Fryer Avatar

    I’m not sure if this has changed since… When I was publishing my first eBook in 2011 it wasn’t possible to publish enhanced eBooks to Apple’s iBookstore via Lulu because the ePUB 3.0 spec wasn’t finalized and there wasn’t a standards-based we to pre-flight / verify the ebook. Now there might be…