We are living in the midst of a “sea change” for books, literacy, reading and writing. The way in which official standards for EPUB eBooks are transitioning to support embedded video and audio files provide vivid a case in point. I spent most of this afternoon and evening debugging and fixing the enhanced (or “multimedia”) version of my eBook, “Playing with Media: simple ideas for powerful sharing,” so the EPUB formatted file can successfully pass “verification” tests and go for sale on the iTunes iBookStore. I learned quite a few new things, and in this post I’ll highlight some of them. This theme of EPUB eBook publishing has become a series here, so if you want to catch up on past, related posts please see:
- 25 Jul 2011: Reading the Fine Print: Considering Different eBook Publishing Options for the iBookstore
- 29 Jul 2011: Publishing eBooks to Amazon and iTunes iBookstore: Learning Curve Continues
- 5 Aug 2011: Validation errors in a multimedia or enhanced EPUB eBook
1. EPUB eBooks are Compressed Zip Files
An EPUB book may look like a single file on your computer’s hard drive or saved to the flash memory storage of your eReader, but it’s actually a collection of files. The “EPUB Overview” portion of the Sigil documentation website explains the different elements of an EPUB file in relatively plain language. In addition to individual HTML (specifically XHTML) pages for each chapter of an EPUB eBook, images and other media elements (including videos) are saved as separate files in their own folders or directories. “The OPF file” includes metadata for the book and the “manifest” which is a comprehensive list of all files comprising the eBook. “The NCX file” is the table of contents and is also included. A working knowledge of these files and directories proved helpful this evening as I sought to validate my EPUB to publish it on iTunes.
Although EPUB books are essentially compressed files using the ZIP file format, I learned it’s not advisable to unzip and zip EPUB files with “standard” file compression software. Instead, it’s best to use freely available EPUB editing software programs like Calibre and Sigil. I’ll discuss more about these later in the post.
2. Viewing an EPUB on iBooks on an iPad Doesn’t Guarantee Format Validity
Apple’s Pages ’09 software makes the process of creating an EPUB eBook relatively straightforward. From the FILE menu choose EXPORT and then EPUB. The resulting EPUB file can be transferred to an iPad for viewing in the iBooks application in several ways:
- Load the file in iTunes on your computer (in the iBooks application) and sync your iPad.
- Upload the EPUB eBook to a website like DropBox (free) and then download it to your iPad using the free DropBox app.
- Use the app AirSharing ($3) to wirelessly connect your computer directly to your iPad, and use a web browser (with the AirSharing provided local URL) to upload the EPUB to the iPad. Once transferred, select it and choose to open it in iBooks.
Just because an EPUB you create in Pages ’09 can be viewed without apparent errors or problems on an iPad, don’t assume your EPUB is ready for upload to the iTunes iBookstore or other websites. Strict coding rules have been created for EPUB eBooks to insure compatibility on multiple devices, and the rules for embedding video as well as audio are still being finalized. I found Apple Pages is very forgiving with filenames and file formats when exporting to the EPUB format, but it does NOT export multimedia or enhanced EPUBS (at least in the version I’m using now) which “validate” properly according to official EPUB standards.
3. Apple Pages ’09 is Insufficient to Create Valid Enhanced or Multimedia EPUB eBooks
The term “enhanced EPUB” is used to describe EPUB ebooks which include embedded video files. These files are saved WITHIN the EPUB file, so a reader does not need to be online with a “live” Internet connection to view them. The phrase “multimedia EPUB” is also used by some individuals to describe these special files. While it’s possible to create working, enhanced EPUB eBooks using Apple Pages which include embedded videos, I found it necessary to use several other programs to “tweak” my exported Pages EPUB so it validated properly with the iTunes Connect website and “iTunes Producer” software. iTunes Producer is provided to people and organizations who register with iTunes Connect.
I used FlightCrew EPUB validator software, Calibre software, QuickTime Pro 7 software, TextWrangler software, and Elizabeth Castro’s $5 eBook, “Audio and Video in EPUB” to tweak my multimedia EPUB today so it validated successfully. I wish I could say Apple Pages ’09 is all you need for this process, but it’s not. If you’re creating multimedia EPUBs for classroom use and will just share them with other iPad users, you might be OK skipping this more rigorous process of “validating” the EPUB. I’m sure Apple will improve its EPUB exporting features in future versions of Pages too, so the steps I went through tonight may be unnecessary down the road.
In its current version, Pages ’09 always converts embedded video files to QuickTime (.mov) videos when a user exports a document to EPUB format. According to Elizabeth Castro, the best video format to include in EPUB eBooks today for compatibility on the Barnes & Noble Nook eReader as well as the iPad is MPEG-4 (.m4v) video files. I followed Elizabeth’s QuickTime Pro export settings precisely and created .m4v versions of my 17 embedded videos. This required a bunch of file and code changes to make these videos play properly within my final EPUB file.
4. Name EPUB Image and Video Files Carefully
Generally I follow strict file naming conventions when sharing online. I use all lowercase characters in file names, avoid special characters and spaces, and use hyphens when needed in longer file names. Thankfully I automatically followed those conventions for most of the 133 images included in my eBook and the 17 referenced videos. I did NOT follow those conventions in all cases, however, and that turned out to be one cause of validation errors in my EPUB files tonight. Apple offers the following five possibilities for why an uploaded EPUB file might not properly validate:
- Your EPUB contains files not listed in the manifest. This error is not generally caught by EPUBCheck. All files must be listed in the manifest, or there is no way to confirm that they are intended for distribution.
- Your EPUB contains characters that are not UTF-8 or UTF-16. This is typically caused by including accented or other special characters in the EPUB. If they are not properly encoded, they will not appear as expected when the customer reads your book. iTunes Producer lists the exact files that have these issues in the delivery error report.
- XHTML namespace is not properly declared in all HTML in your EPUB. This is a strict requirement for the EPUB standard. Check to ensure the proper namespace is declared in each HTML document. See the current spec at http://www.idpf.org/2007/ops/ops2.0/download/.
- All date formats are not full and complete. There are strict requirements for dates listed in the EPUB OPF and NCX files. Check to see that they are complete and properly formatted.
- Your filenames have spaces or encoded characters. If your EPUB has any spaces in filenames, be sure the spaces are properly encoded in the EPUB manifest by using “%20″ in their place. Filenames may not contain periods (“.”) other than to separate the filename from its extension.
I wish I’d known, at the start of this writing project, that I needed to follow strict conventions for filenames. That could have saved several hours of work today. Also, it would have been nice to know about this clever workaround for providing alternative text for embedded videos when people use screen readers with your EPUB eBook.
5. Use Specific M4V (MPEG-4) Video Compression Settings
The “expert settings” in QuickTime Pro can be daunting, and until today I’d just left my screencasts and videos in their original formats when I embedded them in my EPUB. Elizabeth Castro painstakingly details the settings needed for EPUB video compatibility on eReaders in addition to the iPad in her $5 “Audio and Video in EPUB” eBook, however, and I followed her advice. I’m not sure if I’ll sell the enhanced / multimedia version of my eBook outside the iTunes iBookstore for purchase on devices like the Nook, but at some point I might. (Currently B&N’s PubIt website doesn’t accept enhanced EPUB books from independent publishers. They also limit the size of uploaded EPUBs to 20 MB, which is generally too small for enhanced EPUBs.)
6. Try Different EPUB Validation and Editing Tools
I learned today there are at least three different EPUB format validation standards, and it’s a good idea to use several if you’re trying to upload to Apple’s iBookStore. EpubCheck is the most well known validation tool, but I never managed to get it running locally on my Mac. I used the free, web-based version of EpubCheck on Threepress Consulting’s website to validate the “standard” (non-multimedia) version of my eBook, but it limits upload file size to just 10 MB. My standard EPUB is 7.5 MB, but my final enhanced EPUB is 225 MB.
Both EpubCheck and FlightCrew are validating EPUBs using the 2.0 standard, as of this writing. The 2.0 standard does not include videos and multimedia EPUBS. Those specifications are coming with version 3.0. For this reason, I had to disregard the errors FlightCrew turned up for my EPUB which were related to video tag elements, manifest entries, etc. Lulu.com also seems to validate EPUBs currently to the 2.0 standard, and does NOT accept enhanced/multimedia EPUBs.
Sigil lets you edit the contents of your eBook, but you cannot edit the “OPF file” which includes files on the manifest. For this reason, I found Calibre along with TextWrangler to be my software tools of choice for editing my EPUB files after exporting out of Pages.
After loading your EPUB into Calibre, right click (or control-click on a Mac) and choose “Tweak ePub” to make edits to your individual files.
Although it sounds violent, don’t be dissuaded by the name… Choose “Explode ePub” to view all the component parts within your file.
You can then use a text editor (I love TextWrangler, it’s free and powerful) to make needed text changes to your documents. You can also rename and move files as needed from the “images” and “media” folders within your EPUB. When finished, back in Calibre choose to REBUILD your EPUB.
7. Multiple ISBNs Required for Different Book Formats on iTunes
iTunes Connect requires different, unique ISBNs for each eBook you list on the store. ePubBud is one site where I’ve seen discussion forum posts discouraging the use of unique ISBNs for eBooks, but that advice can’t go for iTunes. Your need for an ISBN depends on purpose and context. If you’re not selling on iTunes, you may not need or want to buy an ISBN. ePubBud will sell you an ISBN for $5, which is far cheaper than you can buy them directly (in small quantities) in the United States from the official ISBN agency. I purchased a block of ten ISBNs several months ago, and I’m using three of them currently for “Playing with Media: simple ideas for powerful sharing.”
- One for the Amazon Kindle (.mobi & .azw) version
- One for the “standard EPUB” version
- One for the “enhanced EPUB” or “multimedia EPUB” version
This process of getting an enhanced / multimedia EPUB eBook available for sale and download on the iTunes iBookstore has been a much more tedious and involved process than I anticipated. I knew I’d learn a great deal attempting this process myself, so on that front I haven’t been disappointed! I am hopeful the process will get a bit simpler, but I suspect companies serving as “Apple certified aggregators” for EPUB eBooks will enjoy this complexity since it will bring more business their way. Most people are NOT going to want to make the file conversion and coding changes which took me HOURS today to sell their own eBooks online. The price for having someone else do this “heavy lifting” is steep, however. These companies don’t just want a one-time fee, they require by contract (and it’s not negotiable, from what I’ve found) a PERMANENT PERCENTAGE of your eBook sales forever. Consider carefully if you want to give that up before signing a contract with an aggregator. As of late July 2011, only two companies (Ingram and Libre Digital) were certified “aggregators” by Apple to handle enhanced / multimedia EPUBs.
Now that I THINK I have successfully “validated” my EPUB versions for the iBookStore, it’s time to play “the waiting game” again for awhile.
Hopefully in a week or two, both the “enhanced” and “standard” versions of “Playing with Media: simple ideas for powerful sharing” will be available for sale via iTunes. Until then and afterwards, you can purchase the standard version on Amazon and Barnes & Noble! The book has five reviews on Amazon.com so far, and I encourage you to submit your own review after reading it!
Please also consider “liking” the “Playing with Media” book page I’ve setup on Facebook. I hope to release a complimentary, interactive website within a few months which I eventually hope to make into a mobile application for media project sharing and commenting. I’m hoping it will be “4Square-ish” in some ways. Let the “playing with media” continue!
Did you know Wes has published 9 eBooks, and 1 of them is available free? Check them out! Do you use a smartphone or tablet? Subscribe to Wes' free magazine "iReading" on Flipboard!
If you're trying to listen to a podcast episode and it's not working, check this status page. (Wes is migrating his podcasts to Amazon S3 for hosting.) Remember to follow Wesley Fryer on Twitter (@wfryer), Facebook and Google+. Also "like" Wesley's Facebook pages for "Speed of Creativity Learning" and his eBook, "Playing with Media." Don't miss Wesley's latest technology integration project, "Mapping Media to the Curriculum."
On this day..
- First Android-Edited Video: WeVideo App - 2014
- Wii Remote Interactive Whiteboard for $60 - 2014
- Create iPad eBooks with Recorded Audio and Text Highlights FREE - 2013
- Mapping Media to the Common Core with iPads (Aug 2013) - 2013
- What is it about me you can’t teach? Five Simple Things to Reach Struggling Learners - 2012
- Present to 14 iPads Simultaneously with Idea Flight - 2011
- Tell A Story in 5 Photos for Educators: S'mores! - 2010
- Discouraging lecture hall technology abuse - Encouraging interactive discussions - 2009
- iPhone GS Black Screen During Calls Mystery Solved - 2009
- Digital Witness to the U.S. Presidential Inauguration - 2009