Feb 19
HowTo: Encode a Blu-ray rip into a smaller format without losing quality
Posted on Thursday, February 19, 2009 in Tutorials
Those of you who archive or backup their Blu-ray movie media to hard-drive will already be aware that the average movie comes out at a good 25GB. Some of the bigger titles top out at around 40GB or more. This eats up an awful lot of disk space.
Blu-ray titles are already compressed down using the MPEG2 codec, and quality pundits will abhor the idea of re-compressing the title again for fear of losing image and audio quality. Certainly if you go down the Xvid route, you will definitely lose image quality, but as per my previous DVD HowTo, you can do excellent rips with virtually indistinguishable quality to the original using the x264 codec, and have a significantly smaller footprint to go with it.
The process of encoding a Blu-ray rip isn’t quite the same as doing a DVD, however, so here’s a quick guide on how to take your decrypted .m2ts file and finish up with a much smaller, but 99% perfect copy in a Matroska .mkv file.
Pre-requisites:
- A pre-decoded Blu-ray movie file (.m2ts file).
- Approximately the same amount of free disk space as the size of the movie file. Eg: If you have a 25GB movie file, then you should have another 25GB free space to work with. You can have less, since the final resulting file will be much smaller than the original movie anyway, but since this process can take a number of hours to complete, you don’t exactly want to discover you ran out of disk space and have to start over, do you?
- A nice powerful CPU. I use a Intel quad-core Q9450 CPU at 2.66GHz. It takes my machine roughly 9-12 hours to process just one movie using four threads. A dual-core will take longer.
- Time to let the PC do its work, eg: overnight.
This guide was written using Ubuntu 8.10 Intrepid Ibex 64-bit, but will work quite happily in 32-bit and should also work with most previous versions of Ubuntu.
- You will need some extra software installed if you haven’t already got it. Open a terminal and type in the following at the $ prompt:
$ sudo apt-get install mencoder mplayer gpac x264 mkvtoolnix(Don’t worry if you’ve already got some of those apps installed, Ubuntu will skip over them if they already exist on your system)
- Create a new text file somewhere using your favourite text editor, eg:
$ gedit ~/encodevideo.sh…will create a new text file called “encodevideo.sh” in the root of your Home directory using the GEdit text editor.
- Now copy and paste the following script into it:
#! /bin/bash # ===================================================================== # Blu-ray encoding script by HyRax February 2009 http://www.serenux.com # ===================================================================== # Make sure the user has specified what to work on. if [ -z "$1" ]; then echo "\nBlu-ray movie encoding script\n-----------------------------" echo "Written by HyRax February 2009\nhttp://www.serenux.com" echo "\nUsage: $0 <m2ts file without extension>" echo "\nExample: If your movie file is called TheDarkKnight.m2ts then\nyour usage will be: $0 TheDarkKnight\n" exit fi # The crf=21 option controls encoding quality, and indirectly the final # filesize. The higher the value, the more compression and thus smaller # file size. Reduce the value and file size will go up. A value of 21 # should produce a file of approximately 4GB in size for a typical movie. # Encode the video using x264, ignore the audio for now. mencoder $1.m2ts \ -ovc x264 -x264encopts crf=21:frameref=3:bframes=3:b_pyramid:direct_pred=auto:weight_b:partitions=all:8x8dct:me=umh:mixed_refs:trellis=1:nopsnr:nossim:subq=6:level_idc=41:threads=4 \ -nosound \ -of rawvideo \ -o $1.x264 # Dump the first original audio track (should be the English track) but # don't re-encode it. Ignore the video. mplayer $1.m2ts -dumpaudio -dumpfile $1.ac3 # Copy the raw x264 encoded video into an MP4 container so we can set # the correct framerate (generally 23.976 - adjust it if MPlayer or # MEncoder report something different) MP4Box -add $1.x264 $1.mp4 -fps 23.976 # Finally, merge everything into a single MKV file mkvmerge -o $1.mkv $1.mp4 --track-name 0:Eng $1.ac3 # Remove the hash in front of the next command to have the script delete # your working files when encoding is complete. # rm $1.m2ts $1.x264 $1.ac3 # Tell the user we're done. echo "All done! Your final movie file is called $1.mkv - enjoy!"
- Save and exit your text editor.
. - Change directory to where you have your original .m2ts file, eg:
$ cd ~/Videos/BDRips/MyMovie - Let’s say your movie file is called TheIsland.m2ts. To begin encoding it, you will use the following command:
$ sh ~/encodevideo.sh TheIsland(Notice that we don’t specify the file extension – the script assumes .m2ts on the end already)
- Hit enter and the encoding process will begin. The script does the following in order:
- Extract the video component only and encode it in a single pass to a raw x264 video file called TheIsland.x264 (in this example).
- When that has finished, go back and extract the first audio track only and save it to a file called TheIsland.ac3 without any re-encoding.
- When that has finished, take the raw x264 video data and put it into an MPEG4 container so we can fix the framerate at 23.976 frames per second, typical of most Blu-ray movies. If you don’t do this step, then the video will play faster than the audio in the final product.
- Finally we create a new Matroska container called TheIsland.mkv and in it we place the framerate-adjusted MPEG4 video track and the unmodified audio track, producing our final product.
.
- When the script has finished some 9-12 hours later, you will have a file called TheIsland.mkv ready for playback in Totem, MPlayer, VLC, or whatever your favourite player is. You will notice that the filesize is significantly smaller than the original but when you play it back, the image quality will look pretty much 100% identical to the original. About the only compression artefacts you may notice is around the edges of some text titles such as opening credits, but you’d really have to look hard to spot them.
. - We’ve finished with the working files now, so you can delete the .m2ts, .x264 and .ac3 files now to reclaim the disk space and enjoy your new .mkv file.
.
Enjoy! 
NOTES:
- While not covered by this HowTo (I’ll add to this later), you can optionally extract other audio tracks from the .m2ts file such as other languages or the commentary track, and add them to the .mkv file at any time. You can also download subtitles and add them to the .mkv file at a later date also, all without needing to re-encode the entire video again because all you’re doing is adding tracks to the existing Matroska container. see the man pages for the mkvmerge utility for more information.
.
- A high-definition x264-encoded movie takes a fairly reasonable amount of CPU power to decode. Since Ubuntu Intrepid and previous versions do not make any direct use of on-board video decoding hardware such as that found on NVidia and ATi based video cards, you may find certain busy scenes in your movie may stutter or even skip altogether during playback – this does NOT mean you are missing data from your movie file – it’s just that your PC is having trouble decoding AND displaying the movie all at once because the CPU is doing everything, especially those with slower CPU’s. Indeed you will notice that your CPU usage will probably be quite high in the 80-90% region. Ubuntu Jaunty will bring forth with it VDPAU support for NVidia-based video cards (restricted NVidia driver v180 and above). Using a patched version of MPlayer with VDPAU support, all the decoding work is passed completely to the video card, freeing up your CPU considerably to concentrate on other tasks, and dropping utilisation rates down to about 3%. This makes busy movies that much more watchable. If you can’t wait for Jaunty, there are some backported VDPAU modified versions of mencoder, mplayer and even MythTV available here, however you try them at your own risk. If you are an ATi video card user, you’re out in the dark for the moment. Go buy yourself an NVidia card – they are much better supported under Ubuntu than ATi are.
. - If you are using only a dual-core processor, you should modify the threads=4 section of the mencoder line in the script to read either threads=2 or threads=3. If you are using a single-core processor, change this to threads=1 or threads=2 depending on how your CPU performs (and while you’re at it, seriously consider an upgrade!).
. - If you wish to modify the quality of the encoded video to make the resulting file larger or smaller, re-edit the encodevideo.sh script and change the crf=21 value in the mencoder line to a different value. There is no definitive filesize that the resulting encode will have. This value simply adjusts the quality of the encode. With some trial and error, I have found that the average 1080p movie encode ends up roughly 8GB in size when using a value of 21 and provides excellent image quality. If you make the value larger, this will apply greater compression and will reduce the final file size at the expense of losing some image quality. If you reduce the value, then your final file size will increase, however your image quality will also go up. As a guide, animated movies such as those made by Pixar and Dreamworks and visually dark movies such as Underworld, compress very well. In one instance, the final product was only 3.5GB in size. Visually complex movies such as Transformers blew out to 12GB in size using the same encoding script, so you can see that there is no exact science to this. There are mencoder options to specifically set bitrate and target filesize, but I chose to ignore those options for this HowTo as I’m a bit of a quality freak, not a size freak. I store all my movies on a MythTV server at home and backup to an external drive as I’m not a fan of doubling up my movie purchases on the shelf with a second disc containing the compressed version of the same movie!
8 people like this post.
Hi,
The script fails miserably right on the first stage, please see output below:
mencoder 00011.m2ts -ovc x264 -x264encopts crf=21:frameref=3:bframes=3:b_pyramid:direct_pred=auto:weight_b:partitions=all:8×8dct:me=umh:mixed_refs:brdo:bime:trellis=1:nopsnr:nossim:subq=6:level_idc=41:threads=4 -nosound -of rawvideo -o 00011.x264
MEncoder SVN-r28450-4.1.2 (C) 2000-2009 MPlayer Team
CPU: AMD Opteron(tm) Processor 246 (Family: 15, Model: 5, Stepping: 10)
Option x264encopts: Unknown suboption brdo
Option x264encopts: Unknown suboption bime
success: format: 0 data: 0×0 – 0xa2b54000
TS file format detected.
VIDEO H264(pid=4113) NO AUDIO! NO SUBS (yet)! PROGRAM N. 1
FPS seems to be: 23.976025
[V] filefmt:29 fourcc:0×10000005 size:0×0 fps:23.976 ftime:=0.0417
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg’s libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
VDec: vo config request – 1920 x 1080 (preferred colorspace: Planar YV12)
VDec: using Planar I420 as output csp (no 1)
Movie-Aspect is 1.78:1 – prescaling to correct movie aspect.
FATAL: Cannot initialize video driver.
Ah, these opts have been obsoleted. The script has to be revised not to include the ‘brdo:bime’ opts…
Hi igor,
I notice that the version of MEncoder you’re using is very much out of date – You’re using v4.1.2 whilst I’m using v4.3.2.
I’ll check out the brdo and bime options and if necessary, I’ll update my article to reflect any changes in MEncoder’s command parameters.
Cheers!
Igor was right, the brdo and bime options have been deprecated. I’ve amended my article accordingly. There is not really much or a quality drop except for really subtly dark areas of an image, but it will likely re-justify doing a two-pass encode again.
it seems i hit another snag and dont know yet what to do with it:
MP4Box -add 00011.x264 /home/imanassypov/dvdrip/00011.mp4 -fps 23.976
AVC-H264 import – frame size 1920 x 1080 at 23.976 FPS
Error importing 00011.x264: I/O Error | (76/100)
By encoding the video with the -nosound option, you are running a very significant risk of losing A/V sync. You would probably be better off encoding them all into an AVI container and then using mkvmerge to convert into an mkv.
Igor: I’ve encountered this before – your ripped video has an error in it – MP4Box is touchy when it comes to data integrity. This is likely not to be a fault of the encoder, but of the initial rip of the Blu-ray itself.
Wes: Yes, I’ve considered that but so far I have not got any rips out of sync yet. I guess for those who are worried, they could replace the “-nosound” parameter with “-oac copy” if they are really paranoid.
Most of the sync is maintained with MP4Box anyway, setting the correct frame rate to playback at, which Totem et al will adhere to nicely. Any other sync issues are usually down to errors in the initial rip where frames are missing (eg: scratched disc, etc), and no amount of framerate tweaking will fix that.
On the last step you merge the Audio and Video to create an MKV. Would you provide instructions on how to merge to create an m2ts? I know that it was an m2ts to begin with but I am compressing the m2ts for disk space size.
Hi Ben,
There’s a handy script to convert an MKV file to M2TS right here: http://sticky123.blogspot.com/2008/03/remuxing-mkv-to-m2ts-on-linux.html
Basically it just remuxes the tracks.
Got a script to do that already
Just wanted to see if I could use something like tsMuxeR to avoid making an mkv at all.
Well, the MKV part is not a time-consuming step. After all, it’s just a container. There’s very little processing time involved with that part.
For me personally, my PS3 plays back MKV’s without a problem via “PS3 Media Server”. I don’t store movies on my PS3’s HDD at all.
You Sir, are awesome!
I’d like to know how to demux the DTS audio streams in the m2ts. The m2ts in question had an English HD DTS and a Spanish standard audio.
I tried fiddling with the mplayer options to make it select the English track, but I couldnt get it to demux the English track – it wither defaulted to the first non HD audio track or provided no audio altogether.
I was eventually forced to use eac3to to demux the audio into ac3 and then remux using the rest of this script into mkv.
I use my Windows VM only for eac3to, and it would be great if I could get rid of that requirement.
Any suggestions?
A tool like tsMuxeR can rip out various tracks from an m2ts file which you can then re-mux into whatever container you see fit. There is a version for Linux, but it’s not part of the Ubuntu repositories.
Note that some English tracks on some titles are deliberately setup in convoluted ways on a disc to impede your ability to make a good rip. The English track of Transformers, for instance is not AC3 or DTS at all whilst all other languages are easy to rip in usable formats. The only English track you can easily pull is the English Commentary track, which is of course useless.
HyRax,
- this procedure should preserve DTS tracks, since we are simply dumping the original track into the new container?
Well, the “dumpaudio” parameter of MPlayer isn’t really specific, though the man page refers to “compressed MPEG/AC3″, but it IS dumping the raw audio track without any processing of it. In theory then I imagine it should dump DTS tracks without a problem, but I personally don’t have any titles with DTS tracks handy to test this.
I just run a few tests – dts tracks do indeed get converted to ac3…
Thats a bugger because you loose fidelity
message to uv – you can do your demux in linux, tsMuxer has a version that runs on opensource.
The question still remains – how do people put the audio track back in as the dts? Anything I try ends up having an ac3 format in the mkv container.
How would this script change for .TS files?
MEncoder handles Transport Stream files – it’s just a container, so the script would only need to have any reference from “.m2ts” changed to “.ts” instead.
Honestly no-one should be using .ts containers anymore anyway.
When I run your script, I get these errors…
ProcessInputError r:0xffffff9c=-100 (keyframe: 1)
ProcessInputError r:0×80004005=-2147467259 (keyframe: 1)V:0.000 [0:0]
ProcessInputError r:0×80004005=-2147467259 (keyframe: 1)V:0.000 [0:0]
ProcessInputError r:0×80004005=-2147467259 (keyframe: 1)V:0.000 [0:0]
[snip]
MEncoder is having trouble reading your m2ts file. That could be because it’s corrupted or it’s in a format that MEncoder doesn’t quite understand, eg: you might not be using an original m2ts file – you may be using one remuxed together into a new file by someone else.
Are you able to play the m2ts file with MPlayer?
You may have to demux it manually first and encode the extracted video track directly. A program like tsmuxer from SmartLabs can do this for you (I’ll actually be writing a tutorial about using this tool soon).
Also, try using a more recent version of MEncoder than the one provided by Ubuntu.
I have been having problem with the english DTS-HD audio track. Your script seems to be picking the english commentary track instead I guess because it is in AAC format. Using tsMuxeR I sperated the DTS-HD audio track from the m2st file and used eac3to to convert it to AAC format, then remuxed it all back together with tsMuxeR again. When I played in VLC it seemed to play ok but when I ran your scrip it gave me many of the same errors “paully” had above. And when I played it the audio did not sync up.
This is a great script and I would love to get it working!
Hi Jeff,
I’m actually in the final stages of putting the finishing touches on a new tutorial for ripping and preserving DTS tracks and using a slightly modified method of composing MKV’s using MKVMerge without needing to put them into an MP4 container since MP4Box doesn’t seem to like every converted stream out there. The results are greater success in dealing with multi-part movies, securing the right language track and ensuring the frame rate matches properly.
Stay tuned!
HyRax
The 22Gb m2ts file is taken from disc 1 from the 5 disc USA Blade Runner : Final Cut Collector’s Edition, which has been ripped by DumpHD 0.61
Yes, it plays in MPlayer with the following command:
mplayer -fs -zoom -x 1280 -y 720 -quiet -vo xv -vc ffvc1 -fps 24000/1001 -lavdopts threads=2:fast:skiploopfilter=all -sws 0 -framedrop -ni -cache 9999 Videos/BDMV/STREAM/00047.m2ts
I am a complete newbie with this stuff, so your talk of “demux it manually first and encode the extracted video track directly” is new to me, but I am willing to learn.
If you do this, can you please give me an actual command-line example using the programs needed?
Thanks!
Paully
HyRax
I have been busy
and taken it upon myself to do some searching, reading and working.
I have now successfully ripped a blu-ray movie.
mediainfo, tsmuxer, mkvmerge, mplayer
Content will be added to this, but for now it’s down on something!
http://www.indie-it.com/wiki/index.php/Blu-Ray
Cheers, keep going…!
Paully
Hi Paully,
That’s a nice and complex set of arguments for MPlayer there.
You should look at putting your most-used arguments into an ~/.mplayer/config file so you don’t have to keep typing them all the time. Personally I just use “mplayer -fs mymoviefilename.blah” and my config file does everything else I need (which is simply dumping the decoding to my NVidia gfx card using VDPAU so no frameskips and bugger-all CPU usage – as smooth as playing back on a PS3 console).
As for the link you posted, yes – that is very similar to what my next guide is all about, though I don’t use MediaInfo because you can already glean a lot of info from tsMuxer itself. Hopefully I’ll have my article completed this weekend so you can compare your results.
HyRax
“dumping the decoding to my NVidia gfx card using VDPAU”
…aah, that’s how you do it. Nice.
BTW – used MakeMKV tonight for the first time (bugzilla Gentoo ebuild) and got my UK Blu-Ray disc of “I Am Legend” out to a 15Gb file. It actually did it. Amazing. This was unheard of 9 months ago…
…we are getting there!
Paully
Good stuff! It’s great when you discover something new, isn’t it?
If you have an NVidia 8xxx series gfx card or later, I highly recommend you check out getting VDPAU running. Have a look a JYA’s website here – he maintains his own repository primarily dealing with VDPAU under MythTV, but he also has recent versions of the NVidia gfx drivers and modified versions of MPlayer that utilise VDPAU for very smooth, low-CPU (<10%) playback.
http://www.avenard.org/media/Home.html
Any updates on the new guide?
Question, you said that with the mencoder quality set to 21 you get an average video size of 4 gig. My average, thus far, has been around 10 gig. I adjusted the number to 23 just to see what would happen, and the quality still looks great but the file size didn’t change all that much either. Since it takes so long to encode, any suggestions on a number to use that would get me in the 4 gig per movie range? Thanks!
Hi Jason,
I should update this article – the 4GB tends to apply to animated movies only. Other movies compression will vary. Generally i expect to bring down an average “live actor” movie to no less than 8GB. Anything less tends to have noticeable quality loss in comparison to the original Blu-ray.
Generally movies with high detail will not compress down as much. Typical examples include Batman Begins and Babel.
Personally I find quality to be more important than file size. I am considering a short article discussing playing with the crf value with examples of results it can produce, but as I said, this is an arbitrary figure that will provide different results for different movies. I could safely say, however, that I am averaging about 12GB across all my movies.
Hey, thanks for that! I’ll play around a bit more once I get it working again. (I installed Ubuntu 9.10 64 bit and apparently ffmpeg doesn’t have all the necessary codecs for encoding x264 on the 64 bit system. So now I’m going to have to compile ffmpeg on my own it seems.)
Well, turns out my problem was with VC-1 encoded BD’s specifically. I do not have the proper codec to encode them.
Any suggestions? I’m using the ffmpeg from Medibuntu.
I use MEncoder to re-encode my BD’s and I’ve never had any issues, including VC-1 discs.
[...] HowTo: Encode a Blu-ray rip into a smaller format without losing quality [...]