Go Back   Electric Mole Forums > Community Center > Tutorials
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read

Reply
 
Thread Tools Search this Thread
Old 2007.12.28, 10:30 PM   #1
Glathannus
True Final Boss
 
Glathannus's Avatar
 
Join Date: Feb 2006
Location: United States
Posts: 2,423
Glathannus knows what you did last summerGlathannus knows what you did last summerGlathannus knows what you did last summer
Default WMA-Lossless versus FLAC (and other lossless jargon)

Somebody sent me an email asking about this, but my response got so damn long, I thought I might as well post it publicly so other people don't have to ask.

Hey, what is the difference between FLAC and Windows Media Lossless?? Which is better?
Well, since they're both truly lossless formats, you're capable of converting to the exact same WAV/AIFF/RAW file from either of them, and during playback they should sound the exact same as each other. What makes one lossless format different from another is never the quality, but always the utility. FLAC is widely considered to be more useful than WMA-Lossless because it decodes/playbacks so much smoother when you're doing anything other than realtime, like jumping to a specific point within the song, or converting to some other type of file.

WMA-Lossless files are faster to create, but slower to use. The old FLAC argument is that you encode your lossless file only once, but you decode/playback it many times, so if you can take longer to create a faster-loading file, then the extra time 'spent' early on is actually 'saved' over the long run. I know that sounds counter-intuitive in a world where different compression levels of ZIP or RAR will extract faster if they took less time to make, but audio-specialized compression is very different in that WMA-Lossless or Apple Lossless cut some corners to get a file as small FLAC but in less time. FLAC will take longer to compress because it's creating separate chunks within itself, for each of the different split seconds throughout the entire song. Apple Lossless and WMA-Lossless work with larger chunks, which means you can't hear Point M within a song, without the player also having to quietly load Points L and N first, and that creates noticable delays. Monkey's Audio (another lossless format) is the biggest abuser of this sort of thing, and it squeezes an overall song even further... in the same amount of time, and your ability to convert the song or seek to a specific point within it, are terrible, and arguably not worth the extra 'savings' on storage space.

WavPack (my lossless format of choice) tries to be the best of both worlds, between the decoding speed of FLAC, versus the smaller filesize of Monkey's Audio. It doesn't really beat them both, but I'd say it's a silver medalist in both fields, which arguably makes WavPack a more.... balanced format. WavPack is also the king of 96kHz/24bit, in that no other lossless format has as much Compression Size versus Decoding Speed efficiency for audio of that scale, while other lossless formats unofficially specialize in 44.1kHz/16bit even if they allow 96kHz/24bit.

The thing that's so important about decoding speed (aside from what I mentioned so far), is it affects battery life on portable players. If a format is 'faster'/'smoother', that means on the flipside, it is taking less processing for normal playback. Less processing ends up costing you less power. Lossless formats in general, are energy hogs on portable players, but FLAC is "the lesser of many evils". But no matter which one you use, it's supposed to be effectively the same as having a portable CD player, only it's more compact and more convenient.

The other downside to WMA-Lossless is that it is commonly associated with Digital Rights Management (DRM). That is to say, songs you have ripped or bought, are exclusively locked to a specific computer or a specific player. It doesn't have to be that way - you potentially can create WMA-Lossless files that function on other computers or players without any bullshit, but when you're using convenience tools from Microsoft to perform the rip, they rarely or never give you that privilege. When you have to "Synchonize" songs to your player, there is almost always some degree of DRM going on, and even if you don't have to use a special interface, but can do your own drag & drop or copy & paste, if your portable player shows up in My Computer as anything other than a drive letter (E, F, etc), then Windows is quietly implementing DRM in the background, and that's what "Plays for Sure" is all about. This problem isn't exclusive to WMA-Lossless though - Microsoft has ways of quietly encrypting any format they 'support', including MP3, and this happens on nearly any non-iPod player you can buy at retail stores nowadays - not just Zunes. WMA-Lossless is a commonly DRMed format not because it was specifically designed to be, but because Microsoft loves DRM in general, and in most cases where you'd ever be using WMA-Lossless, you're using Microsoft tools and playing by Microsoft's rules.

The common dilemma with WMA-Lossless is that depending upon your situation, you might end up feeling the same way about it, that some people do about the opposite gender - can't live with it, and can't live without it. FLAC being a better format, is mostly irrelevant if you're stuck using whichever player(s) Apple or Microsoft wants you to use. The best damage control is to make informed decisions from the very foundation, and base your choice of player on the format, rather than base your choice of format on the player. You can live outside of those two corporate spheres of format influence if you use a software player like foobar2000, and a portable player from a company like iAUDIO (or a Rockboxed iPod) - and it's truly liberating, rather than simply trading one brand of bullshit for another. There's also cog for OSX, though I'm not sure that it supports APE tags.

If you should ever decide to try out noncorporate lossless formats, you might be bewildered to stumble upon different compression levels. FLAC has 9 levels that compared to each other, work much like the different degrees of ZIP/RAR compression & decompression. WavPack also has different settings, but instead of numbers, they have names like "Fast", "Normal", "High", etc.. Monkey's Audio also has different compression amounts, and it's highest option is called "Insane", pretty much because you'd have to be insane (in a misinformed way - not a taking things overly seriously kind of way) to actually use it. The problem with Monkey's Audio is that if you're not using Insane for the extra space savings, then there's really no point in using it (in-place of other lossless formats) at all, so that's practically the only Monkey's Audio option that people ever choose, and most folks don't understand the decoding price they pay on the storage 'savings'.

In the end, the 'better' lossless format really depends upon how you intend to use it, since they're each oriented in different ways.

P.S. Even if you choose the right format, it still doesn't necesssarily mean you're ripping CDs properly (I'd tell you to Google around for "Offset Correction" - but the ideal tutorial for it is now gone). Lossless perfectly preserves whatever you feed into it - which most of the time is imperfect to begin with. CDs may be digital, and people think digital copying is automatically perfect, but Audio CDs in particular, have a sloppy standard that introduces little-known inconsistencies on ripping and burning, even on brand new flawless-looking discs. If your CD/DVD drive misperceives any part of an album, this gets passed onto whichever lossless format you're using.

There's the additional Murphy Law that "Everything is always in the last place you look", but I extend that thought into "People will stop looking for something after they think they've already found it". Just keep in mind that lossless audio compression alone, isn't the complete situation of perfect CD ripping. I'll post more about Offset Correction later. Someday I'd like to put together a list of raw CRC/MD5 checksums you get when you rip a Ringo/Jihen CD with a proper setup, so you'll know whether or not your calibration was right.
__________________
You know Tokyo Jihen is a supergroup, when you can't blame most of the members for wanting to pursue other projects.
Glathannus is offline   Reply With Quote
Old 2007.12.29, 12:49 AM   #2
HEDOfloe
Senior Member
 
HEDOfloe's Avatar
 
Join Date: Mar 2007
Location: New Jersey
Posts: 2,671
HEDOfloe puts considerable thought into their posts
Default

Thanks for the info!!

I am definitely tied to Microsoft since I own a Zune and can only play WMA lossless, but I do not have any problems with DMR on anything I've ripped. I've been able to play my own files on other computers with no problem.

I definitely notice the slower speed of WMA lossless when I play it on my Zune. Zune does not feature gapless playback, but now that it allows you to add lossless to it, the gap between Yukiguni and Kabuki seems oh-so much longer, it kills the flow even more than before.

What do you mean that the "better" format depends on "how you intend to use it"? If you can only use Apple or Windows Lossless then those are the best for you, so is that what you mean by it? Because you really did not mention many things wrong with FLAC so I assume that if your player can support it, it would be the best one.
HEDOfloe is offline   Reply With Quote
Old 2007.12.29, 02:16 AM   #3
Glathannus
True Final Boss
 
Glathannus's Avatar
 
Join Date: Feb 2006
Location: United States
Posts: 2,423
Glathannus knows what you did last summerGlathannus knows what you did last summerGlathannus knows what you did last summer
Default

When a portable player supports FLAC, you get the longest battery life possible with lossless, as well as the shortest gaps - if there's even any gaps at all. Ideally the player natively supports FLAC, because I hear Rockbox has a lot of 'overhead' cost on battery life, independently of whether or not you even use lossless.

I don't really judge formats by which companies do or don't support the format. I judge a format solely by how effective it is at saving me space (compared to having WAVs), as well as how quickly it loads. I've cut it close on running out of space, way too many times - so every bit helps, but at the same time, if all I cared about was space, I'd still be using Monkey's Audio (that was actually the first lossless format I got into), and for half a second practically my whole system would lock up whenever I wanted to jump to a specific point within a song. And it would take ten minutes to uncompress a full album, versus one minute for WavPack, and maybe forty seconds for FLAC.

So when I say that the 'best' format depends upon how you intend to use lossless - this has nothing to do with company support. If all you want to do is back up your whole CD collection, and never actually play these backup files you ripped, then Monkey's Audio wouldn't be a bad choice for stashing. If you're using no-nonsense portable players, or you convert to copies in other filetypes all the time, then FLAC is the best choice. If you have a huge collection of music (and a limited amount of space) that you regularly listen to, and casually convert to other filetypes, then WavPack is the best choice.
__________________
You know Tokyo Jihen is a supergroup, when you can't blame most of the members for wanting to pursue other projects.
Glathannus is offline   Reply With Quote
Old 2007.12.29, 02:09 PM   #4
HEDOfloe
Senior Member
 
HEDOfloe's Avatar
 
Join Date: Mar 2007
Location: New Jersey
Posts: 2,671
HEDOfloe puts considerable thought into their posts
Default

I just noticed that I have a "Transcoded files cache" under my "Zune" directory and it is taking up 3.7 gigs!! all the files in it are .tmp, and I don't know what they are for. Does this have anything to do with playback, or syncing? Is it OK if I delete these?
HEDOfloe is offline   Reply With Quote
Old 2007.12.30, 01:49 AM   #5
Glathannus
True Final Boss
 
Glathannus's Avatar
 
Join Date: Feb 2006
Location: United States
Posts: 2,423
Glathannus knows what you did last summerGlathannus knows what you did last summerGlathannus knows what you did last summer
Default

I don't know anything about that. I would Google around for it, if I were you. I'm too accustomed to the dying breed of players that act as true portable hard drives (with drive letters - otherwise Microsoft does sneaky stuff in the background), where you can directly move files onto the player without a special program, and the files are still untouched and unchanged.

Actually now that you mention it, a colleague recently told me that Zunes only pretend to support the lossless form of WMA. When you move WMA-Lossless onto the player, supposedly it creates a high-end lossy form of WMA (on-par or better than 320kbps MP3) in-place of the lossless. That's probably what these tmp files are related to.
__________________
You know Tokyo Jihen is a supergroup, when you can't blame most of the members for wanting to pursue other projects.
Glathannus is offline   Reply With Quote
Old 2007.12.30, 08:40 AM   #6
HEDOfloe
Senior Member
 
HEDOfloe's Avatar
 
Join Date: Mar 2007
Location: New Jersey
Posts: 2,671
HEDOfloe puts considerable thought into their posts
Default

Hmm, I wouldn't doubt that the Zune does that, but it has to be at least better than a 320 kbps mp3. When I first heard a song on "lossless" on it, I was really surprised so I can't be that close to 320 kbps, although I also had my doubts about how lossless it really was. But, it does keep the files the same size and never converts lossless files when you sync, so do you think there is still a possibility that it doesn't play a truly lossless file?
HEDOfloe is offline   Reply With Quote
Old 2008.09.01, 04:40 PM   #7
Ringo~Bingo
Banned
 
Join Date: Nov 2006
Posts: 3,222
Ringo~Bingo knows what you did last summerRingo~Bingo knows what you did last summer
Default

I've decided to go the Wavpack route too!! being all Apple Lossless is now pretty restrictive and all I can do with them is re-encode to mp3 if I want them on anything else.

I dl'ed Wavpack but don't know what to do I have 4 programs that when clicked open a CMD window for a split second ? please help me!! I just wanna rip my CD's to a versatile lossless format!
Ringo~Bingo is offline   Reply With Quote
Old 2008.09.01, 08:56 PM   #8
Glathannus
True Final Boss
 
Glathannus's Avatar
 
Join Date: Feb 2006
Location: United States
Posts: 2,423
Glathannus knows what you did last summerGlathannus knows what you did last summerGlathannus knows what you did last summer
Default

WavPack files are best created from an official commandline encoder behind a GUI. Doesn't mean you have to manually input the commands for each individual file, because the GUI will only ask you for the parameters once ever (during setup). If you tried ripping with a program like dBpowerAMP (extremely easy to setup), it won't offer you WavPack's 'extra' option. If you like giving your computer something to do, you can have WavPack's commandline encoder spend 6x the usual amount of processing, to reduce an overall album by a few more MBs - with no further impact on sound quality or decoding speed.

Use FLAC if you're impatient. There's really not much 'wrong' with it, and if you have a Rockboxed player (or one of the elite Cowons), FLAC is going to give you the most battery life. If you own a lot of CDs and constantly find yourself on the verge of running out on hard drive space, then WavPack is a significant help, and without cutting corners like Monkey's Audio does.

My rips take about a half hour before I can start listening to them. That's mostly compression-related, and that's with two fairly recent CPU cores, each compressing different tracks simultaneously! In this era of on-demand entertainment, the average person doesn't want to wait more than 3-5 minutes before they can start listening to their own rip, because they don't care what the sound quality is, or how efficient the compression is, or whether or not the ripping process was jumping to conclusions about the 1s and 0s the CD/DVD drive thought it was reading.

WavPack's commandline encoder isn't actually very difficult to setup with Exact Audio Copy, but Offset Correction (another configuration that all lossless enthusiasts should go through) is huge pain in the ass if you've never done it before, and it's independent of which format you happen to be using. Almost every disc drive is different about how they rip CDs, even if the drives are the same speed and the same manufacturer. If you had a first or second generation PlexWriter Premium (my favorite drive for CD ripping), I could walk you through the full setup in half a minute. Otherwise if you don't know anything about the drive(s) you own (plus not all drives are created equal when it comes to reliable ripping), and if I don't know anything either, and if you don't have any special Test Discs... my WavPack tutorial would basically be walking you through how to do consistent compression/extraction off of unreliable CD readings (which merely preserves the problem with most people's CD readings rather than fixes it). If you're gonna spend the half an hour on crazy compression, you'd have a certain moral obligation to ensure that whatever raw audio you're feeding into WavPack, was already perfectly-read to begin with.

For any onlookers of this thread who think I'm speaking a bunch of audiophile hyperbole - my checksums don't lie.

This is what OSCA looks like when different drives are calibrated properly:

EAC extraction logfile from 14. July 2007, 19:47 for CD
Tokyo Jihen / OSCA

Used drive  :         16X52X32X52COMBO   Adapter: 1  ID: 0
Read mode   : Secure with NO C2, accurate stream, disable cache
Read offset correction : 738
Overread into Lead-In and Lead-Out : Yes

Used output format : D:\Applications\Exact Audio Copy\WavPack\wavpack.exe   (User Defined Encoder)
                     320 kBit/s
                     Additional command line options : -hh -l -m -t -w "artist=%a" -w "tracknumber=%n" -w "title=%t" -w "album=%g" -w "genre=%m" -w "date=%y" -x6 %s %d

Other options      : 
    Fill up missing offset samples with silence : Yes
    Delete leading and trailing silent blocks : No
    Installed external ASPI interface


Track  1
     Filename O:\Music\Shiina Ringo & Tokyo Jihen\2007.07.11 - OSCA (CD) [Maxi-Single]\01. OSCA.wav

     Pre-gap length  0:00:02.00

     Peak level 95.9 %
     Track quality 100.0 %
     Test CRC CB47BAF1
     Copy CRC CB47BAF1
     Copy OK

Track  2
     Filename O:\Music\Shiina Ringo & Tokyo Jihen\2007.07.11 - OSCA (CD) [Maxi-Single]\02. Pinocchio.wav

     Pre-gap length  0:00:02.68

     Peak level 95.0 %
     Track quality 100.0 %
     Test CRC 3D1B3B7E
     Copy CRC 3D1B3B7E
     Copy OK

Track  3
     Filename O:\Music\Shiina Ringo & Tokyo Jihen\2007.07.11 - OSCA (CD) [Maxi-Single]\03. Kaban no Nakami.wav

     Pre-gap length  0:00:00.45

     Peak level 94.1 %
     Track quality 100.0 %
     Test CRC AE6A1FFD
     Copy CRC AE6A1FFD
     Copy OK

No errors occured


End of status report
EAC extraction logfile from 14. July 2007, 20:25 for CD
Tokyo Jihen / OSCA

Used drive  : PLEXTOR CD-R   PREMIUM   Adapter: 1  ID: 1
Read mode   : Secure with C2, accurate stream, NO disable cache
Read offset correction : 30
Overread into Lead-In and Lead-Out : Yes

Used output format : Internal WAV Routines
                     44.100 Hz; 16 Bit; Stereo

Other options      : 
    Fill up missing offset samples with silence : Yes
    Delete leading and trailing silent blocks : No
    Installed external ASPI interface


Track  1
     Filename O:\Temp\Test Rip\01. OSCA.wav

     Pre-gap length  0:00:02.00

     Peak level 95.9 %
     Track quality 100.0 %
     Test CRC CB47BAF1
     Copy CRC CB47BAF1
     Copy OK

Track  2
     Filename O:\Temp\Test Rip\02. Pinocchio.wav

     Pre-gap length  0:00:02.68

     Peak level 95.0 %
     Track quality 100.0 %
     Test CRC 3D1B3B7E
     Copy CRC 3D1B3B7E
     Copy OK

Track  3
     Filename O:\Temp\Test Rip\03. Kaban no Nakami.wav

     Pre-gap length  0:00:00.45

     Peak level 94.1 %
     Track quality 100.0 %
     Test CRC AE6A1FFD
     Copy CRC AE6A1FFD
     Copy OK

No errors occured


End of status report
This is also what my MD5s (of the WAVs) look like:

746a84dcae7b6fc32cc988da65316c58 *01. OSCA.wav
13d1101e145dde117c8fef0ea0b4d730 *02. Pinocchio.wav
4cf96a0a49fb2406bb287661220b3656 *03. Kaban no Nakami.wav
Now if I were to try ripping that same CD with the same drives uncalibrated, or the applied Read Offset swapped between those drives, I would end up with different CRCs, and different MD5s. Let's see what happens if I set my Plexwriter Premium to a Read Offset of 29 instead of 30...

EAC extraction logfile from 1. September 2008, 19:42 for CD
Tokyo Jihen / OSCA

Used drive  : PLEXTOR CD-R   PREMIUM   Adapter: 1  ID: 1
Read mode   : Secure with C2, accurate stream, NO disable cache
Read offset correction : 29
Overread into Lead-In and Lead-Out : Yes

Used output format : Internal WAV Routines
                     44.100 Hz; 16 Bit; Stereo

Other options      : 
    Fill up missing offset samples with silence : Yes
    Delete leading and trailing silent blocks : No
    Installed external ASPI interface


Track  1
     Filename O:\Temp\Test Rip\01. OSCA.wav

     Pre-gap length  0:00:02.00

     Peak level 95.9 %
     Track quality 100.0 %
     Test CRC F54EF7C0
     Copy CRC F54EF7C0
     Copy OK

Track  2
     Filename O:\Temp\Test Rip\02. Pinocchio.wav

     Pre-gap length  0:00:02.68

     Peak level 95.0 %
     Track quality 100.0 %
     Test CRC 88DB5BA2
     Copy CRC 88DB5BA2
     Copy OK

Track  3
     Filename O:\Temp\Test Rip\03. Kaban no Nakami.wav

     Pre-gap length  0:00:00.45

     Peak level 94.1 %
     Track quality 100.0 %
     Test CRC 12703BA4
     Copy CRC 12703BA4
     Copy OK

No errors occured


End of status report
b1bcbe5ff06d96ab4bd097a910bc0ffb *01. OSCA.wav
64e0bf18edca3ea6de1a7721db829504 *02. Pinocchio.wav
94dbbe659b22273fe4486a3a2dda5823 *03. Kaban no Nakami.wav
Now if you understand how checksums work, then you'd know that if checksums (especially independent MD5s of the WAVs) don't match, then neither do the files they represent! If two different CD drives and/or Offset values do not yield the same checksum, then they do not yield the same rip! Different drives left alone at a Read Offset of '0' (the default behavior of pretty much all rippers with or without the option), will commonly yield different checksums from the same CD than other drives left at '0', because they need different correction values.

It's like firing a gun that has kick to it - you can preemptively aim the gun in the opposite direction by an amount of degrees equal to the kick, knowing that when you pull the trigger, the gun will be on-target. Every gun model is different. The Read Offset value is basically your degree of counter-aim. That's how drives behave on CD ripping, at least with audio CDs - this problem doesn't happen with data CDs, because they contain the mapping information (beyond table-of-contents) that lets the drive know exactly where it's reading. The official Audio CD standard cut some corners on data integrity in order to fit more possible minutes onto a disc. Even if a particular disc doesn't use those extra possible minutes, it still has to conform to the official sloppy standard, or else the players probably won't read it.

You cannot just simply follow my example by punching in '738' or '30', in an attempt to get the same rip as me - not unless you had the same drive model(s), because almost every drive is different! Then even after you know your particular drive's Offset values, if your drive is not capable of overreading into Lead-In and Lead-Out (the vast majority of drives can't - most of the ones that can are older Plextors), there are a small handful of CDs you still won't be ripping properly, and your drive won't even know what it's missing, so neither will you!

This problem is not simply solved by using a ripper that reports nothing but the most obvious problems - that's eliminating the symptoms rather than the disease. You need a ripper that can not only puke (declare the most critical errors), but also cough and sneeze, otherwise you are basing the reliability of your rips (before lossless/lossy compression is applied), solely on the presence or absence of puking. That's a rather blunt and careless regard toward a process that has varying degrees of subtleties.

I implore any of you with the OSCA CD (especially first-press), to take your ripper-of-choice (whether it offers logs or not), and rip all three tracks the same manner you'd rip anything else, except rip them as WAVs this time. Then check the MD5s on those WAVs, using one of dozens/hundreds of MD5 scanners available for download (it's a standardized process - so different scanners will yield the same MD5s from the same files - you can go ahead and check that too). Chances are your MD5s won't match mine, and that means your rips aren't 100% the same as what is on the original disc and how.

I could try to use other discs as examples, but OSCA is probably the best example because it's new enough that everyone's CD of it will be the same (it's also one of the few discs I've bothered to rip with multiple drives already). Over the years the same album/single will be unofficially pressed differently than before, and then checksums won't match on the old pressing versus the new. It's not the most important thing to have the same checksums as someone else for the same general album or single - it is important to rip no more or less than what's actually there, on the particular pressing of the CD that you happen to have. Websites/software such as AccurateRip try to turn Offset Correction into a conformists movement, and that's not what I'm advocating here. It merely helps to have at least one CD that's the exact same as someone else who has already gone through a proper calibration. From a personal ripping standpoint (after you are somehow already calibrated) there is no 'right' or 'wrong' pressing to own even if they are different.

I will try to post logs/MD5s of Watashi to Houden if there's demand for it (because EMFers are more likely to own that than OSCA and have the exact same pressing as me), but the truth is I haven't even ripped mine yet.

And in the future I will TRY to post a full tutorial on Offset Calibration, but it's far trickier to explain than it is to actually implement, especially when everyone's drive is different and I probably don't know more about your specific drive than you do. Now that Coaster Factory is gone, I have some Sasquatch-size shoes to fill.
__________________
You know Tokyo Jihen is a supergroup, when you can't blame most of the members for wanting to pursue other projects.
Glathannus is offline   Reply With Quote
Old 2008.09.02, 04:13 AM   #9
Ringo~Bingo
Banned
 
Join Date: Nov 2006
Posts: 3,222
Ringo~Bingo knows what you did last summerRingo~Bingo knows what you did last summer
Default

EDIT: Removed original post that comprised of noob non-understanding.

Now I'm reborn as the perfect rip King!!!! better watch out or I'll MD5 you!

(I rip with a dirt cheap old school Plextor, slow but still perfect mind you)

Last edited by Ringo~Bingo : 2009.02.03 at 04:33 AM.
Ringo~Bingo is offline   Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -8. The time now is 04:00 PM.