How to Solve Duplicate File Problem on iTunes


I find that I have three or four copies show up in my itunes folder for just about every album I have. Itransferred all my CDs to itunes (Apple Lossless format) a couple of years ago. The only exceptions are recent files I added (new CDs). Is there a way to remove these duplicate references? I'm not sure why they're showing up. I have had one or two harddrive fails, and I created a backup directory for music at one point, which have have had something to do with it - but it should be a backup, not a duplicate.

Any help would be apprecaited in knowing how to remove these dupes, and on keeping proper backup going forwrd.

Am using a PC, iTunes, all Apple Lossless files. Am also open to changing to another interface as long as it's convenient. I do like the itunes remote access via iphone though..
128x128outlier
I'am glad to see this post. I am uploading my collection now. I changed my drive from the C: to an external 1tb for storage and had the same problem. Since I wasnt too far into it, I just deleted them all and started from square one. Still some of them are loading in dbl or tripple. No too many, but still I'd like to fix it.
I will say that the quality with lossless played through my Benchmark DAC-1 USB is tremendous. Lets see if you get your answer.
Regards, John
Are the extra albums exactly alike or does some of the cuts appear under one title and the rest under another?

I ask because iTunes thinks multiple artists on the same album should have another cover folder. There is a way to resolve this in preferences.

I'm now four weeks into working on my Mac Mini music server and still don't have it working "perfect."

Sound is another whole issue, I refuse to put it in my system until it's like I want.
There's a "display duplicate" function under the "file" pull down menu. You should then be able to "delete" unwanted songs from the library. Be careful not to delete the actual song files.

I am having the same issue. When I click on "display duplicates," it shows both the original and duplicate file--the duplicates are directly after the original. Is there a way to get it to delete all the duplicates without deleting the original as well? I am brand new to iTunes, but I am not finding it to be very intuitive so far.
Interesting, Albert. I'm moving toward the Mac Mini . . . would like a "heads up" on problems.
Thanks,
--Neil
.

first step: do you have duplicates of the files on your drive(s) or just duplicate listing of the same file in your library? Is it possible you have several versions of the same song, say an Apple lossless and an MP3?

Do a "get info" on several of the duplicates. If they are the same song but stored in different files then you need to get rid of those duplicates. That will also show the rep rate.

If you just ended up with duplicate listing of the same file (I think that would be unusual) then I would suggest starting over. Delete the iTunes library and reload the files. If you end up with duplicates at that point you surely have duplicate copies of the same files on your drive(s). Go to the artist folder on your drive for the duplicates and see if you have the same song several times, like you ripped them more than once or have them at different rep rates.

.
Neil,

What I'm attempting to do is synchronize WAV, FLAC and Apple Lossless music files into iTunes.

iTunes does not support FLAC so I have four programs that can accommodate all the music files but each work differently and not equally user friendly.

FLAC files don't play or display album art in iTunes normally and each of the other programs are different in appearance, design and quality and (so far) only one plays nice with iTunes and supports all formats up to 384.

That's before we even begin discussing the sound quality.

For ease of use I want absolute control with no monitor, so I have several programs I've experimented with to "see" my Mini with the monitor off. The best is "Remote" which is Apple software, it's slick as can be and is absolutely free to download and use.

The best remote desktop that controls everything (not just iTunes) is Log Me in Ignition which is quite a bit more complicated to download, install and use but it's capable of even running Photoshop if need.

The short answer is, right now it's looking like a runoff between Amarra Pro and Channel D Pure Music.

The user manuals on both of these is enough to make me run to my turntable :^). I do like the fact that 1000 + CDs and high res downloads fit in a couple of 2 TB hard drives I scored at Amazon for $88.00 each.

Last, it winds up being important to do some tweaks to the Mac Mini to make it perform it's best. Solid state drive, good USB cable, best Intel chip and maximum RAM. That should be no surprise to anyone who uses a computer, those facts help everything from Mail and Word to Photoshop.
Hey Albert - I'm marking the day on the calendar that you've adopted PC audio as an option! What's the rest of your digital delivery system chain?

For those who want to use iTunes with a variety of file types, including FLAC, there is one other possible solution, besides the aftermarket piggyback software that Albert suggests, that I use. Squeezeserver, which controls Squeezebox devices like the Touch, can catelogue/control both your iTunes library and a separate music folder, simultaneously, that includes most file types including FLAC. The only downsides is that, A) you are limited to using Squeezebox devices to deliver the goods to your DAC which is not necessarily a bad thing if you use Ethernet cable to network to them (even wireless is pretty damn good), and B) and this may be a deal breaker to some - I'm pretty sure even the best of those devices, the Touch and Transporter, are limited to 96/24 so if you have higher rez you'll have to find a different solution or settle for a downsampled version (buzzkill). Someone correct me if I'm wrong there. There are various controllers that work with SB devices via an iPhone or an iTouch, including iPeng and now Squeezebox makes their own free controller which is actually quite good. The later is free, while iPeng costs a small fee. I control three SB devices from my phone with those apps with the option of synchronizing any two or all three or none at all. The SB Touch has excellent jitter specs on the SPDIF output, and the AKM DAC in the Transporter is quite good and made even better by a Modwright tube output section. The DAC in the Touch is passable but not nearly as good as the one in the Transporter.

Good suggestions from Albert on MacMini configuration.

On duplicate files - check your library and see if there are actually several copies of those files, or whether iTunes is just cataloging a single copy as several copies. If the latter I believe you can rebuild your library by scrapping your existing iTunes folder (save a backup), then pointing iTunes towards your storage drive(s) to build a new library. If you start up iTunes with the "option" key (Mac) held down (not sure what PC uses), iTunes will give you the option of starting up from the music folder of your choice (this is also handy if you want to investigate another library or switch libraries). Perhaps the simplest way to do a check for actual duplicate files on your drive(s) is to choose any song or album that registers more than one copy and do a simple search on that title on the drive(s) that store your music. If it comes up only once in that search then it is an iTunes cataloging issue, and should be solved by rebuilding your library. If you do that you might want to keep a copy of your current iTunes folder as a backup just in case.

Good luck!
Hi Guys
Thanks for the help so far, although none of you solved the problem for me, yet ;-)

All the duplicate files I have are identical to one another (same bit-rate, same size, and even same exact time and date of "date modified"). I get this info by hitting the 'get info' menu.

I'm aware of the "Display Duplicates" action, but it does not appear to be useful. If I use display duplicates, it seems to show me my four versions of the track - am not sure how that's useful when just abotu all my tracks have between two and four duplicates. The ony ones that do not are ones I've added recently.

Do I have to manually go and delete all these dupes in the itunes menu. I could totally understand this if I had multiple (yet different in some way) versions of the same song, or had identical song files in more than one location, but I don't (apart from an itunes music backup folder that is not being referenced). I'm pretty shocked at how bad this experience is.

Any more advice? Thanks!
Just to add to my previous comment, the duplicate files are referencing the same location, so there do not appear to be actual multiple files.
I have had the same problem. Some of the duplicates were caused by what others have posted, but many of the duplicates are just unexplained. I gave up trying to figure it out.
I have also experienced some songs of a copied CD not showing up. Yet when I try to down load the CD again, I get the message "replace the duplicates”?

I think it's a conspiracy...
Albert, have you compared Audirvana to Pure Music or Amarra?
Maybe you could update us on a new thread when you figure this all out. I'm anxious to get on with it, but the tinkering is daunting.

So far, all I've tried, in addition to playing iTunes through my Squeezebox 3 to my DAC, is playing the HDTracks FLAC samples from a folder separate from iTunes. But, I have to change the library in the SB program to access it, then switch back to choose iTunes.

I was thinking of using the Squeezebox Touch as an interface, but, as Jax2 notes, it won't pass higher than 24/96.

Needless to say . . . I'm a bit confused.

Outlier,

I may have found a way out of this mess. I used Mackeeper (which I downloaded) to scan my external hard drive for duplicates, and then simply removed the duplicates there. Then I wiped out everything from iTunes and started over from scratch. Finally, I pointed iTunes to the external hard drive through Preferences, and when the playlist repopulated, it showed only one file per song, as it should. So far, so good...

I hope you get it worked out soon.
06-14-11: Ncarv
Albert, have you compared Audirvana to Pure Music or Amarra?
Maybe you could update us on a new thread when you figure this all out. I'm anxious to get on with it, but the tinkering is daunting.

I have Pure Music and Amarra, I'm told that Amarra is better but it's harder to implement with an iPad or iPhone with Apple Remote app.

I'm hoping for simple elegance with decent sound when everything is complete and I'm trying hard to get a combination of convenience and high performance. So far I have no idea how this will do, not until the DAC is running and all this is in my system.
Hi Onhwy61, the duplicates do not show the same play count - if I play a music file, just that 'version' shows the additional play count.

Waltersales, thanks for the tip although I think that software is only for the Mac, and I'm using itunes on a PC. Also, I'm really not keen on uploading all my music again as it's quite an undertaking - am hoping to somehow bulk remove the duplicate references. Worst case, I'll just ctrl-c the dupes, which should at least be faster than uploading all my music again.

I'm amazed that aa mature and hugely popular software like itunes can be so flakey. The interface and other features have a lot to be desired too.
I used iTunes for the first year or so while the whole music server thing was in it's early stages. During that time my music library became an unpleasant mess that I am still trying to clean up several years later.

Every time an update came out or some other event occured that would result in a full rescan of my library initiated by itunes, it created doubles and triples of every song I own.

In my opinion if you are just starting to dip your toes into the PC based audio world you may save yourself a whole lot of headache (and hard drive space) later if you use something other then iTunes now.

I have since moved to the squeezebox software and transporter in my setup and being able to control your own folder structure within your music library is a huge advantage that can't be ignored. Duplicates are thing of the past now plus I am not locked into Apples proprietary music format which holds MY music library by the sac
I've been using iTunes since version 1 and have never had a duplicate file problem. I've always run it on a Mac platform and I'm currently using versions 9 and 10. Is the duplication issue something specific to Windows?

It's mandatory to have backups of your iTunes library and music files before implementing a software upgrade. If you cannot undo it, you shouldn't do it.
Go to www.gnu.org/copyleft/gpl.html
The software (old DOS interface) has a itunes_remove_duplicates action.

My version 10 generates duplicates all the time

Ken
Very intersting post -thanks for posting duplicate remover utilities. Will try them

I have the same issue on my Mac powerbook using iTunes. Seems t have come about through backups and changing libraries (I tried using an NAS for awhile, gave up.) I wanted to get most of my music off my laptop. Changing libraries every time was a pain, and it was hard to rip a cd to the right library. I now have 3 libraries on the laptop, keep trying to consolidate, and it's like whack a mole.

In reaction, ive been experimenting with Vortexbox (www.vortexbox.org) and love it. It's Linux based, and includes squeezebox server. You can use an old PC and create a nice music server for free in about 15 minutes. Can also work as an NAS for iTunes.

Eventually Iget a higher end vortex box based server like Sonore and just use MOG and streaming radio on the laptop.
iTunes thinks multiple artists on the same album should have another cover folder. There is a way to resolve this in preferences.

How, exactly? I do not have the duplicate file problem, but the breaking up albums drives me crazy.

Glad your foray into computer audio is progressing, Albert.

John
.

When you do a GET INFO for an album there is a space for ARTIST and another for ALBUM ARTIST. Most of the time whoever loads the data for an album with multiple artists puts the names in the ARTIST field.

I have a Richard Thompson box set that on some songs the artist is just him, some listed with and his ex-wife, and some with bands he was in. If you sort by ARTIST you get multiple albums, but if you put just his name in the ALBUM ARTIST field and sort by ALBUM ARTIST then it shows as one album.

To sort by ALBUM ARTIST cick on the ARTIST column and it will change.

Also useful to clean up the artist list if you have someone who records with a lot of different people. I have a bunch of Itzach Perlman albums that have him with another artist or orchestra. By putting his name in the ALBUM ARTIST field it is listed as one artist

.