[Brad]

Some background:

Until QuickTime 7 was released, if you'd enabled "Save movies in disk cache" in the QuickTime pane of the System Preferences, iTunes Music Store and other services would temporarily save downloaded QT video streams:

/private/tmp/###/TemporaryItems/QTPluginTemp######

If you're still using QuickTime 6.x, you'll always be able to go to this directory to get that QTPluginTemp file. You can copy that file to somewhere else, add the .mov extension, and view it whenever you want. Life is good.

Enter QuickTime 7.

One of the many changes in QuickTime 7 is how it caches data. Apple calls it a persistent cache. Now, the cache is stored on a per-user basis in the ~/Library/Caches/QuickTime directory. Also, this brings a new cache format with files created with the .QTCH extension. This new file format is similar to a standard MOV file, but it lacks certain bits of information needed to play it independently.

What does all this mean? It means you can't resort to the old tricks for saving music videos from iTMS! Bummer, eh?

Well, last night I put myself to the challenge of getting around this and I've come up with a solution. It's not pretty, but it gets the job done.

Basically, I decided to use the tcpdump command to monitor transmissions from the iTunes application and Apple's servers. iTunes uses simple HTTP communication to relay its information back and forth. So, if you capture the right data, you can get a URL that you can plug directly into Safari or your web browser of choice!

Here's what you have to do.

Open iTunes and Terminal. In the Terminal window, enter the following command. You'll be asked to authenticate. If you're on an Ethernet connection, change the en1 to en0.

sudo tcpdump -i en1 -A -s 256 | grep 'GET .*mov HTTP'

Now, navigate the iTMS until you get to a video page. When the video starts downloading in iTunes, some data should pop up in the Terminal window similar to this:

1c.....nGET /Features/y2005/m03/d31/h16/dj.tsodbomz.mov HTTP/1.1

There's the key! Now press control-c to stop listening to the network traffic. Between the GET and the HTTP is the path to the video file on Apple's server. What server is it, though? Well, it may change, but right now that server is http://a1020.phobos.apple.com/. So, putting the two together, we get the URL for download:

http://a1020.phobos.apple.com/Features/y2005/m03/d31/h16/dj.tsodbomz.mov

Whew!

I hope you'll find this useful. It's taken a lot of arduous packet-reading to find the right information for this to work. I suppose it's inevitable for this little secret to get out, but please give credit where it's due and reference this page.

[Wrao]

Very neat, helpful and fun.

Using that TCPthingamabob could you get a link to the iTunes music store front end? maybe I haven't been looking right, but it seems every link I see to the iTMS is a pay-by-the-click type advertising link. It would be nice to have a non-ad link handy.

[Brad]

Yes and no.

iTunes uses XML (not HTML) to present everything from the store. Aside from that, it uses standard GIF and PNG graphics that can be discovered in a similar manner to above. Heck, you can even get direct URLs to the song previews (they end in m4p). However, you cannot always simply access the store pages themselves via these XML files (some pages you can, though). There are several HTTP headers that you might have to send along with your URL requests in order for the servers to play along.

Among these include:

User-Agent: iTunes/4.7.1 (Macintosh; U; PPC Mac OS X 10.4)
Accept-Language: en-us, en;q=0.50
X-Apple-Validation: ########-################################
X-Apple-Tz: ######
X-Apple-Store-Front: ######
Accept-Encoding: gzip, x-aes-cbc

I've blanked out those decimal and hexadecimal values because they change based on a session ID and cookies that the iTunes application manages.

Now, I'm sure a determined individual could get a full iTMS interface copy working, but that individual is not me. That involves a bit more work than I want to attempt right now and it could break whenever Apple updates iTunes.

[ast3r3x]

Uh huh, why not just go to the trailer page, search for .mov in the HTML. Then open that address in QT. It's easier

[Brad]

"Hello, McFly!"



Music videos in iTunes. Music. Videos. In iTunes. Not movie trailers on a web site. Big difference.

[ast3r3x]

*Note to everyone.


Don't post right after waking up. Sometimes you miss things


<- Brad

[Messiahtosh]

McFly! hahahaha

God Biff was a retard...

[torifile]

Sweet! Brad, you're an evil genius.

[rasmits]

I've been using the same apple script to get Music Videos out of the iTMS since 4.5, it still works with 4.7 and Quicktime 7.

[Brad]

Really? What does your script say? Where did you get it?

On Macs I've tried with 10.3.9 and 10.4, QuickTime 7 no longer saves to the /private/tmp/ directory.

[Paul]

Hmmm... I'd be interested in seeing that applescript as well.

Brad, have you considered submitting this to www.macosxhints.com ?

[Brad]

Yes I have, but no I'm not.

Because we're directly accessing video files on the servers, this is something I think Apple would be none too happy about. If hundreds of people suddenly started directly accessing the videos directly (which is easily visible in the logs by anyone with 2¢ of knowhow), I suspect Apple would notice and close up this little hole.

I rather keep it on the down-low so we can continue to take advantage of this feature.

[Paul]

Yeah, I guess a simple user agent check would foil this workflow (but there are ways to get around that as well... you don't happen to know the iTunes user agent string off hand, do you?)

[Brad]

Somebody hasn't read the thread, has he?

Look up.

[Paul]

Hah... hidden in the meaty bits...
skimming is a great skill.

[doublem9876]

I have an 800 mhz imac g4 and for some reason, downloading videos from the itunes music store has never worked for me. When I enter my password in the terminal with this command, it gives an error saying "tcpdump: BIOCSETIF: en1: Device not configured." My video downloading still dosn't work. Could someone do me a HUGE favor and download the large video for "Dragostea Din Tei" and send it to me? It's on the second page of the music videos section. If you could that would be ahuge favor. I'll give you my e-mail or IM address if you tell me you can. Thank you. Also, has anyone else encountered the same problem that I have?

[BuonRotto]

Originally Posted by Brad Open iTunes and Terminal...

Oh, God, I'm in way over my head!

[Brad]

Originally Posted by doublem9876 it gives an error saying "tcpdump: BIOCSETIF: en1: Device not configured."

en1 is only for wireless AirPort connections. As I mentioned in the original instructions, you'll have to change that to en0 if you're on a wired Ethernet connection.

[rasmits]

once I have the address, how do I save the video to my hard drive?

[Brad]

In Safari, you can paste the URL (the whole thing, including the beginning "http://") into the download window. That will start downloading it. This actually works with any URL you paste into it. It's rather a hidden feature.

Very neat. Much appreciated!

[rasmits]

Originally Posted by Brad In Safari, you can paste the URL (the whole thing, including the beginning "http://") into the download window. That will start downloading it. This actually works with any URL you paste into it. It's rather a hidden feature.

Oh cooool, thanks.

That'll help with other things too, I was wondering about that...

How do you find out the name of the server you want to download from? For this case it is a1020.phobos.apple.com. I want to save streaming media from a non-apple website, and this told me the path to the file, but I don't know what server it's on.

[Brad]

That's a bit trickier and may require listening to different packets. The level of difficulty depends much on the protocol and application being used.

As a starting place, check the tcpdump man pages. If you know the ports used by the protocol, the port <port>, src port <port>, and dst port <port> primitives may be useful for narrowing it down.

Alternatively, you might find the MacSniffer program useful if you are more comfortable with GUIs.

[Yochanan]

How exactly did you find out what server the file was on? I'm honestly just starting out with all of this, and I find it very fascinating. I tried the a1020.phobos.apple.com, but I guess it's no longer the server being used. Everything in your instructions worked perfectly, except for the last part, and I'm pretty certain that it's the beginning of the URL that causes it to say "file doesn't exist".

So what exactly do I do to find another a1020.phobos.apple.com? Remember that I'm a beginner, so you might have to "talk down" to me.

[Brad]

Okay, I discovered an "easy" way. Try this:

sudo tcpdump -i en0 -A -s 256 | grep 'Host: '

Remember that the en0 is for an Ethernet connection and should be changed to en1 if you're on a wireless (AirPort) connection.

Open iTunes and start to load a video. Go back to the Terminal and press control-c to stop listening to the traffic.

Just now, I tried it and it returned a1619.phobos.apple.com and that works perfectly with the old instructions posted above.

Cheers.

[Yochanan]

Is it a problem that it says "no IPv4 address assigned"?

I started with an open terminal, put in the new sudo tcpdump you gave above, and the immediate answer was "tcpdump: WARNING: en0: no IPv4 address assigned" followed by "tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 256 bytes"

EDIT: Forget all that above, I left out one of your first instructions above, the difference between en0 and en1. Doh! It's okay now :-) I got: a277.phobos.apple.com

Once again thank you very much Brad. This is the first time I've used terminal for anything.

-Switched 3 weeks ago!

[Brad]

Congrats! Yeah, the terminal is a very powerful tool, often frighteningly so. Sometimes I'm still amazed at the things I can do with it. I've added a reminder about the en0/1 to that post.

[nassau]

brad, is there a way to make a http link that will place a song in the iTMS shopping cart, or buy direct? i know how to get the link for a specific song - that's easy. the rest is hard...

any clues?

[Brad]

Not that I know of. That's mostly unrelated to what I'm doing here.