User Name
Password
AppleNova Forums » Apple Products »

Saving iTunes Music Store Videos (new info for QT7)


Register Members List Calendar Search FAQ Posting Guidelines
Saving iTunes Music Store Videos (new info for QT7)
Page 1 of 2 [1] 2  Next Thread Tools
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-06, 03:36

Some background:

Until QuickTime 7 was released, if you'd enabled "Save movies in disk cache" in the QuickTime pane of the System Preferences, iTunes Music Store and other services would temporarily save downloaded QT video streams:

Code:
/private/tmp/###/TemporaryItems/QTPluginTemp######
If you're still using QuickTime 6.x, you'll always be able to go to this directory to get that QTPluginTemp file. You can copy that file to somewhere else, add the .mov extension, and view it whenever you want. Life is good.

Enter QuickTime 7.

One of the many changes in QuickTime 7 is how it caches data. Apple calls it a persistent cache. Now, the cache is stored on a per-user basis in the ~/Library/Caches/QuickTime directory. Also, this brings a new cache format with files created with the .QTCH extension. This new file format is similar to a standard MOV file, but it lacks certain bits of information needed to play it independently.

What does all this mean? It means you can't resort to the old tricks for saving music videos from iTMS! Bummer, eh?

Well, last night I put myself to the challenge of getting around this and I've come up with a solution. It's not pretty, but it gets the job done.

Basically, I decided to use the tcpdump command to monitor transmissions from the iTunes application and Apple's servers. iTunes uses simple HTTP communication to relay its information back and forth. So, if you capture the right data, you can get a URL that you can plug directly into Safari or your web browser of choice!

Here's what you have to do.

Open iTunes and Terminal. In the Terminal window, enter the following command. You'll be asked to authenticate. If you're on an Ethernet connection, change the en1 to en0.

Code:
sudo tcpdump -i en1 -A -s 256 | grep 'GET .*mov HTTP'
Now, navigate the iTMS until you get to a video page. When the video starts downloading in iTunes, some data should pop up in the Terminal window similar to this:

Code:
1c.....nGET /Features/y2005/m03/d31/h16/dj.tsodbomz.mov HTTP/1.1
There's the key! Now press control-c to stop listening to the network traffic. Between the GET and the HTTP is the path to the video file on Apple's server. What server is it, though? Well, it may change, but right now that server is http://a1020.phobos.apple.com/. So, putting the two together, we get the URL for download:

Code:
http://a1020.phobos.apple.com/Features/y2005/m03/d31/h16/dj.tsodbomz.mov
Whew!

I hope you'll find this useful. It's taken a lot of arduous packet-reading to find the right information for this to work. I suppose it's inevitable for this little secret to get out, but please give credit where it's due and reference this page.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Wrao
Yarp
 
Join Date: May 2004
Location: Road Warrior
 
2005-05-06, 03:46

Very neat, helpful and fun.

Using that TCPthingamabob could you get a link to the iTunes music store front end? maybe I haven't been looking right, but it seems every link I see to the iTMS is a pay-by-the-click type advertising link. It would be nice to have a non-ad link handy.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-06, 03:55

Yes and no.

iTunes uses XML (not HTML) to present everything from the store. Aside from that, it uses standard GIF and PNG graphics that can be discovered in a similar manner to above. Heck, you can even get direct URLs to the song previews (they end in m4p). However, you cannot always simply access the store pages themselves via these XML files (some pages you can, though). There are several HTTP headers that you might have to send along with your URL requests in order for the servers to play along.

Among these include:

User-Agent: iTunes/4.7.1 (Macintosh; U; PPC Mac OS X 10.4)
Accept-Language: en-us, en;q=0.50
X-Apple-Validation: ########-################################
X-Apple-Tz: ######
X-Apple-Store-Front: ######
Accept-Encoding: gzip, x-aes-cbc

I've blanked out those decimal and hexadecimal values because they change based on a session ID and cookies that the iTunes application manages.

Now, I'm sure a determined individual could get a full iTMS interface copy working, but that individual is not me. That involves a bit more work than I want to attempt right now and it could break whenever Apple updates iTunes.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
ast3r3x
25 chars of wasted space.
 
Join Date: May 2004
Send a message via AIM to ast3r3x  
2005-05-06, 05:21

Uh huh, why not just go to the trailer page, search for .mov in the HTML. Then open that address in QT. It's easier

Last edited by ast3r3x : 2005-05-06 at 09:13.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-06, 09:06

"Hello, McFly!"



Music videos in iTunes. Music. Videos. In iTunes. Not movie trailers on a web site. Big difference.
  quote
ast3r3x
25 chars of wasted space.
 
Join Date: May 2004
Send a message via AIM to ast3r3x  
2005-05-06, 09:14

*Note to everyone.


Don't post right after waking up. Sometimes you miss things


<- Brad
  quote
Messiahtosh
Apple Historian
 
Join Date: May 2004
 
2005-05-06, 09:52

McFly! hahahaha

God Biff was a retard...
  quote
torifile
Less than Stellar Member
 
Join Date: May 2004
Location: Durham, NC
Send a message via AIM to torifile  
2005-05-06, 10:39

Sweet! Brad, you're an evil genius.
  quote
rasmits
rams it
 
Join Date: Oct 2004
Location: Seattle
 
2005-05-06, 22:01

I've been using the same apple script to get Music Videos out of the iTMS since 4.5, it still works with 4.7 and Quicktime 7.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-06, 22:11

Really? What does your script say? Where did you get it?

On Macs I've tried with 10.3.9 and 10.4, QuickTime 7 no longer saves to the /private/tmp/ directory.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Paul
Veteran Member
 
Join Date: May 2004
Location: New York City (mostly)
Send a message via AIM to Paul  
2005-05-09, 21:32

Hmmm... I'd be interested in seeing that applescript as well.

Brad, have you considered submitting this to www.macosxhints.com ?
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-09, 21:52

Yes I have, but no I'm not.

Because we're directly accessing video files on the servers, this is something I think Apple would be none too happy about. If hundreds of people suddenly started directly accessing the videos directly (which is easily visible in the logs by anyone with 2 of knowhow), I suspect Apple would notice and close up this little hole.

I rather keep it on the down-low so we can continue to take advantage of this feature.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Paul
Veteran Member
 
Join Date: May 2004
Location: New York City (mostly)
Send a message via AIM to Paul  
2005-05-09, 21:55

Yeah, I guess a simple user agent check would foil this workflow (but there are ways to get around that as well... you don't happen to know the iTunes user agent string off hand, do you?)

125/71645 (top .1745%)-Amie Street - awesome (mostly free) independent DRM-free music
People really have got to stop thinking there is only one operating system, one economic system, one religion, and one business model. -EvilTwinSkippy (/.)
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-09, 21:55

Somebody hasn't read the thread, has he?

Look up.
  quote
Paul
Veteran Member
 
Join Date: May 2004
Location: New York City (mostly)
Send a message via AIM to Paul  
2005-05-09, 22:34

Hah... hidden in the meaty bits...
skimming is a great skill.
  quote
doublem9876
Senior Member
 
Join Date: May 2005
Location: In Seine
Send a message via AIM to doublem9876  
2005-05-10, 17:29

I have an 800 mhz imac g4 and for some reason, downloading videos from the itunes music store has never worked for me. When I enter my password in the terminal with this command, it gives an error saying "tcpdump: BIOCSETIF: en1: Device not configured." My video downloading still dosn't work. Could someone do me a HUGE favor and download the large video for "Dragostea Din Tei" and send it to me? It's on the second page of the music videos section. If you could that would be ahuge favor. I'll give you my e-mail or IM address if you tell me you can. Thank you. Also, has anyone else encountered the same problem that I have?
  quote
BuonRotto
Not sayin', just sayin'
 
Join Date: May 2004
Location: Durham, NC
Send a message via AIM to BuonRotto Send a message via Yahoo to BuonRotto  
2005-05-10, 18:24

Quote:
Originally Posted by Brad
Open iTunes and Terminal...
Oh, God, I'm in way over my head!
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-10, 21:59

Quote:
Originally Posted by doublem9876
it gives an error saying "tcpdump: BIOCSETIF: en1: Device not configured."
en1 is only for wireless AirPort connections. As I mentioned in the original instructions, you'll have to change that to en0 if you're on a wired Ethernet connection.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
rasmits
rams it
 
Join Date: Oct 2004
Location: Seattle
 
2005-05-13, 20:23

once I have the address, how do I save the video to my hard drive?
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-13, 20:25

In Safari, you can paste the URL (the whole thing, including the beginning "http://") into the download window. That will start downloading it. This actually works with any URL you paste into it. It's rather a hidden feature.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Captain
New Member
 
Join Date: May 2005
 
2005-05-14, 02:58

Very neat. Much appreciated!
  quote
rasmits
rams it
 
Join Date: Oct 2004
Location: Seattle
 
2005-05-14, 15:41

Quote:
Originally Posted by Brad
In Safari, you can paste the URL (the whole thing, including the beginning "http://") into the download window. That will start downloading it. This actually works with any URL you paste into it. It's rather a hidden feature.
Oh cooool, thanks.

That'll help with other things too, I was wondering about that...
  quote
jepjepjep
New Member
 
Join Date: May 2005
 
2005-05-15, 15:23

How do you find out the name of the server you want to download from? For this case it is a1020.phobos.apple.com. I want to save streaming media from a non-apple website, and this told me the path to the file, but I don't know what server it's on.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-05-15, 15:47

That's a bit trickier and may require listening to different packets. The level of difficulty depends much on the protocol and application being used.

As a starting place, check the tcpdump man pages. If you know the ports used by the protocol, the port <port>, src port <port>, and dst port <port> primitives may be useful for narrowing it down.

Alternatively, you might find the MacSniffer program useful if you are more comfortable with GUIs.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Yochanan
Member
 
Join Date: May 2005
 
2005-06-02, 04:37

How exactly did you find out what server the file was on? I'm honestly just starting out with all of this, and I find it very fascinating. I tried the a1020.phobos.apple.com, but I guess it's no longer the server being used. Everything in your instructions worked perfectly, except for the last part, and I'm pretty certain that it's the beginning of the URL that causes it to say "file doesn't exist".

So what exactly do I do to find another a1020.phobos.apple.com? Remember that I'm a beginner, so you might have to "talk down" to me.
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-06-02, 07:45

Okay, I discovered an "easy" way. Try this:

Code:
sudo tcpdump -i en0 -A -s 256 | grep 'Host: '
Remember that the en0 is for an Ethernet connection and should be changed to en1 if you're on a wireless (AirPort) connection.

Open iTunes and start to load a video. Go back to the Terminal and press control-c to stop listening to the traffic.

Just now, I tried it and it returned a1619.phobos.apple.com and that works perfectly with the old instructions posted above.

Cheers.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
Yochanan
Member
 
Join Date: May 2005
 
2005-06-02, 10:36

Is it a problem that it says "no IPv4 address assigned"?

I started with an open terminal, put in the new sudo tcpdump you gave above, and the immediate answer was "tcpdump: WARNING: en0: no IPv4 address assigned" followed by "tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 256 bytes"

EDIT: Forget all that above, I left out one of your first instructions above, the difference between en0 and en1. Doh! It's okay now :-) I got: a277.phobos.apple.com

Once again thank you very much Brad. This is the first time I've used terminal for anything.

-Switched 3 weeks ago!
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-06-02, 10:56

Congrats! Yeah, the terminal is a very powerful tool, often frighteningly so. Sometimes I'm still amazed at the things I can do with it. I've added a reminder about the en0/1 to that post.

The quality of this board depends on the quality of the posts. The only way to guarantee thoughtful, informative discussion is to write thoughtful, informative posts. AppleNova is not a real-time chat forum. You have time to compose messages and edit them before and after posting.
  quote
nassau
Member
 
Join Date: Jul 2004
 
2005-06-02, 11:56

brad, is there a way to make a http link that will place a song in the iTMS shopping cart, or buy direct? i know how to get the link for a specific song - that's easy. the rest is hard...

any clues?
  quote
Brad
Selfish Heathen
 
Join Date: May 2004
Location: Zone of Pain
 
2005-06-02, 12:21

Not that I know of. That's mostly unrelated to what I'm doing here.
  quote
Posting Rules Navigation
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Page 1 of 2 [1] 2  Next

Post Reply

Forum Jump
Thread Tools
Similar Threads
Thread Thread Starter Forum Replies Last Post
Submissions now being accepted for my first AppleNova article murbot General Discussion 55 2008-11-10 21:53
Welcome to iTunes Music Store Email in Australia Linkoln Speculation and Rumors 1 2005-01-24 17:28
iPod Evaluation Messiahtosh Apple Products 14 2005-01-18 12:36
Cool little trick (saving iTunes videos) Messiahtosh General Discussion 6 2004-11-02 16:15
How much have you spent in the iTunes Music Store? v4 Paul Apple Products 16 2004-06-10 15:32


« Previous Thread | Next Thread »

All times are GMT -5. The time now is 14:34.


Powered by vBulletin®
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright ©2004 - 2014, AppleNova