Archive for September 22, 2007

Saturday, September 22, 2007

Resuming ADC Downloads

Downloading from Apple’s developer seeding site takes a long time with a DSL connection, and when using current versions of Safari and Firefox an interrupted download means that you have to start over. It’s virtually impossible for my download to go uninterrupted long enough to get a whole DVD-sized file. Fortunately, Mac OS X includes a built-in command-line tool called curl that can automatically retry and resume downloads. Normally you can invoke it using curl -O 'URL' to download URL into the current directory. It’s slightly more complicated if you want handle cookie-authorization and URL redirection:

  1. Log in using your browser and copy the file’s URL.
  2. Use curl 'URL' -D FILE to dump the headers into FILE.
  3. Open the file and note the cookie (set-cookie: NAME=VALUE) and new URL (location: NEWURL).
  4. Now download the file using curl -O 'NEWURL' -b 'NAME=VALUE' --retry 100 -C -. This will retry and auto-resume up to 100 times.
  5. You can also add --limit-rate 100K to limit the amount of bandwidth that curl uses if you want to reserve some of your connection for other use.