wget question

Discussion in 'Programming/Scripts' started by bschultz, Feb 19, 2008.

  1. bschultz

    bschultz Member

    I've got two questions on automating a bunch of mp3 downloads. I need to download five mp3's per week. The problem is the files are placed in a directory based on the week of the year. So, for this week, the wget syntax would be

    wget ftp://username:[email protected]/week0807/*mp3
    Next week, it will be:

    ftp://username:[email protected]/week0808/*mp3
    Is there a way to get the directory to change in my cronjob based on what week it is?

    For the second part...and I have no idea how to do this...the files I'll download all have the same name (file1.mp3, file2.mp3...) each week and I don't want to overwrite the old file. Right now, I'm renaming the files to file1.mp3.old and file2.mp3.old, and then downloading the new files.

    Is there a way to only rename the old files if the server can connect to the remote FTP site? Right now (not very often, but occasionally) the files get renamed, but then my server can't connect to the remote FTP server, resulting in broken links until I can connect to the remote site.


  2. topdog

    topdog Active Member HowtoForge Supporter

    You should be able to write a script that keeps a counter or sentry value in a file and increments it each week after the download.

    For the second part wget does not overwrite the current file unless you use the -O option, which means on download the file created will be file1.mp3.1 so your script can check that and move the old file file1.mp3 to a different name and then rename file1.mp3.1 to file1.mp3 if the download does not happen then you still have your old file1.mp3.

    I have not tested this so YMMV.
  3. bschultz

    bschultz Member

    Thanks for the help. Chalk one up for the stupid MAC, but I can't get wget to work on my company's OSX server. It logs in to the remote server, but can't find the mp3. I can however, use curl.

    I did notice, though, that curl doesn't rename the file to file.mp3.1...so do you have any ideas for curl instead of wget?

    Thanks again!
  4. topdog

    topdog Active Member HowtoForge Supporter

    Have you tried wget with --no-passive-ftp i have not used curl itself much just the php extention but i will take a look.
  5. topdog

    topdog Active Member HowtoForge Supporter

    Actually with curl you can get funky, use a temp directory in /tmp to download the stuff then check the curl return value if its an error then don't move the files from the temp location to the actual location, if there is no error condition then move the old files out and move new from the temp location.

    For inspiration you can look at the script for sanesecurity updates it uses the same logic. http://www.sanesecurity.co.uk/clamav/update_sanesecurity.txt
  6. bschultz

    bschultz Member

    no passive didn't work...still couldn't find the file. I'll look into the file you linked...THANKS!
  7. bschultz

    bschultz Member

    I looked into the file you linked...it make a little sense, but where do I set the $clam_db_dir and $filename paramaters? I don't see that in the script anywhere.
  8. topdog

    topdog Active Member HowtoForge Supporter

    $clam_db_dir is defined in find_clam_db_dir() $filename is being passed to the functions as a parameter.

Share This Page