#1  
Old 10th September 2007, 22:27
bschultz bschultz is offline
Senior Member
 
Join Date: Jul 2006
Posts: 223
Thanks: 11
Thanked 10 Times in 10 Posts
Default wget cron job

Hi. I have a bit of a problem. I need to download a bunch of mp3 on a weekly basis for my job. At work, we use a Mac OS X server. I can't get wget to work on it during a cron job. If I type it in on the Mac terminal, everything works fine.

I can however, make my Linux machine at home download the files without problem.

Here's my question. Is it possible to then take those downloaded files on my server, and send them off to another (off-site) server (that is behind a firewall)?

These can't be sent as a tar archive either...they need to be to full files.

Thanks!

Brian
Reply With Quote
Sponsored Links
  #2  
Old 11th September 2007, 09:49
Ben Ben is offline
Moderator
 
Join Date: Jul 2006
Posts: 1,029
Thanks: 7
Thanked 62 Times in 56 Posts
Default

Sure you can, but you have to configure the firewall to allow your incoming traffic on the service's port you want to send the data.
Also the firewall must forward this incoming traffic on that port to the machine you want to reach.
Reply With Quote
  #3  
Old 11th September 2007, 13:40
bschultz bschultz is offline
Senior Member
 
Join Date: Jul 2006
Posts: 223
Thanks: 11
Thanked 10 Times in 10 Posts
Default

I control the firewall, so that's not a problem...can you point me in the direction of the code I will need to use?

Thanks for the help.
Reply With Quote
  #4  
Old 11th September 2007, 16:21
Ben Ben is offline
Moderator
 
Join Date: Jul 2006
Posts: 1,029
Thanks: 7
Thanked 62 Times in 56 Posts
Default

unfortunately I can't give you "code", just some hints

Things are quite easy, at first you could check why wget is not working inside the cron. Maybe you could source your users profile,
e.g. * * * * * * . /root/.bash_profile && wget ...
This should set environment vars etc. But this file(s) differ from distro to distro, and actually I have no clue what it is in MacOS. Then you could write stdout & stderr to a file in the cron, maybe wget is telling you what its problem is:
* * * * * wget ... 2>&1 >> /temp/wget.log

If you want to upload files from @home, you have to check with which method you want to upload the files, e.g. ftp(s) (ftps != sftp, sftp is based on sshd) (needs ftp server in the company site), scp (needs sshd on the company site). Based on this you have to define a firewall rule allowing these ports from outside, if you have a static IP only from that IP, if you have a dynamic one, from all IPs or a possible range. Then forward these packets to the local IP of the machine where the accepting daemon (ftp/ssh, etc) is running.
Reply With Quote
The Following User Says Thank You to Ben For This Useful Post:
bschultz (11th September 2007)
  #5  
Old 11th September 2007, 16:41
bschultz bschultz is offline
Senior Member
 
Join Date: Jul 2006
Posts: 223
Thanks: 11
Thanked 10 Times in 10 Posts
Default

I found some stuff on Google today...just had to figure out how to word my search!... Here's what I camp up with.

Code:
cat /localdirectory/file.mp3  |ssh username:password@domain.com "cd /remotedirectory; cat > file.mp3"
That works...except it asks me for a password (which needs to be fixed if this is run as a cron) and I can't seem to specify the whole directory...just individual files (which will change names on a weekly basis).

Why the cron doesn't work on the mac? I don't know for sure other than to say, it's a M-A-C!

Last edited by bschultz; 11th September 2007 at 16:43.
Reply With Quote
  #6  
Old 11th September 2007, 19:48
Ben Ben is offline
Moderator
 
Join Date: Jul 2006
Posts: 1,029
Thanks: 7
Thanked 62 Times in 56 Posts
Default

well you could write a shellscript with a for loop, or you do it with scp:

scp ./sourcedir/* user@your.server.com:/destination/dir
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
mail_logs.php cron job causing high server loads? jameso General 15 3rd April 2007 12:18
Dynamic IP and cron job satimis Server Operation 11 19th December 2006 15:34
cron job modified after upgrading V 2.2.5 jfruano Installation/Configuration 1 21st July 2006 13:31
Scheduling a cron job Rocky Installation/Configuration 1 5th March 2006 02:06
How to disable the shell cron jobs? raymov General 3 1st November 2005 05:43


All times are GMT +2. The time now is 09:05.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.