Computing.Net > Forums > Unix > Batch ftp download script & append

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Batch ftp download script & append

Reply to Message Icon

Name: Chris
Date: July 11, 2003 at 04:51:54 Pacific
OS: RedHat Linux 7.2
CPU/Ram: PII-400
Comment:

Hi,
I'm new to Shell scripting and I really need to code a tiny script to do the following:
1) Connect via FTP to an FTP server;
2) Download the LIST of all the files in the default folder;
3) FOR EACH file in the LIST, APPEND it's contents to the contents of a bigger file (I want the bigger file to contain all the text files' content in the end);
4) Disconnect from the FTP server.

Here's what I've managed to patch together from online tutorials and documents (it doesn't work):

#!/bin/sh
# Script to fetch the all the text files and append them into one big text file.
USER='username'
PASSWD='password'
HOST='IP'
REPORTS_DIR="DIRNAME"

# FTP to the text file server
ftp -n $HOST > /tmp/logs/textftp.log 2> /tmp/logs/textftp.log > big.txt
done
quit
SCRIPT

EXITSTATUS=$?

if [ $EXITSTATUS != "0" ]
then
# handle the error...
echo 'Failed to connect to FTP or to get the desired files.'
else
# everything's ok...
echo 'Woohoo! We generated the big file!'
fi



Sponsored Link
Ads by Google

Response Number 1
Name: Chris
Date: July 11, 2003 at 05:12:59 Pacific
Reply:

For some reason, my script got cut off:

#!/bin/sh
# Script to fetch the all the text files and append them into one big text file.
USER='username'
PASSWD='password'
HOST='IP'
REPORTS_DIR="DIRNAME"

# FTP to the text file server
ftp -n $HOST > /tmp/logs/textftp.log 2> /tmp/logs/textftp.log > big.txt
done
quit
SCRIPT

EXITSTATUS=$?

if [ $EXITSTATUS != "0" ]
then
# handle the error...
echo 'Failed to connect to FTP or to get the desired files.'
else
# everything's ok...
echo 'Woohoo! We generated the big file!'
fi


0

Response Number 2
Name: Chris
Date: July 11, 2003 at 05:20:32 Pacific
Reply:

I give up, if you could just me a sample script to do the above, please do so asap.

Thanks a million.
P.S. How does one work around the weird script security built into this forum? I can't post my script!


0

Response Number 3
Name: Chris
Date: July 11, 2003 at 05:51:04 Pacific
Reply:

Email is the best way to contact me since this forum seems insanely buggy.

THX!


0

Response Number 4
Name: WilliamRobertson
Date: July 11, 2003 at 06:24:26 Pacific
Reply:

Most likely the message board had problems with "less-than" symbols. In HTML they are the start of a tag e.g. <b>, so if you use one in your post it can mean all the text that follows it is one long invalid tag, and consequently ignored.

You can use HTML's syntax for quoting symbols, but just to keep things interesting this board interprets your posted text twice, once for Preview and again in Confirm, so you have to post something like &amp;lt; or the equivalent &amp;#60;. Check posts 5278 and 5288 where this came up recently.


0

Response Number 5
Name: WilliamRobertson
Date: July 11, 2003 at 06:34:42 Pacific
Reply:

The easiest way to script ftp is with a .netrc file in your home directory to take care of the log-in, and a here-document, e.g:

ftp somehost <<-ENDFTP
ascii
mget $file1 $file2 $file3
bye
ENDFTP

You could cat all the files together at the end as a separate step.


0

Related Posts

See More



Response Number 6
Name: Chris
Date: July 15, 2003 at 05:28:35 Pacific
Reply:

Thanks for everyone who answered.

The main problem here is not the FTP connection, but rather the CAT in the end and the following facts:

- The number of files to get on the FTP server varies from time to time;

- The name of the files is unknown in advance and must be gotten by doing a FILE LIST on the FTP server (I tried saving it to a file for reading later in a FOR loop, but I could manage to parse it one filename at a time).

Please, email me with any suggestions or code samples for the problem I just mentionned.

Thanks again,
CHRIS


0

Response Number 7
Name: WilliamRobertson
Date: July 15, 2003 at 06:46:36 Pacific
Reply:

What happens if you do use mget with a wildcard, e.g:

mget x*.dat

Likewise after fetching all the files, why not

cat x*.dat > allfiles.dat


0

Sponsored Link
Ads by Google
Reply to Message Icon






Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Unix Forum Home


Sponsored links

Ads by Google


Results for: Batch ftp download script & append

ftp shell script skip login www.computing.net/answers/unix/ftp-shell-script-skip-login/4887.html

FTP Shell sample www.computing.net/answers/unix/ftp-shell-sample/2120.html

ftp / unix script www.computing.net/answers/unix/ftp-unix-script/4644.html