Computing.Net > Forums > Unix > Running Unix commands in parallel

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Running Unix commands in parallel

Reply to Message Icon

Name: oracledba
Date: May 19, 2004 at 09:29:05 Pacific
OS: HP-UX
CPU/Ram: 6 CPUs and 8G Ram
Comment:

Hi All,

Could you please let me know if there is a way to run Unix commands like 'mv', 'cp' and 'compress' in parallel. I have a script to 'mv' or cp or 'compress' the file in serial by going in to a 'do loop'. But due to volume I really would like to accomplish the parallel 'mv' or 'cp' or 'compress'.

I greatly appreciate if you let me know a way to accomplish this. A sample script will be of a great help!

Thank you very much for your time and help!
Have a great day!

Regards,
Sam.



Sponsored Link
Ads by Google

Response Number 1
Name: thepubba
Date: May 19, 2004 at 12:48:54 Pacific
Reply:

Put the compress in background. Example:

#!/bin/ksh

for file in *
do
if [[ ! -d $file ]]
then
cp $file /newdirectory/$file
compress /newdirectory/$file &
fi
done

With a little imagination, you can add all sorts of error checking and counters to the script.


0

Response Number 2
Name: Dlonra
Date: May 19, 2004 at 18:31:38 Pacific
Reply:

This can result in many compress'es running at once, slowing system response and PO'ing users.

How large are the files? Are the mv/cp to a different file system?


0

Response Number 3
Name: thepubba
Date: May 20, 2004 at 06:30:10 Pacific
Reply:

So use 2 loops. One that puts a fixed number of processes in background and then waits until they finish before getting the next group. This is not difficult. He just asked how to do it and background processes are the only way this will be accomplished.


0

Response Number 4
Name: Dlonra
Date: May 20, 2004 at 16:38:22 Pacific
Reply:

of course, of course.
However, if the files are not too large and they are not being moved to a different file system, the time savings might be
teeny.


0

Sponsored Link
Ads by Google
Reply to Message Icon

Related Posts

See More







Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Unix Forum Home


Sponsored links

Ads by Google


Results for: Running Unix commands in parallel

Run Unix command in DOS FTP prompt. www.computing.net/answers/unix/run-unix-command-in-dos-ftp-prompt/8494.html

running unix command in vi session www.computing.net/answers/unix/running-unix-command-in-vi-session/7869.html

run unix command thr java program www.computing.net/answers/unix/run-unix-command-thr-java-program-/5887.html