Computing.Net > Forums > Programming > Log the # and size of files copied

Log the # and size of files copied

Reply to Message Icon

Original Message
Name: BC_ca
Date: January 3, 2008 at 08:10:00 Pacific
Subject: Log the # and size of files copied
OS: XP - DOS
CPU/Ram: P4-2GB
Comment:

Hi guys,

I'm creating an automated DOS batch file to copy multiple directories to a different drive. Once the copy is complete, I want to log the number of files and size of the files copied.

DIR /s seems to be the easiest way to get it, but I can't find an efficient way to grab the 2nd last line of the DIR /s output (which lists the # and size of the files. The data I'm moving has a lot of files, so the DIR /s is fairly large (~2MB when piped to a text file).

Any thoughts?


Report Offensive Message For Removal


Response Number 1
Name: klint
Date: January 3, 2008 at 09:39:58 Pacific
Reply: (edit)

Hold on you got me confused here. You state your operating system as "XP - DOS". Which one is it exactly? Are you running a dual-boot configuration? If so, what version of DOS are you running on your secondary partition?


Report Offensive Follow Up For Removal

Response Number 2
Name: Mechanix2Go
Date: January 3, 2008 at 10:46:36 Pacific
Reply: (edit)

You can get the totals with this:

dir/s|find "File(s)"

You can chop it down to the last line [grand total] easily in XP; harder in DOS.



=====================================
If at first you don't succeed, you're about average.

M2



Report Offensive Follow Up For Removal

Response Number 3
Name: BC_ca
Date: January 3, 2008 at 10:59:33 Pacific
Reply: (edit)

I'm running XP, but scripting solely in DOS (so ver 5.1).

I've tried Mechanix2Go's method, which gets my total on the last line. Is there an easy way to grab ONLY the last line?

Thanks


Report Offensive Follow Up For Removal

Response Number 4
Name: klint
Date: January 3, 2008 at 11:32:36 Pacific
Reply: (edit)

I see, I got confused because you called it "DOS" which is an old single-user system. The Win32 Command Processor (which is what you are using, and is quite different from DOS) has much better batch file scripting facilities. I think the following batch file does what you want:

@echo off
setlocal EnableDelayedExpansion
set num=0
set size=0
for /r %%f in (*) do (
set /a num += 1
set /a size += %%~zf
)
echo Total %num% files, %size% bytes


Report Offensive Follow Up For Removal

Response Number 5
Name: BC_ca
Date: January 3, 2008 at 11:56:30 Pacific
Reply: (edit)

Excellent! It took a little while to crank through the size of the directory, but did respond. I'm wondering if the size reached an upper limit, as these were the two lines that returned:

Invalid number. Numbers are limited to 32-bits of precision.

Total 24660 files, 1238806606 bytes


One other thought - what changes would be needed to call this script on a directory tree that the script does not reside in?

Many thanks!


Report Offensive Follow Up For Removal


Response Number 6
Name: klint
Date: January 3, 2008 at 14:59:20 Pacific
Reply: (edit)

Yes, unfortunately it did reach an upper limit (about 2 GB). You could try doing arithmetic in the size calculation so you add kilobytes instead of bytes, but then the totals will be only an approximation.

Something like:

set /a size += (%%~zf + 512) / 1024

Note that the above won't work well if most of your files are small (less than 512 bytes.)

The second part of the question is easier. You could either CD to the required directory before you issue the for /r command, or you could pass the directory as the optional parameter of the for /r command:

for /r %path% %%f in (*) do ...


Report Offensive Follow Up For Removal

Response Number 7
Name: BC_ca
Date: January 3, 2008 at 15:43:51 Pacific
Reply: (edit)

Thanks for the fast turnaround!

The second part worked like a charm (using the %path% to use it on any directory).

The first part (using set /a size += (%%~zf + 512) / 1024) gave me an error in the batch. The error I got was
"/ was unexpected at this time". It looks like the script is having problems with the division by 1024 syntax.

I tried a few variations on the syntax, but couldn't avoid the error. Any ideas?

Thanks!


Report Offensive Follow Up For Removal

Response Number 8
Name: klint
Date: January 3, 2008 at 16:49:24 Pacific
Reply: (edit)

That had me puzzled for a long time too. It seems to be a feature/quirk/bug of the command line interpreter. It seems to think the closing bracket of the expression matches the opening bracket of the for...do command. The answer is to escape the closing bracket with the ^ character:

set /a size += (%%~zf + 512 ^) / 1024

Good luck...


Report Offensive Follow Up For Removal

Response Number 9
Name: klint
Date: January 3, 2008 at 17:06:53 Pacific
Reply: (edit)

Actually, I've just come up with a new version that doesn't have the drawbacks I mentioned earlier about inaccuracies. It works by counting bytes, but accumulating kilobytes. Every time the byte count exceeds 1 kilobyte, the kilobyte count is increased by the number of kilobytes, and the byte count is set to the remainder. At the end of the loop, the kilobyte count is adjusted by the final remaining byte count rounded to the nearest kilobyte. The only remaining drawback is that it doesn't work if you have any individual file larger than about 2 GB.

@echo off
setlocal EnableDelayedExpansion
set num=0
set size=0
set sizekb=0
for /r %%f in (*) do (
set /a num += 1
set /a size += %%~zf
if !size! geq 1024 (
set /a sizekb += !size! / 1024
set /a size %%= 1024
)
)
set /a sizekb += (!size! + 512) / 1024
echo Total %num% files, %sizekb% kilobytes



Report Offensive Follow Up For Removal

Response Number 10
Name: BC_ca
Date: January 3, 2008 at 18:34:43 Pacific
Reply: (edit)

Works like a charm.

I had originally gone down the path of piping DIR /s to a file, and finding a way to grab the 2nd last line. That seemed like an easy enough thing to do, but I was never able to achieve it.

Thanks for your help!


Report Offensive Follow Up For Removal

Response Number 11
Name: Mechanix2Go
Date: January 3, 2008 at 20:45:13 Pacific
Reply: (edit)

@echo off
setLocal EnableDelayedExpansion

pushd x:\mystuff

for /f "tokens=* delims= " %%a in ('dir/s ^|find "File(s)"') do (
echo %%a > copy.log
)



=====================================
If at first you don't succeed, you're about average.

M2



Report Offensive Follow Up For Removal

Response Number 12
Name: BC_ca
Date: January 3, 2008 at 21:13:27 Pacific
Reply: (edit)

Works like a charm.

I had originally gone down the path of piping DIR /s to a file, and finding a way to grab the 2nd last line. That seemed like an easy enough thing to do, but I was never able to achieve it.

Thanks for your help!


Report Offensive Follow Up For Removal

Response Number 13
Name: klint
Date: January 4, 2008 at 02:14:56 Pacific
Reply: (edit)

Glad it helped. For the sake of future usage, M2's solution also seems good, except it seems to me that it gets the last line into copy.log, not the second-last. It can be modified to get the second-last.

Yet another way to do it, there is a free utility available called tail.exe, which is part of the GNU tools ported to Win32. Microsoft also have a version of it; I think it's in one of their resource kits. Then you can pipe the result of dir/s to "tail -2", which gets the last two lines, and then pipe that to "head -1" (another similar GNU utility) which gets the first line of that (ie the second-last line of dir/s.)


Report Offensive Follow Up For Removal

Response Number 14
Name: Mechanix2Go
Date: January 4, 2008 at 03:34:14 Pacific
Reply: (edit)

Sure, but what's needed is the ;ast line containing File(s). That's what the bat does.


=====================================
If at first you don't succeed, you're about average.

M2



Report Offensive Follow Up For Removal

Response Number 15
Name: klint
Date: January 4, 2008 at 04:20:39 Pacific
Reply: (edit)

M2, you're right. Your code does indeed do the job.


Report Offensive Follow Up For Removal






Use following form to reply to current message:

   Name: From My Computing.Net Settings
 E-Mail: From My Computing.Net Settings

Subject: Log the # and size of files copied

Comments:

 


  Homepage URL (*): 
Homepage Title (*): 
         Image URL: 
 
Data Recovery Software




Have you ever used OpenOffice?

Yes, as my main suite.
Yes, occationally.
Yes, but only once.
No, never.


View Results

Poll Finishes In 5 Days.
Discuss in The Lounge