Solved Count for files in directory and start xcopy

June 24, 2011 at 03:42:38
Specs: Windows XP, Core2Duo/2GB
Hello gurus,


I am facing the below issue.
I have some files in directory Dir_1 , and I need to copy them in another directory Dir_2.

Well, here I go.
I thought this would have done the job, but it isn't.
What am I missing?

@echo off & setLocal EnableDELAYedeXpansion

pushd D:\Dir_1

for /F %%j in (dir D:\Dir_1 /A-D | find /c "OK_ARCHIVE") do set Count=%%jdo set Count=%%j

if %count% EQU 0 (
    echo No files found in the specified folder>>FilesError.log
) else (
for /L %%b in (1 1 %count%) do ( xcopy "D:\Dir_1"  "G:\Dir_2" /D /I /Y)
echo A total of !count! files have been copied>>FilesSuccess.log
)
move FilesSuccess.log D:\Dir_3
move FilesError.log D:\Dir_3

after executing, I am getting

| was unexpected at this time.


Please help,
Thnx and regards


See More: Count for files in directory and start xcopy

Report •


✔ Best Answer
June 28, 2011 at 07:26:13
@echo off & setlocal

set source=D:\Dir_1
set target=G:\Dir_2

cd /D "%source%"
if not exist "%target%" md "%target%"
if not exist "D:\Dir_3\FilesList.log" type nul > "D:\Dir_3\FilesList.log"
set stamp=%date:~-10% on %time:~0,8%

set cnt=0
for %%j in (*) do if %%~zj geq 50 if not exist "%target%.\%%j" (
  type "D:\Dir_3\FilesList.log" | find /I "%%j" > nul
  if ErrorLevel 1 (
    copy "%%j" "%target%" > nul
    echo.%stamp% %%j>> "D:\Dir_3\FilesList.log"
    set /A cnt+=1
  )
)

if %cnt% equ 0 (
  echo.no files found
  echo.%stamp% no files found>> "D:\Dir_3\FilesError.log"
) else (
   echo.%cnt% files copied
   echo.%stamp% %cnt% files copied>> "D:\Dir_3\FilesSuccess.log"
)



#1
June 24, 2011 at 05:45:18
for /F "delims=" %%j in ('dir D:\Dir_1 /B /A-D ^| find /C "OK_ARCHIVE"') do set Count=%%j


Report •

#2
June 24, 2011 at 06:07:02
Thank you for you answer Ivo
Just one more thing ...
Why am I getting the following output?
15 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
0 File(s) copied
        1 file(s) moved.
The system cannot find the file specified.

Of course the files are copied (15) , but what about the
0 File(s) copied ... why does it apear?


Report •

#3
June 24, 2011 at 06:25:51
Replace

for /L %%b in (1 1 %count%) do (xcopy "D:\Dir_1"  "G:\Dir_2" /D /I /Y)

with

xcopy "D:\Dir_1"  "G:\Dir_2" /D /I /Y

otherwise you repeat xcopy N times while just the first one moves the files.

Report •

Related Solutions

#4
June 24, 2011 at 06:32:37
Thank you again Ivo,

just one more help

I just found out, I have better test if any of the files has a size of <=50 byte
If file_Size <= 50byte, than I should not copy it.

Can you assist me on these?

@echo off & setLocal EnableDELAYedeXpansion

pushd D:\Dir_1

for /F "delims=" %%j in ('dir D:\Dir_1 /A-D ^| find /c "OK_ARCHIVE"') do set count=%%j
for %%a in ('dir D:\Dir_1 /A-D ^| find "OK_ARCHIVE"') do set size=%%~za

if %count% EQU 0 (
    echo No files found in the specified folder>>FilesError.log
) else (
for /L %%b in (1 1 %count%) do ( 
if %size% GTR 50 (
xcopy "D:\Dir_1"  "G:\Dir_2" /D /I /Y)
echo A total of !count! files have been copied>>FilesSuccess.log
) else (
echo U gjet skedar me size 0>>FilesError.log
)
) 
move FilesSuccess.log D:\Dir_3
move FilesError.log D:\Dir_3


Thank you indeed


Report •

#5
June 24, 2011 at 06:54:01
Sorry, I don't clearly understand what you need...

Do you have to copy the files with size less or equal 50 bytes only?

Do the files have in their name the string "OK_ARCHIVE"?

Maybe I can give you the script later since I am leaving now.


Report •

#6
June 24, 2011 at 07:05:42
Let me clarify it...
I want to copy only the files with size greater than 50 bytes
But firstly, I have to check that the file name contains the string "OK_ARCHIVE"

Just for extra tip ...
If the size of the file in Dir_1 is greater than 50 bytes, than its name starts with the string "OK_ARCHIVE" for sure. So I just have to check in Dir_1 for the files with the size greater than 50 bytes, and copy them, ... no matter their name ....

Looking forward to hearing from you.


Regards,
Pupli


Report •

#7
June 24, 2011 at 12:43:27
@echo off & setlocal

set source=D:\Dir_1
set target=G:\Dir_2

cd /D "%source%"
if not exist "%target%" md "%target%"

set cnt=0
for %%j in (*) do if %%~zj geq 50 (copy "%%j" "%target%" >nul & set /A cnt+=1)

if %cnt% equ 0 (
  echo.No files found
  echo.at %date:~-10% on %time:~0,8% no files found>> "D:\Dir_3\FilesError.log"
) else (
  echo.%cnt% files copied
  echo.at %date:~-10% on %time:~0,8% %cnt% files copied>> "D:\Dir_3\FilesSuccess.log"
)


Report •

#8
June 27, 2011 at 01:21:41
Hello Ivo and thank you so much...
It works like a charm :))

Just one more issue ...
I have to schedule this batch to run every 4 hours...
It can happen that, in the second run ... in Dir_1 may exist files that have already been copied in the first run

I am not clear wheather

copy "%%j" "%target%"

Overwrites or just skips if the file already exist in Dir_2
(actually it seems like it is overwriting being that I am getting the same number in the

echo.at %date:~-10% on %time:~0,8% %cnt% files copied
when running twice with the same files in Dir_1)

I need to skip if the file does already exist in Dir_2...
Moreover, how can I manage to write into FilesSuccess.log only the "really" copied files? What I mean is, if it happens that a previous copied file is still in Dir_1, I do not want it to be counted in the result.

Thank you again,


Report •

#9
June 27, 2011 at 03:01:14
If you don't want to copy a file that already exists in Dir_2, replace

for %%j in (*) do if %%~zj geq 50 (copy "%%j" "%target%" >nul & set /A cnt+=1)

with

for %%j in (*) do if %%~zj geq 50 if not exist "%target%.\%%j" (
  copy "%%j" "%target%" >nul & set /A cnt+=1
)

The above code returns the effective number of files copied at each run too, but it doesn't copy a more recent version of a file if already in Dir_2.

Let me know if you have to handle the case since a slightly more complex code is needed.


Report •

#10
June 27, 2011 at 06:09:11
Thank you Ivo,

This is exactly what I needed ...
Just to be sure, is there any way to keep logs of the file I copy?
E.g. if I copy file OK_ARCHIVE_A123 in the first run of the batch , I do not want to copy it again any more ...
Can I just ... store the name of the files I "really" copy in a file (e.g. .txt) and after that, every time the batch starts it firstly will check if the file name is present there, if not , then go and copy ...


Report •

#11
June 27, 2011 at 06:55:25
I do assure the code I posted absolutely doesn't copy files already stored in the destination directory. What you outline is feasible but doesn't add any more safety, just does heavily slow down script performance. Supersafety must be always avoided since it leads to the opposite result.

Report •

#12
June 27, 2011 at 07:27:21
Yes I understand ... You are right, but I am considering the above being that there would be a maximum of 500 files to be copied (on daily basis) ... in my case performance would not be affected that much

Report •

#13
June 27, 2011 at 09:52:40
If you feel safer with that extra check, I can post the modified code, but, I repeat, it doesn't increase any safety. Let me know about your opinion.

Report •

#14
June 28, 2011 at 01:07:06
it doesn't increase any safety. 

I am afraid, I am not fully understanding you ...
Are you saying that keeping the file names of the copied files will not asure that the same files will NOT be copied again?

Report •

#15
June 28, 2011 at 03:18:41
NO, I am saying that keeping track of the file names copied is just redundant in assuring the files will not be copied again.

Just as if you have a door with two locks and two keys.


Report •

#16
June 28, 2011 at 04:27:41
aha ... no I understand your point of view
you are right, I should have given more details
No let me just explain it for you ...

The process, is like this ...
I have to copy every 4 hours, files from Dir_1 to Dir_2
On the other hand, every 7 or 8 hours another process starts in Dir_2
The files copied here (Dir_2), are then FTP-ed at another site, and after "uploading" is finished, files in Dir_2 are deleted!
The scenario is this:
(1) - I copy files from Dir_1 to Die_2 every 4 hours
(2) - As long as the same files are present in Dir_2, thanks to the batch script it is possible to avoid coping duplicates
(3) - The FTP process starts ... it uploads the files that it finds in Dir_2 and after that it deletes the files. We can not change this !!!
(4) - Dir_2 is empty
(5) - The batch starts again (after 4 hours) and it copies from Dir_1 to Dir_2, but being that Dir_2 is empty, the batch will copy to Dir_2 every file ... Including the ones he copied in its first run .. !!! Dir_1 is revised for "deletion" on daily basis (every 24 hours) and we can not change this !!!

What I want, is to keep track of the files copied, thus avoiding multiple copying of files ...!


Report •

#17
June 28, 2011 at 06:27:38
Well, now your process specs are clear but quite different from your original post. That requires a slightly modified script and it is not a case of redundancy.

Keep in touch and I'll post the required code.


Report •

#18
June 28, 2011 at 07:26:13
✔ Best Answer
@echo off & setlocal

set source=D:\Dir_1
set target=G:\Dir_2

cd /D "%source%"
if not exist "%target%" md "%target%"
if not exist "D:\Dir_3\FilesList.log" type nul > "D:\Dir_3\FilesList.log"
set stamp=%date:~-10% on %time:~0,8%

set cnt=0
for %%j in (*) do if %%~zj geq 50 if not exist "%target%.\%%j" (
  type "D:\Dir_3\FilesList.log" | find /I "%%j" > nul
  if ErrorLevel 1 (
    copy "%%j" "%target%" > nul
    echo.%stamp% %%j>> "D:\Dir_3\FilesList.log"
    set /A cnt+=1
  )
)

if %cnt% equ 0 (
  echo.no files found
  echo.%stamp% no files found>> "D:\Dir_3\FilesError.log"
) else (
   echo.%cnt% files copied
   echo.%stamp% %cnt% files copied>> "D:\Dir_3\FilesSuccess.log"
)


Report •

#19
June 28, 2011 at 23:56:30
Ivo, thank you so much ... really!
I was checking the FilesList.log, and I got a little bit surprised by the date and time previous to the file name ... it will not affect the "copmarison" phase, will it?
May I ask just one last thing ?

If I want to move/cut (not just copy) files from Dir_1, is it enough to replace copy command with move

Thank you,
Pupli


Report •

#20
June 29, 2011 at 02:35:50
I do assure date/time in FilesList.log doesn't affect the comparison process. Timestamp is just an aid if you want to browse the log to see when a file was processed.

Yes, to move (i.e. copy and delete the original file) just replace copy with move.


Report •

#21
June 29, 2011 at 07:15:36
If you decide to move the files from Dir_1 to Dir_2, the exisistence and comparison checks are meaningless, so better you turn back to the code in post #7 replacing copy with move.

Anyway pay attention to the size of FilesList.log since it may assume huge dimension as time goes, slowing down processing because it is browsed for each file to be transferred. It needs to be resetted on periodic intervals.


Report •

#22
June 30, 2011 at 06:28:50
Ivo,


May I ask you help regarding the following?
I have scheduled the batch file to run every 3 hours , via Windows scheduler ...
Ok, it runs well at first ... but, the second (or third try) IF there is no file in the source directory (Dir_1) ... the next time it won't lunch the batch ...

In the properties, it shows like it is going to run after 24 hours ...

Now, I am considering the followings, but I can not state which best does the job.
I want the batch to run every day, every 2 hours ...

 AT 19:00 /interactive every:M,T,W,Th,F,S,Su every:2:00 cmd /c D:\FTP\Logica_Files_Transferr\OK_TP_DWH.bat
 
 schtasks /create /sc hourly /mo 2 /sd 06/30/2011 /tn "G_DWHFILE_TP" /tr D:\FTP\Logica_Files_Transferr\OK_TP_DWH.bat


Report •

#23
June 30, 2011 at 06:46:17
Let me examine the issue since I am not experienced with Windows Task Scheduler. When needed I use AT command but its set up may be tricky so sometime I rely on a script coded by myself that I am ready to give you.

Stay in touch for my next post since now I am busy and short on time. See you as soon.


Report •

#24
June 30, 2011 at 07:08:26
Thank you Ivo

I am waiting for your advice than ...


Report •

#25
June 30, 2011 at 13:44:22
No doubt you have to schedule using schtasks since it allows multiple runs daily while AT doesn't support that (your AT command syntax is not correct!).

schtasks /create /sc hourly /mo 2 /sd 06/30/2011 /tn "G_DWHFILE_TP" /tr D:\FTP\Logica_Files_Transferr\OK_TP_DWH.bat

If the above fails let me know and I will give you my in-house scheduler script (that does work indeed).


Report •

#26
June 30, 2011 at 23:17:10
Hello Ivo,


I scheduled with the above.
Just a little bit surprised as a new task apeared like the one I created via Windows
Task Scheduler. Just wish it works this time.


Report •


Ask Question