Hi,
I have a batch script which, amongst other things, parses and rewrites a text file using the "for /f" command. The reason it does this is to append some of the lines in the text file with the current date.

New lines are added to the text file every day, and as it grows, I have noticed that the text file is taking longer to be processed by the for /f command. There are only about 70 lines in the text file and already the time to parse and re-write it is up to about 5 seconds (this may not seem long but i expect the text file to grow to several times its current size of 9kb).

I have tried several methods of constructing the "for /f" - all of them are successful but none of them is any quicker than what I have now. I even tried starting a separate cmd.exe process (within my batch file) to run the "for /f" command, and this worked ok, but I still had to wait for control to return to the batch file before it continued. I imagined that the main batch script would continue whilst the separate cmd process was doing its thing ...

My question is, does anybody know of a way to pass the processing of a subroutine (in a batch script) to another cmd.exe whilst the main batch script continues to process the statements after the subroutine? I have a feeling this isn't possible ... Or can anybody offer any other techniques to solve this kind of problem?

By the way, this is my "for /f" command, which does exactly what I want it to do:

for /f "tokens=*" %%f in (myfile.txt) do (
echo/%%f|findstr /b /c:"%ref_no%" >nul
if not errorlevel 1 (
echo/%%f%date% >>myfile.new
) else echo/%%f >>myfile.new
)

Any info/assistance much appreciated.

To attach a concurrent task to parse the file and continue to process the main one code

Start "" /Min Cmd.exe /C For /f "tokens=*" %f in (myfile.txt) do (
echo/%f|findstr /b /c:"%ref_no%" >nul
if not errorlevel 1 (
echo/%f%date% >>myfile.new
) else echo/%f >>myfile.new
)

Replace %%f with %f as it is interpreted as a command from prompt.
I noticed however you run in the For /f statement the FindStr command that is an external one (i.e. an executable stored on disk). That may slowdown the execution as the executable must be reloaded at each cycle. The logic why you are checking for the %ref_no% variable is unclear to me, but this is a point worth of attention.

Be aware if you code the Start as I posted above the For /f MUST be typed fitting ONE line only without CR/LF.

If you prefer to encapsulate the For into a batch (e.g. MyParser.bat) then code

Start "" /Min Cmd /C MyParser

IVO,
Thanks for your response. This is very helpful. Previously, when I was experimenting with cmd.exe, I use /c switch but not the /min switch and I had to wait for the cmd.exe process to end before continuing with the main batch. Now I find I don't have to wait.

I will continue experimenting with this new info you have given me. The one thing that may cause a problem is the proviso that the for /f statement has to be all on one line with no crlf. This might be a problem because my "for /f" incorporates a "if/else" statement. I am new to batch coding, but if there is one thing I have learnt so far it's that the if/else statement is very temperemental! Maybe I will have to put the sub-routine into its own batch file, although I wanted to avoid having lots of little batch files all over the place. Like I said, I will experiment with using "cmd.exe /min /c" to see how best I can fit it into my script.

In the meantime, I carried out some timing experiments with "findstr", and of course you are so right about this command causing the delays. It would be great if I could avoid using it but I don't think I can. This is what I am trying to do:

I have a text file (myfile.txt) which holds a number of records, one record per line. Each record represents a different job. Each record begins with a unique ref number, which is just a simple integer. The idea is that when a job is done, its record is appended with the date, and the text file is rewritten to reflect this. So, that bit of code I posted previously is meant to test the ref number of each line in the text file, and if it matches the one I am looking for (%ref_no%), it appends the date to that line and re-writes the record to myfile.new, otherwise it just re-writes the record as is to myfile.new. Then myfile.new gets renamed myfile.txt. Hope this makes sense.

Like I said before, it all works fine, but it's getting slower as more records are added. But then again I told you I was a newbie at all this ...!

Many thanks again for the info you gave me.

Hi Andrew,

You can avoid having lots of separate BATS by coding them in as subroutines as in the simole example below:

::== some.bat
@echo off
for %%f in (1 2) do call :sub1 %%f
goto :eof

:sub1
echo this is :sub1
echo the parameter is "%*"
goto :eof
:: DONE

As for speeding up the whoke process, you may be able to avoid the FINDSTR.

Would you care to post a few of the records so we can see how they are laid out?

BTW, I have no idea why you use echo/%%f but the slash seems to fall through the cracks, so it doesn't hurt anything.

If at first you don't succeed, you're about average.

M2

Let's go on step by step...

First - The /Min switch is NOT related to Cmd.exe BUT to the Start command and it is not the key to solve your problem. It just minimizes the task attached by the Start command (Start /? to know more). In fact it is in front of the Cmd /C instance and it is Start that allows multitasking.

Second - To avoid the use of a batch routine just type the following statement without CR/LF and don't care how long it blows up:

Start "" /Min Cmd /C For /f "tokens=*" %f in (myfile.txt) Do (@Echo %f | FindStr /b /c:"%ref_no%" > nul &
If not ErrorLevel 1 (@echo %f%date% >> myfile.new) else (@Echo %f >> myfile.new))

Third - If your interger number is at the beginning of each record and separated from the remainder of each line by a separator (e.g. blank or comma) the FindStr can be easily avoided. Post a samplr line and the solution will follow.

Guys
I appreciate the interest you've shown in this.

Here is a sample of the format of my text file, just the first three lines of it. Each line (or record) comprises 9 fields, and the fields in each line are delimited by semi-colons. Corresponding fields in each line are padded out with spaces so that they are the same number of characters in length. Currently there are about 80 lines in total in the file. Line number 2 in the sample below has been appended with a date to signify that an event has occurred, and this is where the code sample I posted previously comes into play.

The sample lines will probably wrap when I post them, but the first field on each one starts with an integer (this is the ref number that I want to match against):

1 ;20060601;A;fox otter ;bluewalk ;carry out operation nineteen ;N; ;
2 ;20060601;D;hedgehog ;alderminster ;update tin job ; ;DQ;20060608
3 ;20060602;B;badger fox ;warwick ;carry out operation forty-eight ; ; ;

I hope what I have written makes sense and thanks again for your time on this, it's been quite educational from my point of view.

Hi Andrew,

Makes sense to me. Sadly, clear exposition seems all but extinct.

Try this:
::== refno.bat
@echo off > new.csv
set /p ref_no=ref no?
for /f "tokens=* delims=;" %%C in (my.csv) do call :sub1 %%C
goto :eof

:sub1

if %ref_no% equ %1 (
echo %* %date% >> new.csv
)else echo %* >> new.csv
)
:: DONE

If at first you don't succeed, you're about average.

M2

Mechanix,
Thanks for your help with this. I tried the code you posted and it works fine. I would never have guessed that calling the sub with just one parameter (ie, the first token) actually brings all the other tokens with it! Anyway, this is a lot faster than using findstr, which was the problem I was having. Also, in the subroutine, I like the way it strips the trailing spaces off the %1 when comparing it to %ref_no%, but then includes all the trailing spaces when echoing the full line with the appended date! This was important because the values in my records are each padded out with trailing spaces as necessary to make the fields specific lengths, so that the semi-colon delimiters are all aligned in columns. Amazing.
I will have to adapt the code so that I can correctly update the other fields in my records, as required. I guess this will be slightly more difficult than just appending the date, but it should be pretty straightforward if I use the same techniques that you used.

Thanks again, much appreciated.

Hi Andrew,

GTH

Yeah the action of tokens and the chunking up of strings is anything but intuitively obvious.

Let us know how you get on with the rest.

If at first you don't succeed, you're about average.

M2