Computing.Net > Forums > Programming > Batch file for processing logfiles

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Batch file for processing logfiles

Reply to Message Icon

Name: ikerstges
Date: August 5, 2008 at 10:29:35 Pacific
OS: Windows XP Professional
CPU/Ram: Core2 Duo/1024 Mb
Product: HP
Comment:

Hi All,

I'm monitoring log files of testcases. The logfile always contains a last line where the status is reported (successful/NOT successful). I'm able to find these status reports with following commandline script:

For /F "tokens=*" %%A in ('Dir /B /A:-D %1') Do (
type %%~fA | find "INFO - Testcase was successful!" >> D:\scripts\splitfiles\logs\temp\successful.txt
For /F "tokens=*" %%A in ('Dir /B /A:-D %1') Do (
type %%~fA | find "INFO - Testcase was NOT successful!" >> D:\scripts\splitfiles\logs\temp\not_successful.txt

But now my question:

How can I find the last 2 ERROR lines in my logfile?

Here is an example of my last logfile:

04 Aug 2008, 11:24:06 ERROR - Assertion failed param1: 1 matching "MercuryTest_21588" param2: No matches assertionType: CONTAINS
04 Aug 2008, 11:24:06 INFO - Condition was false
04 Aug 2008, 11:24:29 INFO - ***** Executing Step: Step 3 *****
04 Aug 2008, 11:24:31 INFO - Loaded URL: http://test.wap.xxxxxx.com/xxxxxxxx... successfully
04 Aug 2008, 11:24:31 ERROR - Failed at Step 3: field 1 - 2 (MercuryTest_21588) not found(Control type: [link] Criteria: [.text] Criteria value: [1 - 2 (MercuryTest_21588)])
04 Aug 2008, 11:24:31 ERROR - Failed at Step 3: field not found(Control type: [link] Criteria: [.text] Criteria value: [1 - 2 (MercuryTest_21588)])
04 Aug 2008, 11:24:31 INFO - ***** Executing Step: Reset Promos *****
04 Aug 2008, 11:24:31 INFO - start call of sequence: WAP:xxxxCommons.ResetAllPromotionAreas
04 Aug 2008, 11:24:31 INFO - start call of sequence: WAP:xxxxCommons.SchedulePromotion
04 Aug 2008, 11:24:33 INFO - Loaded URL: http://test.xxxxxxxx.xxxx.com/defau... successfully
04 Aug 2008, 11:25:04 INFO - Useragent set to IE6
04 Aug 2008, 11:25:04 INFO - Loaded standard security configuration
04 Aug 2008, 11:25:04 INFO - Testcase was NOT successful!

I'm only interested in the last two ERROR lines, I want to pipe the contents of these to the "not_successfull.txt" but I cannot find how to achieve that?

Any help will be greatly appreciated!


Thanks in advance!
Igor

(This is also posted at Windows XP, but I figured out that it belongs in the Programming section, so apologies for the cross-post. I will be more careful next time!!)



Sponsored Link
Ads by Google

Response Number 1
Name: cup
Date: August 5, 2008 at 11:40:31 Pacific
Reply:

DOS doesn't have a head/tail command so it is not that easy. You can do it programatically using vbscript and run the script file using cscript.


0

Response Number 2
Name: ghostdog
Date: August 5, 2008 at 18:37:01 Pacific
Reply:

vbscript

Set objFS = CreateObject("Scripting.FileSystemObject")
Dim ArrStore()
i=0
strFileName="test.txt"
Set objFile = objFS.OpenTextFile(strFileName)
Do Until objFile.AtEndOfLine
strLine=objFile.ReadLine
ArrLine = Split(strLine)
If ArrLine(4) = "ERROR" Then
ReDim Preserve ArrStore(i)
ArrStore(i)=strLine
i=i+1
End If

Loop
For j=UBound(ArrStore)-1 To UBound(ArrStore)
WScript.Echo ArrStore(j)
Next



0

Response Number 3
Name: ikerstges
Date: August 6, 2008 at 12:13:23 Pacific
Reply:

Hi all,

Thanks for your quick resposes!

I have NO experience with VBScript AT ALL, but I have tried.. What I did:

- I copied the code into a file and named the file: "C:\Scripts\LogFiles\ExtractErrorLines.vbs"
- I created a text-file: "test.txt" in the same directory containing 3 lines where only the 3rd line contains the word "ERROR"

Then I ran the VBScript.

It generates an error though. Here the Error-message:

Script: C:\Scripts\LogFiles\ExtractErrorLines.vbs
Line: 16
Char: 1
Error: Subscript out of range: 'UBound'
Code: 800A0009
Source: Microsoft VBScript runtime error

I'm almost sure this can be done with plain commandline batchfile (prefferred solution), but I will be happy to use VBScript if I can get a working solution from it.

Maybe someone can help?


Thanks in advance!
Igor


0

Response Number 4
Name: Mechanix2Go
Date: August 6, 2008 at 12:39:42 Pacific
Reply:

@echo off > not_successfull.txt
setLocal EnableDelayedExpansion

find "ERROR" < logfile > E

for /f "tokens=* delims= " %%a in ('find /c "ERROR" ^< E') do (
set /a skip=%%a-2
)

for /f "tokens=* delims= " %%a in (E) do (
set /a N+=1
if !N! gtr !skip! echo %%a >> not_successfull.txt
)


=====================================
If at first you don't succeed, you're about average.

M2


0

Response Number 5
Name: ikerstges
Date: August 6, 2008 at 13:34:13 Pacific
Reply:

@Mechanix2Go: EXACTLY!! ;-)

Works like a charm! Thanks a lot!!

Cheers,
Igor


0

Related Posts

See More



Response Number 6
Name: ikerstges
Date: August 7, 2008 at 01:01:49 Pacific
Reply:

I hope somebody can help a bit more..

I have changed the @Mechanix2Go script to process a list of files:

--------------------

@echo off > not_successfull.txt
setLocal EnableDelayedExpansion

For /F "tokens=1,2 delims=. " %%i in ('Dir /B /A:-D "C:\Scripts\Mercury\Logs\TE_*.log"') Do (
find "ERROR - " < "C:\Scripts\Mercury\Logs\%%i.%%j" > E
)

for /f "tokens=* delims= " %%a in ('find /c "ERROR - " ^< E') do (
set /a skip=%%a-2
)

for /f "tokens=* delims= " %%a in (E) do (
set /a N+=1
if !N! gtr !skip! echo %%a >> not_successfull_%%i%%.txt
)

del E /q/f
)
--------------------

But now, two new problems occur:
1) The file-list of: Dir /B /A:-D "C:\Scripts\Mercury\Logs\TE_*.log" produces many files containing ip-addresses and several random spaces in the filename. So the %%i and %%J variables in the first 'For'-loop result in incorrect filenames when used as "%%i.%%j"
Example: the filename TE_100.130.131.10_Test login link_20080807.log results in %%i= TE_100 and %%j=130

Is there a way to process the filenames containing dots and spaces?

2) I tried to construct a not_successfull.txt file for each logfile by using the %%i variable. But when using the not_successfull_%%i%%.txt as in my example above, the script only produces one file: not_successfull_logfile3.txt

I would expect to end-up with files like:
not_successfull_logfile1.txt
not_successfull_logfile2.txt
not_successfull_logfile3.txt

Any help would be greatly appreciated (again)!!

Thanks in advance,
Igor


0

Response Number 7
Name: ghostdog
Date: August 7, 2008 at 01:27:47 Pacific
Reply:

can you show a sample of your test.txt? I tested the vbscript as per your first post.


0

Response Number 8
Name: ikerstges
Date: August 7, 2008 at 01:46:41 Pacific
Reply:

Hi ghostdog:

Here the contents of my test.txt:

--------------------
07/08/2008 05:03:25 - SUCCESS - This is the 1st line!
07/08/2008 05:19:07 - SUCCESS - This is the 2nd line!
07/08/2008 06:12:49 - ERROR - This is the 3rd line!
--------------------

Thanks for your help! It is nice to see the two different approaches of a VBScript and a commandline script!

I'm most familiar with the command line myself.. ;-)


0

Response Number 9
Name: ghostdog
Date: August 7, 2008 at 03:07:59 Pacific
Reply:

dude, your input data has changed! That's why you have error when running the vbscript

Set objFS = CreateObject("Scripting.FileSystemObject")
Dim ArrStore()
i=0
strFileName="test.txt"
Set objFile = objFS.OpenTextFile(strFileName)
Do Until objFile.AtEndOfLine
strLine=objFile.ReadLine
ArrLine = Split(strLine)
If ArrLine(3) = "ERROR" Then
ReDim Preserve ArrStore(i)
ArrStore(i)=strLine
i=i+1
End If
Loop
If UBound(ArrStore) = 0 Then
'when you only have one line of ERROR found
For j=UBound(ArrStore) To UBound(ArrStore)
WScript.Echo ArrStore(j)
Next
Else
'For 2 or more ERROR found
For j=UBound(ArrStore)-1 To UBound(ArrStore)
WScript.Echo ArrStore(j)
Next
End If


i suggest you understand more about vbscript( if you want to use it) by downloading the help file: here

0

Response Number 10
Name: Mechanix2Go
Date: August 7, 2008 at 12:00:20 Pacific
Reply:

A couple ways you can go. One is to 'wrap' the main code in a FOR. For me, that gets confusing if the main chunk is complex.

So you could take the other route and CALL a subroutine.

[NOTE: you can save clutter by using PUSHD and keeping paths short.]

::=======================================
@echo off
setLocal EnableDelayedExpansion

if exist not_successfull*.txt del not_successfull*.txt

::pushd "C:\Scripts\Mercury\Logs"

for /f "tokens=* delims= " %%t in ('dir/b TE_*.log') do (
set /a logNUM+=1
call :sub1 %%t
)

goto :eof

:sub1

find "ERROR" < %1 > E

for /f "tokens=* delims= " %%a in ('find /c "ERROR" ^< E') do (
set /a skip=%%a-2
set /a N=0
)

for /f "tokens=* delims= " %%a in (E) do (
set /a N+=1
if !N! gtr !skip! echo %%a >> not_successfull!logNUM!.txt
)

goto :eof


=====================================
If at first you don't succeed, you're about average.

M2


0

Response Number 11
Name: ikerstges
Date: August 8, 2008 at 00:09:47 Pacific
Reply:

@ghostdog: You are right, mea culpa! Your new script works great, thanks! I appreciate your patience and I have to admit: you made me interested and I have actually downloaded the helpfile! ;-)

@Mechanix2Go: Thanks again to you too! The script works great too. I actually learned many new commandline-options. The call to a subroutine is obvious, stupid I didn't think of it! :-/

Thanks guys!


0

Sponsored Link
Ads by Google
Reply to Message Icon






Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Programming Forum Home


Sponsored links

Ads by Google


Results for: Batch file for processing logfiles

Batch file for Outlook www.computing.net/answers/programming/batch-file-for-outlook/15439.html

Batch file for renaming folders www.computing.net/answers/programming/batch-file-for-renaming-folders/15430.html

automating a batch file process www.computing.net/answers/programming/automating-a-batch-file-process/18203.html