Computing.Net > Forums > Programming > Filter IIS log files to remove internal IP's

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Filter IIS log files to remove internal IP's

Reply to Message Icon

Name: aNoMiS66
Date: April 29, 2009 at 02:45:30 Pacific
OS: Windows 2003 Server
CPU/Ram: 3Ghz / 1GB
Subcategory: Batch
Comment:

It's a long time since I last wrote a .bat file. I need to parser an IIS log file and and output another file which contains just external IP addresses. I have written two batch files to do this. The 1st contains the lines:

SET yy=%date:~8,4%
SET mm=%date:~3,2%
SET dd=%date:~0,2%
SET N=%yy%%mm%%dd%

FOR /F "TOKENS=* SKIP=3 DELIMS=\n" %%a IN (^%N%.log) DO (
	ECHO %%a > log_temp.tmp
	CALL log_filter.bat log_temp.tmp
)

And the 2nd called log_filter.bat contains

TYPE d:\log_temp.tmp|FIND "192.168"
IF ERRORLEVEL 1 GOTO NOT_FOUND
IF ERRORLEVEL 0 GOTO XIT 

:NOT_FOUND
	TYPE d:\log_temp.tmp >> d:\log_cleaned.txt
:XIT

It works - but hits the server resources, CPU 100% whilst processing a 35MB file and takes an hour to complete.

Is there a way to read each line into memory to improve performance, rather than output for each line being written to a temp file?



Sponsored Link
Ads by Google

Response Number 1
Name: Mechanix2Go
Date: April 29, 2009 at 04:52:23 Pacific
Reply:

I think you'll probably want a real program to get any speed. But if you're stuck with a bat here's a couple observations.

The first FOR CALLs the other bat for every line in the log. A recipe for overload.

IF ERRORLEVEL 0 GOTO

is meaningless because it's ALWAYS 0 or more.


=====================================
If at first you don't succeed, you're about average.

M2


0

Response Number 2
Name: aNoMiS66
Date: April 29, 2009 at 05:41:25 Pacific
Reply:

Thanks for your input. I'll have a review of my options. Maybe a BATCH file isn't the best solution.


0

Response Number 3
Name: klint
Date: April 29, 2009 at 06:24:07 Pacific
Reply:

Try this:

SET yy=%date:~8,4%
SET mm=%date:~3,2%
SET dd=%date:~0,2%
SET N=%yy%%mm%%dd%

(FOR /F "TOKENS=* SKIP=3 DELIMS=\n" %%a IN (^%N%.log) DO (
   ECHO %%a|find /v "192.168"
)) > d:\log_cleaned.txt

By the way, I'm not sure what the delims=\n is for, or what ^ is doing in ^%N% in your code. But I've left them as they were.


0

Response Number 4
Name: Mechanix2Go
Date: April 29, 2009 at 07:40:17 Pacific
Reply:

I don't figure delims=\n but I don't have the log.

If you need to skip 3 lines and get out all the lines not containing 192.168 try this:

=======================
@echo off > log_temp.tmp & setLocal EnableDelayedExpansion

FOR /F "TOKENS=* SKIP=3 DELIMS=\n" %%a IN (the.log) DO (
ECHO %%a >> log_temp.tmp
)

FIND /v "192.168" < log_temp.tmp > d:\log_cleaned.txt


=====================================
If at first you don't succeed, you're about average.

M2


0

Response Number 5
Name: ghostdog
Date: April 29, 2009 at 18:57:08 Pacific
Reply:

tedious. use logparser.

Unix Win32 tools | Gawk for Windows


0

Related Posts

See More



Sponsored Link
Ads by Google
Reply to Message Icon

c++ structure Formula is changing on it...



Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Programming Forum Home


Sponsored links

Ads by Google


Results for: Filter IIS log files to remove internal IP's

Script to remove TCP/IP Printers www.computing.net/answers/programming/script-to-remove-tcpip-printers/15955.html

Bat file to remove antivirus 2009 www.computing.net/answers/programming/bat-file-to-remove-antivirus-2009/17484.html

Help ... READ a log file... www.computing.net/answers/programming/help-read-a-log-file/16493.html