Solved How to? find in log last 15 lines with txt criteria on line

Mobile computing solutions Mini itx pfse...
December 4, 2014 at 01:01:15
Specs: Windows XP, HexaCore AMD FX-6100, 1400 MHz (7 x 200) / 4gb DDR
Dear Computing net,
How Can I find Last line in log with specific text content.

I made a batch file that checks my log files every 15 seconds BUT it finds the same entries within
my log and not th last active download below is one sample line from my log which I want to parse
and find the following.
----------------------------------------------------------------------------------------------------------------------------------
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> RETR autoclean.new
----------------------------------------------------------------------------------------------------------------------------------
THE LOG FILE HAS A FORMAT OF EACH SESSION STARTING WITH A 6 DIG Index (000065)
THIS INDEX GENERATED BY FILEZILLA Increments for each ftp session logged.
So need to find LAST index block i.e. 000065 and search from their for new entries with
RETR FILENAME

I am using a time variable to get current time into a VAR but don't know how to parse the log
file to find the last download success, so I wonder if anyone can show me HOW to find the
line with the current hour and minute, OR last Index Marker and if that line contains RETR FILENAME

When it does I am incrementing my COUNTER BY ONE, the problem is the script I currently
use finds any entry in the log containing these words.

To simplify what command I am currently using is as below
>nul find "RETR FILENAME.bat" %filezilllog%

This is where I am getting stuck can anyone help ?

Michelle xoxox

If Dreams Come True Oleg Would be Famous so far he's very shy, so much for Being Famous ;) http://tinyurl.com/pnenqgb


See More: How to? find in log last 15 lines with txt criteria on line

Report •


✔ Best Answer
December 6, 2014 at 15:57:08
Hello again. Yeh, I just used chksum for my testing. Just put the files you want to track into a control-file (called, oddly enough, CONTROL in my code)
::======= begin batchscript ORT4.BAT
@echo off & setlocal
set ftplog=filezilla.log
for /f %%z in (control) do call :poll %%z %%~nz
goto :eof

:poll
set log=%~n2
set retr=%1
set last=
echo polling %retr%
set q=---------- polled: %date% %time%
set scan=more

::create new log
if not exist %log% (
>%log% echo delete_this
set scan=type
set skip=
goto :zz
)

::get first line to get the last RETR polled
set /p k=<%log%
echo k is !k!
:: now find the line# where that RETR occured, to skip past it in zz
for /f "tokens=1 delims=[]" %%a in ('find /n "!k!"^<%ftplog%') do (
echo line %%a
set skip=+%%a
goto :zz
)
goto :eof

:zz
for /f "tokens=*" %%b in ('%scan% %skip% %ftplog%^|find "RETR %retr%"') do (
set last=%%b
if defined q >>%log% echo !q!&set q=
>>%log% echo %%b
) 2>nul

:done
if not defined last goto :eof
>tmp echo !last!
more +1 %log%>>tmp
move /y tmp %log%
goto :eof
::====== END BATCHSCRIPT



#1
December 4, 2014 at 17:45:07
Hello again my friend! maybe something like this:
@echo off & setlocal
for /f "tokens=1 delims=:" %%a in ('findstr /n /r "^([0-9][0-9][0-9][0-9][0-9][0-9])" ort.log') do set k=%%a
set /a k-=1
echo %k%
more +%k% ort.log|find "RETR">ortlast.log
::===== end snippet
not sure exactly what you want. PM me with some sample logs and how you want to parse/extract the data, and I will improve on this (primarily the findstr RE will need tweaked to grab the RETR, i think).
I'm not sure how this logging works, but there might be more efficient ways to do what you want, f/e: testing for filesize change before doing the extraction...

message edited by nbrane


Report •

#2
December 5, 2014 at 05:41:48
Thanks nbrane,
Yes sorry my question was a bit grey around the edges as was still trying to assimilate the ordering in my head, but from what I have worked out it will probably need to do a backquote of sorts and search from THE BOTTOM of the log file up! In reverse order of the date the entries were written.

I will paste some examples below: ( I CHANGED EXTERNAL I.P.'S TO PROTECT Whoever They were but left the LAN I.P.'s ) for example I will PM you with a full log nbrane

Regards Michelle xoxoxo

(000063) 04/12/2014 05:58:25 - (not logged in) (640.19.16.235)> Connected, sending welcome message...
(000063) 04/12/2014 05:58:25 - (not logged in) (640.19.16.235)> could not send reply, disconnected.
(000064) 04/12/2014 06:00:49 - (not logged in) (148.135.195.12)> Connected, sending welcome message...
(000064) 04/12/2014 06:00:49 - (not logged in) (148.135.195.12)> could not send reply, disconnected.
(000065) 04/12/2014 06:01:33 - (not logged in) (192.168.2.1)> Connected, sending welcome message...
(000065) 04/12/2014 06:01:33 - (not logged in) (192.168.2.1)> USER AUTOCLEAN
(000065) 04/12/2014 06:01:33 - (not logged in) (192.168.2.1)> 331 Password required for autoclean
(000065) 04/12/2014 06:01:33 - (not logged in) (192.168.2.1)> PASS ******
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 230 Logged on
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> pasv
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 227 Entering Passive Mode (192,168,2,10,8,48)
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> TYPE I
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 200 Type set to I
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> CWD /batch
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 250 CWD successful. "/batch" is current directory.
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> TYPE A
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 200 Type set to A
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,7
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> LIST
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 150 Opening data channel for directory list.
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> TYPE I
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 200 Type set to I
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,8
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> RETR autoclean.new
(000065) 04/12/2014 06:01:33 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:35 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:35 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,9
(000065) 04/12/2014 06:01:35 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:35 - autoclean (192.168.2.1)> RETR apps/SayStatic.exe
(000065) 04/12/2014 06:01:35 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:38 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:38 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,10
(000065) 04/12/2014 06:01:38 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:38 - autoclean (192.168.2.1)> RETR apps/ftps.exe
(000065) 04/12/2014 06:01:38 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,11
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> RETR ortnew.txt
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,12
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> RETR version
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,13
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> RETR chksum
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> PORT 192,168,2,1,22,14
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 200 Port command successful
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> RETR UNINSTALL-I.E..BAT
(000065) 04/12/2014 06:01:48 - autoclean (192.168.2.1)> 150 Opening data channel for file transfer.
(000065) 04/12/2014 06:01:49 - autoclean (192.168.2.1)> 226 Transfer OK
(000065) 04/12/2014 06:01:49 - autoclean (192.168.2.1)> QUIT
(000065) 04/12/2014 06:01:49 - autoclean (192.168.2.1)> 221 Goodbye
(000065) 04/12/2014 06:01:49 - autoclean (192.168.2.1)> disconnected.

If Dreams Come True Oleg Would be Famous so far he's very shy, so much for Being Famous ;) http://tinyurl.com/pnenqgb


Report •

#3
December 5, 2014 at 05:53:45
1/ I have no problem finding the log file date itself

2/ Problem is finding last index i.e. (000001) However the numbering system is not
accurate if server restarts so does the numbering change to earlier numeric values

3/ Finding the line with date appears to be more reliable i.e. 04/12/2014 then find if on
that line RETR FILENAME.new EXISTS

Because I also do not want to keep finding the same entries either as each entry adds 1 to my counter to keep track of download NUMBERS
I am NOT logging personal i.p.'s or anything else in my counter, I am
just trying to determine over time if my Auto scheduler batch is in demand, if over time it is not I can decide to either not continue
updating it or just keep it private but for now I don't know.

Michelle

If Dreams Come True Oleg Would be Famous so far he's very shy, so much for Being Famous ;) http://tinyurl.com/pnenqgb

message edited by Ortorea


Report •

Related Solutions

#4
December 5, 2014 at 21:18:57
Ha ha! You're such a whirl-wind, this lazy cretin cannot keep up! As for the log file, I say, bugger the dates. I hate trying to deal with them. I think I get the gist of what you want:
poll the logfile, looking for RETR
count/monitor any RETR since the last poll, but ignore any earlier
add the new-count to a counter for xx.bat (or any other extension)

Here's my approach. For any given file:
find the last RETR for that file
check if that is same as last stored
If same, do nothing, ELSE
find the line-number of the last stored (to skip prior entries),
from there, (more +n) count all RETR targeting that file and accumulate into a results file for that file
Then store the LAST RETR as a benchmark for the next round.
Maybe it sounds complicated. It's not. We're using (for any given file) the last RETR, as a tag. Look in the log for that tag, then accumulate any RETR after that point (with the filename as target). Put last RETR into benchmark...
I don't want to write script till I'm sure we're on the same page. Also there's the consideration of sampling-rate vs on-demand sampling. There's really no need to monitor every 15 seconds, but that's up to you. My personal preference would be either on-demand or longer sample-span (I doubt if even Michael Jackson got a hit every 15 seconds, Ha ha!)


Report •

#5
December 5, 2014 at 23:48:19
Hi nbrane, no I am not that fast, I PM'd you about a silly thing really, I got this other
script to run as ADMIN and suddenly realised all my junk was being stored in
system32 OMG yes what a mess had to weed through the system files looking for
my junk, thought I found the answer but OH OHHH nope still not there then found this page

http://serverfault.com/questions/25...

So the mystery about where all my flags were ending up on my friends PC solved :D

As when one uses UAC you find your working path changed so setting a VAR early on to store the path is handy so later you just type.

chdir /d %mypath%

- to get back to your batch working directory finally after hours of searching got it.and I am absolutely whacked out so need sleep now and will have to play with your new code tomorrow I am sorry to say as, I know once I start I can't put it down lol.

Michelle xoxox

PS, Yes just reading your comments above, some of the RETR should not be counted as they are companion files such as chksum or ortnew.txt is basically my equivalent README.txt kind of slang for us Brits who might say Is their anything New, or ORT-NEW as slang, so just the main files to watch for like RETR Autoclean.new which when retrieved gets compared against the also sent checksum file to make sure it is not damaged, then it over writes the batch on the other end.

Michelle

If Dreams Come True Oleg Would be Famous so far he's very shy, so much for Being Famous ;) http://tinyurl.com/pnenqgb

message edited by Ortorea


Report •

#6
December 6, 2014 at 15:57:08
✔ Best Answer
Hello again. Yeh, I just used chksum for my testing. Just put the files you want to track into a control-file (called, oddly enough, CONTROL in my code)
::======= begin batchscript ORT4.BAT
@echo off & setlocal
set ftplog=filezilla.log
for /f %%z in (control) do call :poll %%z %%~nz
goto :eof

:poll
set log=%~n2
set retr=%1
set last=
echo polling %retr%
set q=---------- polled: %date% %time%
set scan=more

::create new log
if not exist %log% (
>%log% echo delete_this
set scan=type
set skip=
goto :zz
)

::get first line to get the last RETR polled
set /p k=<%log%
echo k is !k!
:: now find the line# where that RETR occured, to skip past it in zz
for /f "tokens=1 delims=[]" %%a in ('find /n "!k!"^<%ftplog%') do (
echo line %%a
set skip=+%%a
goto :zz
)
goto :eof

:zz
for /f "tokens=*" %%b in ('%scan% %skip% %ftplog%^|find "RETR %retr%"') do (
set last=%%b
if defined q >>%log% echo !q!&set q=
>>%log% echo %%b
) 2>nul

:done
if not defined last goto :eof
>tmp echo !last!
more +1 %log%>>tmp
move /y tmp %log%
goto :eof
::====== END BATCHSCRIPT


Report •

#7
December 6, 2014 at 17:07:28
Dear nbrane wow, that's awesome I doubt I could have come up with that, as it's way too advanced for me lol Something else I never mentioned is I have filezilla set to create a new log each day so that should simplify caching issues as each day a new log is created it never really gets too large usually under 100 kb

I will test this out tonight but my log just reset for now as it is past midnight.

Michelle

If Dreams Come True Oleg Would be Famous so far he's very shy, so much for Being Famous ;) http://tinyurl.com/pnenqgb


Report •


Ask Question