Computing.Net > Forums > Programming > Extract rows from pipe delim file

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Extract rows from pipe delim file

Reply to Message Icon

Name: b1pal
Date: October 1, 2009 at 09:18:32 Pacific
OS: Windows XP
Subcategory: Batch
Comment:

I have a file with header and trl and has below data. I need to count # of rows in a file.

1111111|R8|12/31/2007|03/06/2007|03/08/2007|03/06/2007|77.7778|0|||||A|4||||
2222222|R8|12/31/2007|03/06/2007|03/08/2007|03/08/2007|66.6667|0|||||Am|4||||

The pipe is causing an issue. It gives error: "|| was upexpected at this time."

Below is my code:
@echo off & setLocal enableDELAYedexpansion

:today
set today=%DATE:~10,4%
set today=%today%%DATE:~4,2%
set today=%today%%DATE:~7,2%
set processfile=c:\V1

echo.
:main
for /f "tokens=* delims= " %%a in ('dir/b *.txt') do (
set input=
set input=%%a
:count lines; set T & R
for /f "tokens=* delims= " %%d in ('find /v /c "" ^< %%a') do (
set T=%%d
set /a R=!T!
)
:chk contents of one csv & set head & tail
set N=0
for /f "tokens=* delims= " %%i in (%%a) do (
set /a N+=1
if !N! equ 1 set head=%%i
if !N! equ !T! set tail=%%i
)

rem stripping off the zeros
set tail=!tail:~3!
:loop
if !tail:~0^,1! equ 0 (
set tail=!tail:~1!
call :loop
)
: chk date and record cout from head and tail
if !head:~3^,8! equ !today! echo current date is ok
(if !tail! equ !R! echo record count is ok
(if !head:~3^,8! equ !today! (
if !tail! equ !R! (
call :noFIRST
call :chopLAST
))
)
)
)
:end chk contents of one csv & set head & tail
)
:end main
goto :eof

:noFIRST
::== noFIRST.bat
@echo off > NewViasinc1.txt
set FIRST=Y
for /f "tokens=*" %%L in (!input!) do call :1 %%L
goto :eof
:1
if %FIRST%==N echo %>>NewViasinc1.txt
set FIRST=N
goto :eof
:chopLAST
::==chopLAST.bat

find /v "" NewViasinc1.txt > ~Temp.txt
for /f "tokens=1,* delims=[]" %%a in (~Temp.txt) do set Last=%%a
type NewViasinc1.txt | find /v "%Last%" >> NewViasinc2.txt
copy NewViasinc2.txt !input!
del New*.txt
del ~Temp.txt
move !input! !processfile!
set input=
set R=
set T=
goto :eof
::== DONE

goto :eof

Thanks for your help.



Sponsored Link
Ads by Google

Response Number 1
Name: Mechanix2Go
Date: October 1, 2009 at 10:16:51 Pacific
Reply:

1111111|R8|12/31/2007|03/06/2007|03/08/2007|03/06/2007|77.7778|0|||||A|4||||
2222222|R8|12/31/2007|03/06/2007|03/08/2007|03/08/2007|66.6667|0|||||Am|4||||

Are those the lines in the file?


=====================================
Helping others achieve escape felicity

M2


0

Response Number 2
Name: Razor2.3
Date: October 1, 2009 at 10:57:28 Pacific
Reply:

I need to count # of rows in a file.

1111111|R8|12/31/2007|03/06/2007|03/08/2007|03/06/2007|77.7778|0|||||A|4||||
2222222|R8|12/31/2007|03/06/2007|03/08/2007|03/08/2007|66.6667|0|||||Am|4||||
2.


Also, this:

for /f "tokens=2 delims=:" %%a in ('find /v /c "" someFile.txt') do @echo %%a


0

Response Number 3
Name: Judago
Date: October 1, 2009 at 11:05:33 Pacific
Reply:

@Razor2.3


for /f "tokens=2 delims=:" %%a in ('find /v /c "" subMenu.js') do @echo %%a

If you pipe or redirect into find /c you won't get the leading space
(or have to worry about spaces in file names). Though you could
always use set /a to kill off the space....


Batch Variable how to


0

Response Number 4
Name: Razor2.3
Date: October 1, 2009 at 11:13:06 Pacific
Reply:

Judago: Though you could always use set /a to kill off the space....
It's what I'd do, as I like to avoid special characters in the (set) part of my FOR statements as much as possible. It's one of those style things, more often than not.


0

Response Number 5
Name: Judago
Date: October 1, 2009 at 11:23:34 Pacific
Reply:

It's what I'd do, as I like to avoid special characters in the
(set) part of my FOR statements as much as possible. It's one
of those style things, more often than not.

Yeah everyone has their own way of doing things, readability
usually wins out in the end. My view to it is if I can come back to
something complex after a little while and still understand it I did
it right, doesn't always happen though.....


Here is a novel way to count the columns by delimiter, you may
may want to add one to the result if their is data after the last delimiter:

setlocal enabledelayedexpansion
set "test=1111111|R8|12/31/2007|03/06/2007|03/08/2007|03/06/2007|77.7778|0|||||A|4||||"
for /f %%a in ('cmd /u /c echo "!test!"^|find /c "|"') do echo %%a
pause

[edit]
For some strange reason I counted the columns??.....
[/edit]


Batch Variable how to


0

Related Posts

See More



Response Number 6
Name: b1pal
Date: October 1, 2009 at 11:38:31 Pacific
Reply:

File has around 20K rows:

HDR20091001
1111111|R8|12/31/2007|03/06/2007|03/08/2007|03/06/2007|77.7778|0|||||A|4||||
2222222|R8|12/31/2007|03/06/2007|03/08/2007|03/08/2007|66.6667|0|||||Am|4||||
--
--
--
TRL000020001

I am getting pipe error on line:

if %FIRST%==N echo %>>NewViasinc1.txt

when it is reading the file.

Can you please let me know what I need to change in the above script?

Thank you all for replying.


0

Response Number 7
Name: Mechanix2Go
Date: October 1, 2009 at 11:48:25 Pacific
Reply:

Ignoring the date check and the stripping of zeros...

If the header and footer are strictly one line each, with no pipe, and there are no blank lines, is it still a problem to count lines?

Maybe I'm missing something here.


=====================================
Helping others achieve escape felicity

M2


0

Response Number 8
Name: Judago
Date: October 1, 2009 at 11:56:49 Pacific
Reply:

Perhaps that single % is meant to be %1.

Instead of "if %FIRST%==N echo %>>NewViasinc1.txt"

I would suggest:

set "somevar=%1"
if %FIRST%==N >>NewViasinc1.txt echo !somevar!


Batch Variable how to


0

Response Number 9
Name: b1pal
Date: October 2, 2009 at 11:22:47 Pacific
Reply:

Yes, header and trailer are on one line each. I tried what Judago suggest and it still did not work and gives | (pipe) error => unexpected at this time.

Not sure what's going on.


0

Response Number 10
Name: Razor2.3
Date: October 2, 2009 at 13:00:10 Pacific
Reply:

Have you considered a language which won't try to parse your text as a command?


0

Response Number 11
Name: Judago
Date: October 2, 2009 at 19:59:19 Pacific
Reply:

Ok I *think* I have found the problem.

for /f "tokens=*" %%L in (!input!) do call :1 "%%L"
goto :eof
:1
set "somevar=%~1"
if %FIRST%==N >>NewViasinc1.txt echo !somevar!


Batch Variable how to


0

Response Number 12
Name: Mechanix2Go
Date: October 3, 2009 at 07:52:07 Pacific
Reply:

@echo off & setLocal EnableDELAYedExpansion

set N=
for /f "tokens=* delims= " %%a in (myfile) do (
set /a N+=1
)

set /a N-=2
echo !N! lines, excluding head and tail


=====================================
Helping others achieve escape felicity

M2


0

Response Number 13
Name: b1pal
Date: October 6, 2009 at 07:34:37 Pacific
Reply:

Its working!!! Thank you so much Judago.


0

Sponsored Link
Ads by Google
Reply to Message Icon





Use following form to reply to current message:

Login or Register to Reply
LoginRegister


Sponsored links

Ads by Google


Results for: Extract rows from pipe delim file

Extracting rows from pipe delimited file www.computing.net/answers/programming/extracting-rows-from-pipe-delimited-file/19338.html

Extracting rows from multiple csv files using www.computing.net/answers/programming/extracting-rows-from-multiple-csv-files-using/20204.html

batch to extract rows from files www.computing.net/answers/programming/batch-to-extract-rows-from-files/15625.html