one line parsing

Custom / CUSTOM
January 24, 2010 at 15:48:56
Specs: Microsoft Windows XP Professional, 2.401 GHz / 2047 MB
Having trouble to parse one line of output, breaking it down into specific parts. To be more precise, I have a command with an output like :

1 12 56 7 9 56

It's always one line, separated by whitespaces. The amount of columns is variable however, and it can be quite some.

So, I was thinking to use the FOR /F loop:

for /F "tokens=* delims= " %a in ('mycommand') do (
echo %a
)

That shows the complete line, and I know how to show the first one only (tokens=1) ... but I don't know how many columns follow (and as stated, it may be a lot, so more than 26 is possible) ... so, how do you do that ?

How do you smartly cover all of the columns, without too much coding ?


See More: one line parsing

Report •


#1
January 24, 2010 at 16:48:17
here's one way:
for /F "tokens=* delims= " %%a in ('mycommand') do set test=%%a
set /a nn=1
:01
for /f "tokens=1* delims= " %%b in ("%test%") do (
set v%nn%=%%b
set test=%%c
echo.%nn%: !v%nn%!
)
set /a nn+=1
if "%test%" neq "" goto :01

Report •

#2
January 24, 2010 at 21:21:29
For loop variables don't run from a-z they run over ascii.

Here is a way over the top example, 91 tokens.

I just wrote so there may be an error or two. Note that
two of the variables are reset.

If there is a way to use "%" and "~", it could go up a couple
of vars....

@echo off
set t=1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 
for /f "tokens=1-3,* delims= " %%! in ("%t%") do (
    echo %%! + %%" + %%# +
    echo %%$
    echo.
    for /f "tokens=1-30,* delims= " %%^& in ("%%$") do (
        echo %%^& + %%' + %%^( + %%^) + %%* + %%+ + %%, + %%- + %%. + %%/ + 
        echo %%0 + %%1 + %%2 + %%3 + %%4 + %%5 + %%6 + %%7 + %%8 + %%9 +
        echo %%: + %%; + %%^< + %%= + %%^> + %%? + %%@ + %%A + %%B + %%C + 
        echo %%D
        echo.
        rem look at the abuse %%D take below for an extra token
        for /f "tokens=1-30,* delims= " %%D in ("%%D") do (
            echo %%D + %%E + %%F + %%G + %%H + %%I + %%J + %%K + %%L + %%M +
            echo %%N + %%O + %%P + %%Q + %%R + %%S + %%T + %%U + %%V + %%W +
            echo %%X + %%Y + %%Z + %%[ + %%\ + %%] + %%^^ + %%_ + %%` + %%a +
            echo %%b
            echo.
            rem note to abuse %%b
            for /f "tokens=1-28 delims= " %%b in ("%%b") do (
                echo %%b + %%c + %%d + %%e + %%f + %%g + %%h + %%i + %%j + %%k +
                echo %%l + %%m + %%n + %%o + %%p + %%q + %%r + %%s + %%t + %%u +
                echo %%v + %%w + %%x + %%y + %%z + %%{ + %%^| + %%}
            )
        )
    )
)
pause


Here is the same script with all of the variables being used in the inner loop:

@echo off
set t=1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
for /f "tokens=1-3,* delims= " %%! in ("%t%") do (
    for /f "tokens=1-30,* delims= " %%^& in ("%%$") do (
        rem look at the abuse %%D take below for an extra token
        for /f "tokens=1-30,* delims= " %%D in ("%%D") do (
            rem not to abuse %%b
            for /f "tokens=1-28 delims= " %%b in ("%%b") do (
                echo %%! + %%" + %%# +
                echo %%^& + %%' + %%^( + %%^) + %%* + %%+ + %%, + %%- + %%. + %%/ + 
                echo %%0 + %%1 + %%2 + %%3 + %%4 + %%5 + %%6 + %%7 + %%8 + %%9 +
                echo %%: + %%; + %%^< + %%= + %%^> + %%? + %%@ + %%A + %%B + %%C + 
                echo %%D + %%E + %%F + %%G + %%H + %%I + %%J + %%K + %%L + %%M +
                echo %%N + %%O + %%P + %%Q + %%R + %%S + %%T + %%U + %%V + %%W +
                echo %%X + %%Y + %%Z + %%[ + %%\ + %%] + %%^^ + %%_ + %%` + %%a +
                echo %%b + %%c + %%d + %%e + %%f + %%g + %%h + %%i + %%j + %%k +
                echo %%l + %%m + %%n + %%o + %%p + %%q + %%r + %%s + %%t + %%u +
                echo %%v + %%w + %%x + %%y + %%z + %%{ + %%^| + %%}
            )
        )
    )
)
pause


Batch Variable how to


Report •

#3
January 25, 2010 at 00:41:07
Tested up to 550 tokens. After that, who knows.

=============================

@echo off & setLocal enableDELAYedeXpansion

set N=

for /f "tokens=* delims= " %%a in (myfile) do (
call :sub1 %%a
)

echo there are !N! tokens
echo the last one is !T%N%!
set /p C=to see another, emter it's number : 
if !C! gtr !N! echo nice try && goto :eof
echo !T%C%!

goto :eof

:sub1
  :loop
    set /a N+=1
    set T!N!=%1
    if not "%2"=="" (
      shift
      goto :loop
    )

goto :eof


=====================================
Helping others achieve escape felicity

M2


Report •

Related Solutions

#4
January 25, 2010 at 06:45:40
I'm with nbrane on this one. If you need that much data, the tokens=1* would be your best bet. If you need to actually manipulate each column, you should probably look to a better scripting language, like VBScript.

Report •

#5
January 25, 2010 at 10:03:41
I'll have a test with the above ones ... but, I already fired up my old BorlandC compiler ;)

Report •

#6
January 28, 2010 at 04:21:01
> for /F "tokens=* delims= " %%a in ('mycommand')
> do set test=%%a
> set /a nn=1
> :01
> for /f "tokens=1* delims= " %%b in ("%test%")
> do (
> set v%nn%=%%b
> set test=%%c
> echo.%nn%: !v%nn%!
> )
> set /a nn+=1
> if "%test%" neq "" goto :01
>

My fault, by I didn't mention that the one line of code also contains multiple occurances of the double quote sign ... it messes up the code ...
The code may still work, with some adaptations, I'll see if I can get it working. Note that I do not really need to store each whitespace separated value as a variable. I'm doing some IF's and such, but I don't need each one in a separate variable ..


Report •

#7
January 28, 2010 at 04:41:49
M2, was trying your solution as well, but I can't get to grips with the FOR /F loop ... it doesn't do what it should. (Got that one well now, seems that we cannot use "tokens=*" but rather "tokens=1-31" ... gives a different result)

Judago : will try yours as well...

edit: Judago, that's what I need. Obviously it needs some tweaking to the code, but using a lot of tokens, and using the single-digit characters, but other than a-z and A-Z, is what I need ...

26 would not be enough, but provided with the example code of yours, I see how to expand it. I'm also seeing that FOR /? is totally out of what is actually possible, so you would never get it out of the manual, would you ?


Report •

#8
January 28, 2010 at 05:58:39
I don't know what you're after that #3 doesn't do.


=====================================
Helping others achieve escape felicity

M2


Report •

#9
January 28, 2010 at 06:45:53
tvc,

The problem with the for /f loop is that it will take only 31 tokens so it has to be "1-30,*" with "*" being parsed in the next loop and if "*" happens to be blank the inner loops won't execute. So the first example is probably the way to go, making the inner loops automatically conditional, even then there could be as many as 29 empty variables in the last loop to execute.

Also escaping some of the characters can prove annoying.....

It's purely academic but I managed to get 94 tokens, though "%" and "~" must be used with the tilde, striping any surrounding quotes : http://pastebin.com/m22023a08. It displays the chars that are used as the variables.

Also I got the trick from here, but expanded it.


Batch Variable how to


Report •

#10
January 28, 2010 at 07:10:46
> I don't know what you're after that #3 doesn't do.
>

It's my fault, I have additional requirements that I've underestimated ...

Have a look at this, it's your code, but I've changed the input values, and I changed the delimitor:

---
This is the ouput:

D:\Temp>test.cmd
there are 6 tokens
the last one is 56
to see another, emter it's number : 4
7

D:\Temp>


---
This is the code:

@echo off & setLocal enableDELAYedeXpansion

echo 1 12;56 7;9 56> myfile

set N=

for /f "tokens=* delims=;" %%a in (myfile) do (
call :sub1 %%a
)

echo there are !N! tokens
echo the last one is !T%N%!
set /p C=to see another, emter it's number :
if !C! gtr !N! echo nice try && goto :eof
echo !T%C%!

goto :eof

:sub1
:loop
set /a N+=1
set T!N!=%1
if not "%2"=="" (
shift
goto :loop
)

goto :eof




Report •

#11
January 28, 2010 at 07:12:21
Judago, I perfectly understand ... it's just that I used "tokens=*" in what I thought is a replacement for "tokens=1-xx" (where xx is some maximum number)

... but that does not seem to be true


Report •

#12
January 28, 2010 at 07:29:31
tvc,

I see the snag but not the cure.

I'll work on it.


=====================================
Helping others achieve escape felicity

M2


Report •

#13
January 28, 2010 at 08:33:39
"Judago, I perfectly understand ... it's just that I used "tokens=*" in what I thought is a replacement for "tokens=1-xx" (where xx is some maximum number)

... but that does not seem to be true"

Yeah * tells the for loop not to split or not to split after x tokens, but I guess you have figured that out already. There is no way to implicitly split on the maximum tokens of the output, the only option is to specify your max and check that the output vars aren't blank.


Batch Variable how to


Report •

#14
January 28, 2010 at 09:34:30
This is a workaround. I can't think of one to handle general cases.

It replaces spaces with _ before doing, in your case, the semicolons.

=============================

    @echo off > nospfile & setLocal enableDELAYedeXpansion
    
    echo 1 12;56 7;9 56> myfile

    set N=

for /f "tokens=*" %%i in (myfile) do (
set S=%%i
set S=!S: =_!
>> nospfile echo !S!
)

    for /f "tokens=* delims=;" %%a in (nospfile) do (
    call :sub1 %%a
    )

set X=!T%N%!
set X=!X:_= !

    echo there are !N! tokens
    echo the last one is !X!
    set /p C=to see another, emter it's number :
    if !C! gtr !N! echo nice try && goto :eof

set X=!T%C%!
set X=!X:_= !

    echo !X!

    goto :eof

    :sub1
    :loop
    set /a N+=1
    set T!N!=%1
    if not "%2"=="" (
    shift
    goto :loop
    )

    goto :eof


=====================================
Helping others achieve escape felicity

M2


Report •

#15
January 30, 2010 at 02:53:02
What is one line checkout buddy!!!

Report •

#16
January 30, 2010 at 05:09:57
> What is one line checkout buddy!!!
>

What is what ?


M2: Thanks, I'll have a look


Report •


Ask Question