Solved weird execution delay in large.bat file

Hidde663 / My first build
March 2, 2018 at 03:50:25
Specs: Windows 10 Pro 64bit, i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
Hey guys, i have 2 .bat files, both have the same loop in them, same code & all, the only diffrence is filesize.
One of which is executing the loop in 18,5 seconds, the other 3,5 seconds.

Does anyone know why & is there a workaround for this?
(for /l loop does not work in this case)
(placing all the code to the front of the file does not work)

anyways, ive been working on a game with batch only which used strings as maps.

here is part of one such string(lenght is 1023 chars):
ÿÿÿÿÿÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛ±ÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÿÿÿ°°ÿÿÿÿÿÛ66ÿÿÿÛÿÿÿ°°ÿÿÿÿÿÛ66ÿÿÿÛÿÿÿ°°ÿÿÿÿÿÛÿÿÿÿÿÿÿÿÿÿÿÿÛ6ÿÿÿÿÛÿÿÿ°°ÿÿÿÿÿÛÄÄÄÄÄÄÄÄÄÄÄÄÛÛÛÛÛÛÛÛÛÛÿÿÿÿÿÿÿÿÿÛÿÿÿ°°ÿÿÿÿÿÛÿÿÿÿÿÿÿÿÿÿÿÿÛ9Ûÿ

The numbers represent data that is repetitive, in other words long known strings which can be set to a variable& then be de-compressed while loading a map.(im doing this all to make the file size smaller, about 43KB smaller).

the thing is, i have 2 bat files: a debugger/mapmaker & the main game.
the main game is 346KB & the debugger is 31KB (might be the issue?)

echo main 1:%time%>>timepls.txt gives me the start of the loop in time & main 2: gives me the endtime of the loop
same for the debugger only there its called debug 1 & 2

again debug takes 3,5 seconds to finish while main takes 18,5 seconds

this is a snipp of code:

:SLR
set strg=-data
if %music% EQU 1 taskkill /F /IM wscript.exe>nul&start Sound.vbs !SURL!
for /f "skip=2 tokens=1 delims=[]" %%a in ('find /N ":map%strg%" "PokeBatch2(D).bat"') do set map-data-line=%%a
for /f "skip=2 tokens=2 delims=[];" %%a in ('find /N /I ";%lm%" "PokeBatch2(D).bat"') do set/a test1=%%a+%map-data-line%-1&for /f %%a in ('more "PokeBatch2(D).bat" +!test1!') do set "m=%%a"&goto SLRNext
:SLRNext
for /f %%a in ('more "PokeBatch2(D).bat" +%test1%') do set "m=%%a"&goto SLRNext2
:SLRNext2
set "Noth=ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ"
set "bloc=ÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛÛ"
set "wat=~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
set "grasbl=°°°°°°°°°°°°°°°°°°"
set "ndbl=%bloc:~-17%"
set bef=0
set aft=1
echo main 1:%time%>>timepls.txt
:lprr
if "!m:~%bef%,1!"=="6" set "m=!m:~0,%bef%!%Noth%!m:~%aft%!"&set/a bef+=14&set/a aft+=14&goto lprr
if "!m:~%bef%,1!"=="7" set "m=!m:~0,%bef%!%bloc%!m:~%aft%!"&set/a bef+=40&set/a aft+=40&goto lprr
if "!m:~%bef%,1!"=="8" set "m=!m:~0,%bef%!%wat%!m:~%aft%!"&set/a bef+=33&set/a aft+=33&goto lprr
if "!m:~%bef%,1!"=="9" set "m=!m:~0,%bef%!%grasbl%!m:~%aft%!"&set/a bef+=16&set/a aft+=16&goto lprr
if "!m:~%bef%,1!"=="5" set "m=!m:~0,%bef%!%ndbl%!m:~%aft%!"&set/a bef+=17&set/a aft+=17&goto lprr
::1031-9-14
if !aft! LEQ 1008 set/a bef+=1&set/a aft+=1&goto lprr
echo main 2:%time%>>timepls.txt
::fkn +15 second delay for this compared to the same code in the debugger
if exist tlm.txt echo off>tlm.txt
set c1=0&for /L %%a in (1,1,24) do for /L %%A in (1,1,43) do echo %%a,%%A=!c1!>>tlm.txt&set/a c1+=1
for /f "tokens=1,2 delims==" %%a in (tlm.txt) do set %%a=!m:~%%b,1!
for /L %%a in (36,1,43) do set 24,%%a=Û

Please tell me if i was not clear enough on something

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450


See More: weird execution delay in large.bat file

Report •

#1
March 2, 2018 at 05:00:25
I just made a stripped down version, all it does is load the variables into memory, check what map to load, decompress, then load the map to the 1031 variables & display the screen.

took just 5 seconds, compared to 20 in main, it must have something to do with "goto"
or reading special characters, while using "find" in a "for loop"

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450


Report •

#2
March 2, 2018 at 05:12:37
ok found the problem myself,

turns out for every 2500 lines (they do not need to contain anything execution is delayed by 5 seconds), im blaming "goto" for this.

de-compressing the map-data will get rid of this problem but increase my file size by 43KB, i guess there is no other way to fix this

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450


Report •

#3
March 2, 2018 at 18:42:56
✔ Best Answer
I guess you've tried and rejected using a self-extracting file? Also, considering current drive capacity and data-transmission speeds, 45K is, like, diddly. I wouldn't sweat it, especially for that small gain in size. Speed is always more important these days. If you want native batch, I think that there's a way to zip/unzip files without 3rd party app, but I may be delusional about that...
Yes, using vbscript, zipping and extracting can be done. Also using powershell, but I still consider that "3rd party" at least for XP.
Here are the very crude prototypes, but I'm sure anyone can figure out where to "fill in the blanks". (I didn't create these, these were culled from the forum or elsewhere on internet)

vbscript to zip a file:

'======================= testing
'zip-a-file prototype. Needs "prettied" up, but it works ok. NOTE that the
'target-file (the zip file) MUST have a .zip extension. ALL paths MUST be fully
'qualified.
set fso=createobject("scripting.filesystemobject")
set shellapp=createobject("shell.application")
set ws=createobject("wscript.shell")
'make a PK header
fso.opentextfile("c:\vbs\zzz.zip",2,true).write("PK"&chr(5)&chr(6)&string(18,chr(0)))

on error resume next
'target directory of the file: c:\vbs
set f=shellApp.NameSpace( "c:\vbs\" ).Items

'target filename to zip, this file
c=f.item("zzz.vbs")
wscript.echo "c:"&c
a=wscript.stdin.readline

' no clue what this does, or did! doesn't seem to matter much
d=f.item("novile")
wscript.echo err.number
a=wscript.stdin.readline
if d="" then wscript.echo "bad"
wscript.echo "c:"&c&" d:"&d

shellApp.NameSpace( "c:\vbs\zzz.zip" ).CopyHere f.item("zzz.vbs")
'---- done with main
'just to view the file:
ws.run "edit /70 c:\vbs\zzz.zip"

wscript.quit

'=============
vbscript to extract to subdir 'zzzz':

set fso=createobject("scripting.filesystemobject")
set objshell=createobject("shell.application")

zipfile="c:\vbs\zzz.zip"
extractto="c:\vbs\zzzz"
If NOT fso.FolderExists(ExtractTo) Then fso.CreateFolder(ExtractTo)
set objShell = CreateObject("Shell.Application")
set FilesInZip=objShell.NameSpace(ZipFile).items
objShell.NameSpace(ExtractTo).CopyHere(FilesInZip)
wscript.quit

message edited by nbrane


Report •

Related Solutions

#4
March 3, 2018 at 11:22:57
Thanks for the help

Self extracting file did not work for me, nor do i want to put time into figuring out how i can make it work.
also tried this online converter:http://www.f2ko.de/en/ob2e.php
which does work, but i can see it is making a .bat file in %temp%/1/E451.bat
containing basically the entire .bat file i started with with 1 line added at the start.....

http://www.bat2exe.net/ does work flawlessly, but does the same as above

EDIT: execution time increased by a lot tho, from 18,5 seconds to 35 seconds

Forgot to mention main.bat is reading data from itself at line 7000+
then converting the string, then putting the generated 1023+8 values in variables.
And the reason for compressing the file is because pastbin.com has an uploadlimit of 500KB (UTF8), so when i upload a 400KB ansi file its converted to utf8, making me almost reach the 500KB limit, stopping me from finishing making the game in 1 file.

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450

message edited by hidde663


Report •

#5
March 3, 2018 at 11:46:12
I've decided to split the workload up between %number_of_processors%, by just lettings in this case 4 cmd.exe's decompress the string & then let main.bat combine them again.
this should make it 2~4 times faster on a dual core with HT, making it go from 18,5 seconds to about 4~6 seconds.(imagine on a TR 1950X's 32 logical cores)

Which is quite nice :).

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450


Report •

#6
March 3, 2018 at 17:51:29
I hereby bequeath the "best answer" to yourself! good work.

Report •

#7
March 3, 2018 at 20:22:40
Took me quite long time, but i split the workload between 4 cores, dropping loadtime from 18,5 single core 4.8 Ghz to about 5 seconds as guessed on a dual core with HT 2.7Ghz

thanks anyways nbrane for putting the time into trying to help me.

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1340c/1965m BiosMod
VS450


Report •

#8
March 6, 2018 at 01:52:40
this is the result in performance increase(13.72 times faster) on an i5-6500T

OLD single thread(24.29 seconds)
start 23.72
end 48.01

NEW multi thread(3.69 seconds)
start 48.78
end 52.47

Optimized Multi(1,77 seconds)
start:18.65
end:20.42

i5-6600K[delid]@4.814GHz/4.613GHz cache@1.385v | 2x4GB Crucial-DDR4-2133@14-14-14-28 1T 2808MHz@1.37v
ASUS Z170K
Samsung 250GB SSD 850 EVO
MSI Armor RX 570 4GB@1380c/2050m BiosMod
VS450

message edited by hidde663


Report •

Ask Question