npocmaka
While the other options are more or less obvious skip and eol were not for me.skip is with higher prio and is executed first:
"2" is not skipped , but only -1 (which is also omitted because of the eol=- ):
Code: Select all
C:\>for /f "skip=1 eol=-" %n in ('echo -1 ^& echo 2 ^& echo 3') do @echo %n
2
3----------------------------
#2 04 Dec 2013 01:19
Aacini
It seems rather logical to me: first skip the number of lines, then check the eol in the rest. This way, the skip option indicate to skip a number of lines from the beginning of the file, not after the first line that pass the eol test.
----------------------------
#3 04 Dec 2013 10:52
npocmaka
It's seems logical because 'eol' check looks more complicated (and slow) and it's better to be left after 'skip' in sake of the performance .
But as it is not explicitly mentioned in 'FOR /?' it could be misleading.
----------------------------
#4 04 Dec 2013 12:54
foxidrive
If EOL was in force at the beginning then skip would be unpredictable in the actual number of lines that were skipped.
It makes sense for skip to function on an exact number of lines.
----------------------------
#5 04 Dec 2013 13:34
dbenham
The thing that surprised me a bit is that SKIP includes empty lines, but after the SKIP is complete, FOR /F ignores (does not iterate) empty lines.
So if the first line of FILE.TXT is empty, then the following commands will give identical results:
Code: Select all
for /f "delims=" %A in (file.txt) do @echo %A
for /f "skip=1 delims=" %A in (file.txt) do @echo %ADave Benham
----------------------------
#6 04 Dec 2013 13:47
foxidrive
I agree, that was a stupid design decision.dbenham wrote:
(I guess it really comes down to the fact that I hate that empty lines are ignored)
You could have just tested for "" in the code and you'd know it was an empty line.
Having a default EOL character (which is really a Start Of Line character) was equally stupid IMO.
Last edited by foxidrive (04 Dec 2013 13:47)
----------------------------
#7 21 Jan 2014 10:57
npocmaka
another thing that came to me to test."tokens=*" is faster thant "delims=" :
Code: Select all
@echo off
echo "delims="
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
for /l %%a in (;;;1=1=500000;;) do (
for /f "delims=" %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)
echo(
echo(
echo "tokens=*"
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
@echo off
for /l %%a in (;;;1=1=500000;;) do (
for /f "tokens=*" %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)
echo(
echo(
echo delims^^^=
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
for /l %%a in (;;;1=1=500000;;) do (
for /f delims^= %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)
echo(
echo(
echo delims^^^=^ tokens^^^=*
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
for /l %%a in (;;;1=1=500000;;) do (
for /f delims^=^ tokens^=* %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)
echo(
echo(
echo tokens^^^=*
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
for /l %%a in (;;;1=1=500000;;) do (
for /f tokens^=* %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)
echo(
echo(
echo "tokens=* delims="
for /f %%t in ('powershell "(get-date).tofiletime()"') do set mst=%%t
@echo off
for /l %%a in (;;;1=1=500000;;) do (
for /f "tokens=* delims=" %%z in ("text with delimiters. word word word word word word word word word word word word word word word word word word word word word word word word") do break
)
powershell ((get-date).tofiletime() - %mst%)EDIT:
At least on my machine tokens^=* gives the fastest results.It was consistent after a few plays.While the "tokens=* delims=" is the slowest.
Last edited by npocmaka (21 Jan 2014 13:32)
----------------------------
#8 21 Jan 2014 14:21
foxidrive
tokens=* removes all leading whitespace, while delims= retains any leading spaces in filenames.
If your filenames could have leading spaces then using tokens=* will mean they will not be processed properly when gathering a list of filenames.
Last edited by foxidrive (21 Jan 2014 14:23)
----------------------------
#9 21 Jan 2014 23:37
npocmaka
Yep.And now it's sounds reasonable after I saw it writtenfoxidrive wrote:
tokens=* removes all leading whitespace, while delims= retains any leading spaces in filenames.
If your filenames could have leading spaces then using tokens=* will mean they will not be processed properly when gathering a list of filenames.
.Looks like tokens is doing more work in this case , but anyway it's faster.----------------------------
#10 22 Jan 2014 07:03
foxidrive
This is the results I got from your code. Is the figure returned 7.93 seconds the first one?
Code: Select all
"delims="
79369416
"tokens=*"
78598961
delims^=
80249059
delims^= tokens^=*
80929589
tokens^=*
78299241
"tokens=* delims="
79875805#11 22 Jan 2014 09:17
npocmaka
Yes.Just used filetime format for easier calculations.foxidrive wrote:
This is the results I got from your code. Is the figure returned 7.93 seconds the first one?
#12 06 Feb 2014 14:38
MC ND
Maybe this is known and this post is not needed, and maybe it is a little off topic with the "priority" title, but i've seen (sorry dbenham, you were the first)
And, as i see it, for does not skip empty lines. for skips lines in where none of the replaceable parameters (declared or generated by the token list) get data assigned.dbenham wrote:
I hate that empty lines are ignored
With this file
Code: Select all
a,b,c
d,e,f
g,h,
i,j,kCode: Select all
for /f "delims=, tokens=3" %f in (file.txt) do echo %f----------------------------
#13 06 Feb 2014 23:12
MC ND
Just to try to make it clear. Running
Code: Select all
for /f "tokens=2,3" %a in ('@echo a,b,') do ( echo %a & echo %b)But running
for /f "tokens=3" %f in ('@echo a,b,') do ( echo %a & echo %b)
we get no output. No echo status line. Line is ignored/skipped as no variable gets data.
So, for command is ignoring not the emtpy lines but the lines that get no data asigned into the replaceable parameters.
----------------------------
#14 07 Feb 2014 03:53
foxidrive
Another way to look at this is that lines with data can be echoed and lines without data cannot be echoed or included in the procedure.
As there is no way that empty lines can be included in the procedure then it is fair to say that it is ignoring them.
----------------------------
#15 07 Feb 2014 09:09
MC ND
Of course, FOR command is ignoring them, but the question is why they are ignored, because lines with data are also ignored when this data is not assigned into any of the FOR replaceable parameters.foxidrive wrote:
Another way to look at this is that lines with data can be echoed and lines without data cannot be echoed or included in the procedure.
As there is no way that empty lines can be included in the procedure then it is fair to say that it is ignoring them.
And this "why" is the reason for the npocmaka first post.
When FOR command start to process data, SKIP is evaluated, and lines are skipped with or without data, with or without EOL. Just skip n lines.
Then, lines start to be parsed. And here TOKENS, DELIMS and EOL determine if the line is ignored or not. If after parse, any of the replaceable parameters get data, the code into the FOR is executed. If no data is assigned, line is ignored.
With only "tokens=*" leading spaces are removed because they are a delimiter, and if they are at the beginning of the line, no data has still be retrieved to be asigned, so, FOR continues trying to find the first token in the line to see if the line should be processed or not. Once data is found, no further processing is done in the line (no need to search for end of token), as we are asking to retrieve all the remaining data.
"delims=" does not remove the leading spaces obviously because we are excluding them as delimiters. But the rest of the line is parsed to determine start and end of the tokens that will be mapped into replaceable parameters.
----------------------------
#16 07 Feb 2014 10:48
foxidrive
The coder managed to do it that way.MC ND wrote:
the question is why they are ignored
Like you, there are several blokes who delight in dissecting cmd and creating wonderful workarounds to perform batch-only solutions, just for the pleasure of coding.
I had that bug too for many many years, in times when there were no other ways to solve the problem,
but these days there is sed and repl and findrepl and WSH with Jscript and VBS, and Wmic,
and command line compilers in windows, so my enjoyment now is solving a question in an easy/simple/robust way.
You might like to create an account at dostips.com too, if you haven't already, where a lot of discussion of techniques goes on.
A recent thread devised a pure batch method to create binary files with all 256 characters.
----------------------------
#17 17 Sep 2014 16:31
npocmaka
One more thing that caught my attention. If there are few option of the same type only the last one is taken into consideration:
Code: Select all
>for /f "eol=1 eol=2" %a in ('echo 1^&echo 2') do @echo %a
1