setlocal enabledelayedexpansion

Microsoft Windows
Post Reply
User avatar
MigrationUser
Posts: 336
Joined: 2021-Jul-12, 1:37 pm
Contact:

setlocal enabledelayedexpansion

Post by MigrationUser »

15 Feb 2009 09:57
NYTReader123

Hi everyone:

I am a new member of the forum, and am hoping some of the members here could help me out with what is likely a very basic question.

In the code below I am extracting some rows from a file (tmp.csv) and then saving the results to a new file (MyFile.csv).

Code: Select all

@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION

For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*"  %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
            REM check code
            echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!

set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
   set /a row=row + 1
   if !row! LEQ !showRows! echo %%r>>"=MyFile.csv"
)
My problem is in the last FOR loop. I get an error,

" was unexpected at this time

which I am pretty sure means it is not using the value for "firstrow". This is puzzling since in the earlier check,

echo Info: row before data=!firstrow! ...

it outputs correct numbers.

I am pretty sure this has to do with the SETLOCAL ENABLEDELAYEDEXPANSION but I have not been able to figure out what I am doing wrong:
- I tried "!firstrow!" in the FOR loop and I get a message that this is unexpected at this time
- I tried creating a new variable (set var temp=!firstrow!) and same error as before

If someone can point out what is probably a very stupid mistake on my end I would be greatly in their deb

----------------------------

#2 16 Feb 2009 10:35
bluesxman


It's hard to give a definitive answer without seeing samples of your input data.

But if %firstrow% is zero, you'll get that error the same as if it were blank -- I.E. you can't tell FOR to skip zero lines.

You could, however, try something like this:

Code: Select all

if %firstrow% GTR 0 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
for /F "tokens=* %_skip%" %%r in (tmp.csv) do (
Hope that helps.

cmd | *sh | ruby | chef

----------------------------

#3 16 Feb 2009 17:16
avery_larry


I tried your code as is and I cannot get it to fail UNLESS %firstrow% is undefined.

Are you certain that this is an exact copy of all your code? Are you certain that "City,Visits" exists in the tmp.csv file exactly (including case)? Are you certain that the "echo Info:" line properly defines firstrow? What happens when you try this code:

Code: Select all

rem @ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION

For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*"  %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=%firstrow%, num rows=%numrows%, num rows with data=%showRows%, headers=%headers%

set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
   set /a row+=1
   if !row! LEQ %showRows% echo %%r>>"=MyFile.csv"
)
Last edited by avery_larry (16 Feb 2009 17:24)

----------------------------

#4 16 Feb 2009 17:21
avery_larry
bluesxman wrote:

It's hard to give a definitive answer without seeing samples of your input data.

But if %firstrow% is zero, you'll get that error the same as if it were blank -- I.E. you can't tell FOR to skip zero lines.
Assuming that the OP has posted all of the code, then firstrow must be greater than 0 as it is defined by the line number as outputed by the find /n command -- which means it will be greater than zero OR undefined.

You could, however, try something like this:

Code: Select all

    if %firstrow% GTR 0 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
Which will fail if firstrow is undefined. I always do this:

Code: Select all

if 1%firstrow% GTR 10 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
or add an "if defined" statement.

----------------------------

#5 16 Feb 2009 19:09
NYTReader123


Hi everyone:

Thanks for all of the responses-- I really appreciate it.

A couple of quick points:

- %firstrow% is always greater than zero (typically 10) and is never undefined. I am sure this is fine since when the echo row listed in the second block of my code always outputs a number

- there is some omitted code before what is listed here. I will try and run this after omitting the initial code to see if this makes a difference

I will try out each of the suggestions listed here and report back, hopefully later today. If I still have problems I will post the complete code and the tmp.csv I am using.

Again my thanks for all of your help.

----------------------------

#6 17 Feb 2009 04:56
NYTReader123


Hi everyone again:

Thanks again to everyone for the suggestions. After trying each of them without any luck, I realized you all were right and I was making a mistake elsewhere in the code.

The problem actually seems to have something to do with the few lines I omitted from my original post which involve a FOR loop. This seems to be my problem, but I am not sure what is going wrong.

When I use the code:

Code: Select all

@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION

for /l %%d in (1,1,30) do (
    For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
    For /F "tokens=*"  %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
    set /a numrows=0
    FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
    set /a showRows=numrows-firstrow-1
        REM check code
        echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!
    set /a row=0
    for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
        set /a row=row + 1
        if !row! LEQ !showRows! echo %%r>>"MyFile.csv"
    )
)
I get the error message,

" was unexpected at this time.
The system cannot find the path specified.

Interestingly the initial for loop is working to the extent that it repeats this 30 times as expected.

Now when I omit the initial FOR loop along with the last parenthesis and just run,

Code: Select all

@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
    For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
    For /F "tokens=*"  %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
    set /a numrows=0
    FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
    set /a showRows=numrows-firstrow-1
        REM check code
        echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!
    set /a row=0
    for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
        set /a row=row + 1
        if !row! LEQ !showRows! echo %%r>>"MyFile.csv"
    )
everything works.

For reference here is the tmp.csv:

# ----------------------------------------,,,,,,
MY FILE,,,,,,
Detail:,,,,,,
19-Jun-05,19-Jun-05,,,,,
# ----------------------------------------,,,,,,

# ----------------------------------------,,,,,,
# Table,,,,,,
# ----------------------------------------,,,,,,
City,Visits,Pages/Visit,Avg. Time on Site,% New Visits,Bounce Rate,Visits
Newark,25,34,1212,1,0.8888,2222222
New Castle,69,56,45646,0,1,5555555
Hockessin,36723,78,1,0,1,278678
Lewes,89,90,5,1,1,12
Middletown,1,1112111,0,0,0,1
Rehoboth Beach,0,0,6,0,0,0
# --------------------------------------------------------------------------------

Can someone spot the error? Thanks again for any advice which you can offer.

----------------------------

#7 17 Feb 2009 12:38
bluesxman


Try this:

Code: Select all

    for /F "tokens=* skip=!firstrow!" %%r in (tmp.csv) do (
cmd | *sh | ruby | chef

----------------------------

#8 17 Feb 2009 19:10
avery_larry
bluesxman wrote:

Try this:

for /F "tokens=* skip=!firstrow!" %%r in (tmp.csv) do (
Doesn't work. Can't use the ! expansion inside the for /f "". Don't know why -- it just returns an error.

----------------------------

#9 17 Feb 2009 19:13
avery_larry


You're going to have to pull that for loop out of the original for loop like this: (untested)

Code: Select all

@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION

for /l %%d in (1,1,30) do call process

:process
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*"  %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
    REM check code
    echo Info: row before data=%firstrow%, num rows=%numrows%, num rows with data=%showRows%, headers=%headers%
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
    set /a row+=1
    if !row! LEQ %showRows% echo %%r>>"MyFile.csv"
)
goto :eof
----------------------------

#10 17 Feb 2009 19:17
bluesxman


I don't think that's necessary, avery_larry.

Delayed expansion is already on, so all that needs to change is the % to ! (to take advantage of Del. Exp.) where the "skip" value is being set to FIRSTROW

Last edited by bluesxman (17 Feb 2009 19:18)

cmd | *sh | ruby | chef

----------------------------

#11 17 Feb 2009 19:39
avery_larry
bluesxman wrote:

I don't think that's necessary, avery_larry.

Delayed expansion is already on, so all that needs to change is the % to ! (to take advantage of Del. Exp.) where the "skip" value is being set to FIRSTROW
Try it. For some reason for me (in my tests on WinXP Pro) the ! will not expand inside the for /f "!anyplace inside these quotes" even with delayedexpansion. Otherwise I'd agree with you. (You'll notice also that the OP said he already tried that.)

I'm assuming that there's some special case with the for /f "" where you can't use delayedexpansion, and thus (since the entire original code posted is actually inside another for loop) my suggestion to pull it out of the for loop completely so you don't have to delayedexpansion that particular for /f "" command (noting that delayedexpansion is used inside that loop).

----------------------------

#12 18 Feb 2009 05:40
NYTReader123


Thanks bluesxman and avery_larry!

I followed avery_larry's suggestion and now it works (small point: "do call :process" rather than "do call process")!

I doubt I would have ever figured this out, though I have no clue as to why moving the FOR loop to a sub-routine is needed.

Again my sincere thanks. I was ripping my hair out over this.

PS Just for completeness, I have also been using Win XP Pro.

----------------------------

#13 18 Feb 2009 13:14
bluesxman


@ avery_larry ... oops, I obviously didn't read the OP closely enough (and haven't tried my suggestion) ... my bad!

cmd | *sh | ruby | chef

----------------------------

#14 18 Feb 2009 18:32
avery_larry
NYTReader123 wrote:

though I have no clue as to why moving the FOR loop to a sub-routine is needed.
The point of delayedexpansion is to delay the expansion of variables as long as possible. When evaluating anything inside parentheses, ALL "regular" variables inside the parentheses are evaluated immediately.

Since the

Code: Select all

for /F "tokens=* skip=%firstrow%"
is actually nested inside another for loop, the %firstrow% variable is evaluated at the time the original for loop is called. Again -- you properly used delayedexpansion to allow for variables to be evaluated when they are actually called inside the for loop. That's why this works:

Code: Select all

echo Info: row before data=!firstrow!, num rows=!numrows!, . . .
while this wouldn't work:

Code: Select all

echo Info: row before data=%firstrow%, num rows=%numrows%, . . .
because firstrow and numrows are not defined (or if they are defined, they aren't "current") when the original for loop is called (because everything inside the parentheses of the first for loop is evaluated immediately).

Now in theory, the following should be the solution, as you and blue and I all tried -- well, blue only suggested :P

Code: Select all

for /F "tokens=* skip=!firstrow!"
as that's exactly the purpose of delayedexpansion. However, for reasons we don't know (ask Microsoft, right?) it just doesn't work. Can't use the ! character inside the quoted portion of a for /f loop. So in order to make the code work, we had to pull that part of the code out of the first for loop so that your code wouldn't attempt to evaluate that variable until it was already defined. We pull the code outside of the for loop (or more specifically, outside of the parentheses) by calling a label. As it processes the label, each variable is evaluated as each command comes along, instead of all of them getting evaluated inside the first for loop.

Hope that wasn't just more confusing!

Last edited by avery_larry (18 Feb 2009 18:32)

----------------------------

#15 18 Feb 2009 22:32
NYTReader123


avery_larry: thanks that helps me understand your solution (well all except that strange case you mention; sigh...I wish this stuff was better documented).

Anyway thanks again to both you and bluesxman. I owe you guys a beer!

Last edited by NYTReader123 (19 Feb 2009 00:51)
Post Reply