NYTReader123
Hi everyone:
I am a new member of the forum, and am hoping some of the members here could help me out with what is likely a very basic question.
In the code below I am extracting some rows from a file (tmp.csv) and then saving the results to a new file (MyFile.csv).
Code: Select all
@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*" %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
set /a row=row + 1
if !row! LEQ !showRows! echo %%r>>"=MyFile.csv"
)
" was unexpected at this time
which I am pretty sure means it is not using the value for "firstrow". This is puzzling since in the earlier check,
echo Info: row before data=!firstrow! ...
it outputs correct numbers.
I am pretty sure this has to do with the SETLOCAL ENABLEDELAYEDEXPANSION but I have not been able to figure out what I am doing wrong:
- I tried "!firstrow!" in the FOR loop and I get a message that this is unexpected at this time
- I tried creating a new variable (set var temp=!firstrow!) and same error as before
If someone can point out what is probably a very stupid mistake on my end I would be greatly in their deb
----------------------------
#2 16 Feb 2009 10:35
bluesxman
It's hard to give a definitive answer without seeing samples of your input data.
But if %firstrow% is zero, you'll get that error the same as if it were blank -- I.E. you can't tell FOR to skip zero lines.
You could, however, try something like this:
Code: Select all
if %firstrow% GTR 0 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
for /F "tokens=* %_skip%" %%r in (tmp.csv) do (
cmd | *sh | ruby | chef
----------------------------
#3 16 Feb 2009 17:16
avery_larry
I tried your code as is and I cannot get it to fail UNLESS %firstrow% is undefined.
Are you certain that this is an exact copy of all your code? Are you certain that "City,Visits" exists in the tmp.csv file exactly (including case)? Are you certain that the "echo Info:" line properly defines firstrow? What happens when you try this code:
Code: Select all
rem @ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*" %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=%firstrow%, num rows=%numrows%, num rows with data=%showRows%, headers=%headers%
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
set /a row+=1
if !row! LEQ %showRows% echo %%r>>"=MyFile.csv"
)
----------------------------
#4 16 Feb 2009 17:21
avery_larry
Assuming that the OP has posted all of the code, then firstrow must be greater than 0 as it is defined by the line number as outputed by the find /n command -- which means it will be greater than zero OR undefined.bluesxman wrote:
It's hard to give a definitive answer without seeing samples of your input data.
But if %firstrow% is zero, you'll get that error the same as if it were blank -- I.E. you can't tell FOR to skip zero lines.
You could, however, try something like this:
Code: Select all
if %firstrow% GTR 0 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
Code: Select all
if 1%firstrow% GTR 10 (set "_skip=skip=%firstrow%") ELSE (set "_skip=")
----------------------------
#5 16 Feb 2009 19:09
NYTReader123
Hi everyone:
Thanks for all of the responses-- I really appreciate it.
A couple of quick points:
- %firstrow% is always greater than zero (typically 10) and is never undefined. I am sure this is fine since when the echo row listed in the second block of my code always outputs a number
- there is some omitted code before what is listed here. I will try and run this after omitting the initial code to see if this makes a difference
I will try out each of the suggestions listed here and report back, hopefully later today. If I still have problems I will post the complete code and the tmp.csv I am using.
Again my thanks for all of your help.
----------------------------
#6 17 Feb 2009 04:56
NYTReader123
Hi everyone again:
Thanks again to everyone for the suggestions. After trying each of them without any luck, I realized you all were right and I was making a mistake elsewhere in the code.
The problem actually seems to have something to do with the few lines I omitted from my original post which involve a FOR loop. This seems to be my problem, but I am not sure what is going wrong.
When I use the code:
Code: Select all
@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
for /l %%d in (1,1,30) do (
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*" %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
set /a row=row + 1
if !row! LEQ !showRows! echo %%r>>"MyFile.csv"
)
)
" was unexpected at this time.
The system cannot find the path specified.
Interestingly the initial for loop is working to the extent that it repeats this 30 times as expected.
Now when I omit the initial FOR loop along with the last parenthesis and just run,
Code: Select all
@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*" %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=!firstrow!, num rows=!numrows!, num rows with data=!showRows!, headers=!headers!
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
set /a row=row + 1
if !row! LEQ !showRows! echo %%r>>"MyFile.csv"
)
For reference here is the tmp.csv:
# ----------------------------------------,,,,,,
MY FILE,,,,,,
Detail:,,,,,,
19-Jun-05,19-Jun-05,,,,,
# ----------------------------------------,,,,,,
# ----------------------------------------,,,,,,
# Table,,,,,,
# ----------------------------------------,,,,,,
City,Visits,Pages/Visit,Avg. Time on Site,% New Visits,Bounce Rate,Visits
Newark,25,34,1212,1,0.8888,2222222
New Castle,69,56,45646,0,1,5555555
Hockessin,36723,78,1,0,1,278678
Lewes,89,90,5,1,1,12
Middletown,1,1112111,0,0,0,1
Rehoboth Beach,0,0,6,0,0,0
# --------------------------------------------------------------------------------
Can someone spot the error? Thanks again for any advice which you can offer.
----------------------------
#7 17 Feb 2009 12:38
bluesxman
Try this:
Code: Select all
for /F "tokens=* skip=!firstrow!" %%r in (tmp.csv) do (
----------------------------
#8 17 Feb 2009 19:10
avery_larry
Doesn't work. Can't use the ! expansion inside the for /f "". Don't know why -- it just returns an error.bluesxman wrote:
Try this:
for /F "tokens=* skip=!firstrow!" %%r in (tmp.csv) do (
----------------------------
#9 17 Feb 2009 19:13
avery_larry
You're going to have to pull that for loop out of the original for loop like this: (untested)
Code: Select all
@ECHO off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
for /l %%d in (1,1,30) do call process
:process
For /F "tokens=1,2,* delims=[]" %%A in ('type tmp.csv^|Find /N "City,Visits"') Do set /a firstrow=%%A
For /F "tokens=*" %%B in ('type tmp.csv^|Find "City,Visits"') Do set headers=%%B
set /a numrows=0
FOR /f %%n in ('type tmp.csv^|find "" /v /c') do set /a numrows=%%n
set /a showRows=numrows-firstrow-1
REM check code
echo Info: row before data=%firstrow%, num rows=%numrows%, num rows with data=%showRows%, headers=%headers%
set /a row=0
for /F "tokens=* skip=%firstrow%" %%r in (tmp.csv) do (
set /a row+=1
if !row! LEQ %showRows% echo %%r>>"MyFile.csv"
)
goto :eof
#10 17 Feb 2009 19:17
bluesxman
I don't think that's necessary, avery_larry.
Delayed expansion is already on, so all that needs to change is the % to ! (to take advantage of Del. Exp.) where the "skip" value is being set to FIRSTROW
Last edited by bluesxman (17 Feb 2009 19:18)
cmd | *sh | ruby | chef
----------------------------
#11 17 Feb 2009 19:39
avery_larry
Try it. For some reason for me (in my tests on WinXP Pro) the ! will not expand inside the for /f "!anyplace inside these quotes" even with delayedexpansion. Otherwise I'd agree with you. (You'll notice also that the OP said he already tried that.)bluesxman wrote:
I don't think that's necessary, avery_larry.
Delayed expansion is already on, so all that needs to change is the % to ! (to take advantage of Del. Exp.) where the "skip" value is being set to FIRSTROW
I'm assuming that there's some special case with the for /f "" where you can't use delayedexpansion, and thus (since the entire original code posted is actually inside another for loop) my suggestion to pull it out of the for loop completely so you don't have to delayedexpansion that particular for /f "" command (noting that delayedexpansion is used inside that loop).
----------------------------
#12 18 Feb 2009 05:40
NYTReader123
Thanks bluesxman and avery_larry!
I followed avery_larry's suggestion and now it works (small point: "do call :process" rather than "do call process")!
I doubt I would have ever figured this out, though I have no clue as to why moving the FOR loop to a sub-routine is needed.
Again my sincere thanks. I was ripping my hair out over this.
PS Just for completeness, I have also been using Win XP Pro.
----------------------------
#13 18 Feb 2009 13:14
bluesxman
@ avery_larry ... oops, I obviously didn't read the OP closely enough (and haven't tried my suggestion) ... my bad!
cmd | *sh | ruby | chef
----------------------------
#14 18 Feb 2009 18:32
avery_larry
The point of delayedexpansion is to delay the expansion of variables as long as possible. When evaluating anything inside parentheses, ALL "regular" variables inside the parentheses are evaluated immediately.NYTReader123 wrote:
though I have no clue as to why moving the FOR loop to a sub-routine is needed.
Since the
Code: Select all
for /F "tokens=* skip=%firstrow%"
Code: Select all
echo Info: row before data=!firstrow!, num rows=!numrows!, . . .
Code: Select all
echo Info: row before data=%firstrow%, num rows=%numrows%, . . .
Now in theory, the following should be the solution, as you and blue and I all tried -- well, blue only suggested
Code: Select all
for /F "tokens=* skip=!firstrow!"
Hope that wasn't just more confusing!
Last edited by avery_larry (18 Feb 2009 18:32)
----------------------------
#15 18 Feb 2009 22:32
NYTReader123
avery_larry: thanks that helps me understand your solution (well all except that strange case you mention; sigh...I wish this stuff was better documented).
Anyway thanks again to both you and bluesxman. I owe you guys a beer!
Last edited by NYTReader123 (19 Feb 2009 00:51)