#1 24 Apr 2010 00:46

RG
Member
From: Minnesota
Registered: 18 Feb 2010
Posts: 321

SETLOCAL ENABLEDELAYEDEXPANSION

Whoever is interested,

In another post on this forum there were some questions about enabling delayed expansion.  I decided to start another post and expound on it to give the topic the attention it deserves.  There are probably many ways to explain delayed expansion.  Here is my explanation.  I encourage you to play with this a little and then we will all have benefited from this exercise.

For those of you that are 'programmer type' people, you could compare this to compile-time and run-time behavior.
In this environment it is referred to as load-time and run-time behavior.
Variables are normally expanded when loaded.
Enclosing a variable in ! instead of % allows us to 'delay expansion' of the variable until run-time.
Keep in mind that shell scripts (bat files) are loaded , then executed one line at a time.
We need to have a clear understanding of what a 'line' is.  Typically it is 1 line.
However in the case of FOR loops and IF statements the entire FOR and IF constructs are treated as a 'line'... even if they span multiple lines.
So in my example below, everything from the 'for' though the ')' is loaded, variables expanded and then executed. Then we move on to the next line.
So when you use %InnerVar% you are getting the 'load-time' value... when you use !InnerVar! you are getting the 'run-time' value.
Therefore it is correct to use %OuterVar% within the loop because it was set outside the loop... in a previous line, so it was already set at the time that we loaded the FOR construct.
In the same manner, it is correct to use %InnerVar% once we are outside the FOR construct because that line is loaded after the FOR construct has executed.
Take a look at the simple example below and the results generated by the echo statements.
Note that the use of %InnerVar% within the FOR loop gives us the load-time value (Wrong InnerVar).
The use of %InnerVar% outside the FOR loop gives us the expected value.  It is still the load-time value, but the value was already set at the time the line was loaded.

I used a FOR construct for this example.  The same applies to an IF construct.

@echo off
setlocal enabledelayedexpansion
set /a OuterVar=1
set /a InnerVar=1
REM Notice that variables set inside the loop must be enclosed in !
REM Variables set outside the loop are enclosed in %
REM The replaceable parameter %%i is an exception to this
for /L %%i IN (1,1,5) do (
   set /a InnerVar+=%OuterVar%
   echo.OuterVar=%OuterVar% InnerVar=!InnerVar! Wrong InnerVar=%InnerVar%
)
echo.OuterVar= %OuterVar% InnerVar=%InnerVar%
endlocal
pause

Result:
OuterVar=1 InnerVar=2 Wrong InnerVar=1
OuterVar=1 InnerVar=3 Wrong InnerVar=1
OuterVar=1 InnerVar=4 Wrong InnerVar=1
OuterVar=1 InnerVar=5 Wrong InnerVar=1
OuterVar=1 InnerVar=6 Wrong InnerVar=1
OuterVar= 1 InnerVar=6

Forgive my long-windedness, but I use this extensively and I think this is a very important concept.
Take this for a spin and ask questions if you have them... there are plenty of folks on here that will answer your questions.


CMD and InstallShield

Offline

#2 24 Apr 2010 18:42

retro-starr
Member
Registered: 03 Oct 2009
Posts: 87

Re: SETLOCAL ENABLEDELAYEDEXPANSION

Would it hurt anything to have all variables with '!'? http://ss64.com/nt/setlocal.html makes me believe that you must have all variables with '!'. I would imagation that you would have a slightly slower script if all variables were expanded right at run-time.

Offline

#3 24 Apr 2010 20:42

RG
Member
From: Minnesota
Registered: 18 Feb 2010
Posts: 321

Re: SETLOCAL ENABLEDELAYEDEXPANSION

retro-starr wrote:

Would it hurt anything to have all variables with '!'? http://ss64.com/nt/setlocal.html makes me believe that you must have all variables with '!'. I would imagation that you would have a slightly slower script if all variables were expanded right at run-time.

I think you are saying if you have DELAYEDEXPANSIONENABLED, why not always use !.
I don't know if it would hurt anything but I don't think it is good practice to do so.
For one thing, if you moved the SET or ENDLOCAL so that variables enclosed in ! are now outside the area where delayed expansion is enabled the following statement:
   echo.Var=!Var!
Woud display
   Var=!Var!
Not what you wanted.


CMD and InstallShield

Offline

#4 25 Apr 2010 02:08

retro-starr
Member
Registered: 03 Oct 2009
Posts: 87

Re: SETLOCAL ENABLEDELAYEDEXPANSION

If this is what your example was showing then of course outside the localization wouldn't do anything.

setlocal enabledelayedexpansion
set "var=[whatever]"
echo.!var!
endlocal

:: notice it's out of the localization
echo.var=!var!

Output:

[whatever]
!var!

Where might a good place to be setting delayed variables? For me it was just guess and check that it worked!

Offline

#5 25 Apr 2010 03:05

RG
Member
From: Minnesota
Registered: 18 Feb 2010
Posts: 321

Re: SETLOCAL ENABLEDELAYEDEXPANSION

Generally you put the SETLOCAL near the beginning of the bat file or subroutine(s) if you localize them.
I should have mentioned that there are times when you don't want to enable delayed expansion, such as when there are ! present in the data because they will be consumed.
Go ahead and add delayed expansion to the example below and see what happens (exclamation point will not be displayed).

You indirectly brought up a subject that I intentionally did not get into in the post above.  There are times when you need to 'return' a value(s) from a routine whose variables are localized.  This is easily done with a technique called 'tunneling'.  See the endlocal line in my example below.  Keeping in mind that everything on the line is expanded at load time... %var% is expanded at load time.  The 'set var' is done at run time.  so we made a new variable with the same name that exists outside the localization, using the localized value.  I could have used another name instead of var... your choice.  It is indeed another variable though.

@echo off
setlocal
set "var=[whatever!]"
echo.%var%
endlocal & set var=%var%

:: notice it's out of the localization
echo.var=%var%

CMD and InstallShield

Offline

#6 25 Apr 2010 05:31

retro-starr
Member
Registered: 03 Oct 2009
Posts: 87

Re: SETLOCAL ENABLEDELAYEDEXPANSION

I intentionally didn't tunnel as it was how I thought you worded your example, but hey! If the tunneling technique expands the whole concept of 'setlocal' why not put it! My qeustion was more of where to put !var! instead of 'setlocal'. Like in the other thread your helping me on, nobody pointed out that !upx! was going to work or even !cmd_string! (for those who don't know, upx is a tool set as a variable and cmd_string was a using upx's variable to run inside of a subroutine that ran it). Was it just luck that I remembered that I had to do !upx! (which was also luck when I figured it out long time ago)?

Offline

#7 25 Apr 2010 14:28

RG
Member
From: Minnesota
Registered: 18 Feb 2010
Posts: 321

Re: SETLOCAL ENABLEDELAYEDEXPANSION

Use !var! when delayed expansion is enabled and the variable is referenced within an IF or FOR construct.  In those cases you want the expansion of the variable to occur at run-time instead of at load-time.  Outside the IF or FOR construct the same variable is referenced as %var%.


CMD and InstallShield

Offline

#8 25 Apr 2010 19:08

retro-starr
Member
Registered: 03 Oct 2009
Posts: 87

Re: SETLOCAL ENABLEDELAYEDEXPANSION

Since 'setlocal enabledelayedexpansion' provides localization would it be better to just use that instead of 'setlocal'? I think I can remember to use it for 'for' and 'if' functions. Thanks!

Offline

#9 25 Apr 2010 19:42

RG
Member
From: Minnesota
Registered: 18 Feb 2010
Posts: 321

Re: SETLOCAL ENABLEDELAYEDEXPANSION

retro-starr wrote:

Since 'setlocal enabledelayedexpansion' provides localization would it be better to just use that instead of 'setlocal'? I think I can remember to use it for 'for' and 'if' functions. Thanks!

I don't think so.  It is good practice to always use one or the other.  If you don't use either, you run the risk of 'overloading' variables that you did not intend to.
I think I posted an example in one of these posts that shows that you can lose the ! in text if delayed expansion is enabled, so I only use it when needed... but as you sort of implied, that can be most of the time.

I have one very long script that I did years ago.  It gathers data about the machine and runs many scripts to gather tons of information out of a SQL database.  I used SET LOCAL ENABLEDELAYEDEXPANSION almost exclusively through out it.  Then I got into trouble because one of the registry entries I was picking up occasionally had an ! in it.  Got to be messy to go back and sort out where I should have delayed expansion enabled and where not.


CMD and InstallShield

Offline

#10 19 Nov 2010 22:41

jeb
Member
From: Germany
Registered: 19 Nov 2010
Posts: 99

Re: SETLOCAL ENABLEDELAYEDEXPANSION

http://ss64.com/nt/setlocal.html
It describes it not very correct (like this: With delayed expansion the caret ^ escapes each special character all the time, not just for one command)

I suppose the main problem with variables, expansion and special characters is to understand the moments of there "activity".

As long as you don't understand how and when they work, the results are unpredictable,
therefore I make some (thousands) experiments.

The short form is:
A line is a line (Like RG posted), but can spawn multiple lines in parenthesis blocks or with multiline characters (^)

Each line is parsed and expanded by the BatchLineParser in multiple steps
1. Replace all %vars%
2. remove all <CarrigeReturn> (most of the time of no importance)
3. special char parser for carets ^ quotes " redirection <> pipes |, ampersands & and parenthesis, stops on the first <LineFeed>
    REM, IF, FOR recognizer - they have their own special handling way
    Building the primary token list
5. echo the result if echo is on
6. Expanding of %%v in for-loops (Not really step6 - but it makes no difference)
7. DELAYED-EXPANSION (if it is enabled) recognized only ^ and !  (therefor you have to write echo ^^!),
   this can expand also the primary tokens, but without creating or removing tokens from the list!
   the expanded result will not analyzed anymore!
8. call starts the parser again, but with a new step - the ZERO step
   0. doubling all carets ^ -the most of them are removed in step 3
9. execute the cmd - the command decide of  using the primary token list or take something else, like set "var=bla bla"

And a more complete description are here http://www.lingubender.com/forum/viewto … f=12&t=615

In my opinion, delayed expansion is nearly always better than percent expansion.
You never have to take care of the variable content, like quotes, ampersands, carets or spaces.

hope it helps
jeb

Offline

#11 20 Nov 2010 11:23

Simon Sheppard
Super Administrator
Registered: 27 Aug 2005
Posts: 761
Website

Re: SETLOCAL ENABLEDELAYEDEXPANSION

jeb wrote:

http://ss64.com/nt/setlocal.html
It describes it not very correct (like this: With delayed expansion the caret ^ escapes each special character all the time, not just for one command)

Good point, I've reworded it now to (hopefully) make this a bit clearer:

http://ss64.com/nt/setlocal.html#enabledelayedexpansion

Offline

#12 20 Nov 2010 12:57

jeb
Member
From: Germany
Registered: 19 Nov 2010
Posts: 99

Re: SETLOCAL ENABLEDELAYEDEXPANSION

Sorry, now you are completely wrong smile

From your link

Simon wrote:

Setting EnableDelayedExpansion will reverse this behaviour.

No. It only adds a new way of expanding variables, nothing else.


Simon wrote:

Escaping control characters:

@echo off
setlocal
Set _html=Hello^>World
Echo %_html%

In the above, the Echo command will create a text file called 'world' - not quite what we wanted! This is because the '^' caret works once for the SET command, but then vanishes.
If we now try the same thing with EnableDelayedExpansion, the caret works all the way through the script:

SETLOCAL EnableDelayedExpansion
Set _html=^<title^>Hello world ^</title^>
Echo !_html!
<title>Hello world </title>

With delayed expansion the caret ^ escapes each special character all the time, not just for one command.
This makes it possible to work with HTML and XML formatted strings in a variable.

The content in both variants is the same
Set _html=Hello^>World
always set the content to Hello >World
The difference is, that expanding with percents are just before the special characters are handeld.
So
echo %_html% expands to echo Hello > World  --- Step 1 of the BatchLineParser
Detecting the ">"                                       --- Step 3 of the BatchLineParser
So it fails (you could get it working with set _html=Hello ^^^> World)

But with delayed expansion it is:
echo !_html!  nothing happens in step 1 to 6
In step7 of the BatchLineParser it expands to
echo Hello > World  --- And then it is executed, parsing ends after this step!


That's the cause why this work this way

@echo off
setlocal DisableDelayedExpansion
set "var1=One !var2!"
set "var2=Two %%var3%%"
set "var3=Three !var4!"
set "var4=Four"
set var
echo Sample1 %var1%
setlocal EnableDelayedExpansion
echo Sample2 %var1%
call echo Sample3 %var1%
call call call call echo Sample4 "^"

Output:
var1=One !var2!
var2=Two %var3%
var3=Three !var4!
var4=Four
Sample1 One !var2!
Sample2 One Two %var3%
Sample3 One Two Three !var4!
Sample4 "^^^^^^^^^^^^^^^^"

hope it helps
jeb

Offline

#13 20 Nov 2010 14:58

Simon Sheppard
Super Administrator
Registered: 27 Aug 2005
Posts: 761
Website

Re: SETLOCAL ENABLEDELAYEDEXPANSION

OK I think the page is right now
http://ss64.com/nt/setlocal.html#enabledelayedexpansion

The best explanation of Delayed Expansion I've found is this page on Raymond Chens blog:
http://blogs.msdn.com/b/oldnewthing/arc … 14650.aspx

Offline

Board footer

Powered by FluxBB