using a DelayedExpansion index variable for an array within an IF statement fails

Question

I have a batch file where I am displaying all *.pem files in a directory, giving the user an option to choose one, and at this time just trying to show the user what file they picked by using the number they chose to ECHO the contents of the array. I believe that somehow because it is inside of the IF statement, the index variable "selectedPem" is not expanding in the array's index brackets because it is using the % and not using the delayed expansion !, which doesnt appear to work in the array's index brackets. I think this because if I set the "selectedPem" variable before the IF statement, it ECHOs properly. I think the only option for this to work is to somehow make use of a subroutine. Why is selectedPem variable not working inside of the IF statement?

@echo off
setlocal enabledelayedexpansion 
set dir=%~dp0
cd %dir%

ECHO Performing initial search of private and public keys...
set pemI=0
set keyI=0
for %%p in (*.pem) do (
    REM echo %%~nxp
    set pems[%pemI%]=%%~nxp
    REM echo !pems[%pemI%]!
    set /a pemI=%pemI%+1
)
REM echo %pems[0]%
set /a totalPems=%pemI%-1
REM This is the test selectedPem variable that showed me the variable works 
REM if I set it outside 
REM of the "if defined pems[0]" statement
REM set selectedPem=
if defined pems[0] (
    echo PEM Files Found:
    for /l %%p in (0,1,%totalPems%) do (
        echo %%p^) !pems[%%p]!
    )
    ECHO Select a pem file to be used as your private key. Otherwise, press 
    Q to not select one.
    set /P selectedPem=Enter a number or Q:
    IF "!selectedPem!"=="q" (
        ECHO Skipping private key selection.
    ) ELSE (
        ECHO !selectedPem!
        ECHO %pems[0]%
        REM The below ECHO only works when the above selectedPem is set 
        REM outside of the "IF defined" statement
        ECHO !pems[%selectedPem%]!

        REM Tried these other implementations:
        REM ECHO !pems[!!selectedPem!!]!
        REM ECHO %pems[!selectedPem!]%
        REM setlocal disabledelayedexpansion
        REM ECHO %pems[%%selectedPem%%]%

        set privatekey=!pems[%selectedPem%]!
        ECHO You chose %privatekey%
    )   
)`

Output:

Performing initial search of private and public keys...
PEM Files Found:
0) brandon.pem
Select a pem file to be used as your private key. Otherwise, press Q to not 
select one.
Enter a number or Q:0
0
brandon.pem
ECHO is off.
You chose

I understand that "ECHO is off." is outputted instead because it is interpreting my array variable reference as empty. I appreciate your time in reading over my script, I am likely over-complicating it.

Answer 1

You appear to be well aware of delayed expansion since you are correctly using it most of the time.

However, what you have in your code I usually call nested variables, because there are no real arrays in batch scripting since every array element is nothing but an individual scalar variable (pems[0], pems[1], pems[2], etc.); so we might refer to something like this also as pseudo-arrays.

Anyway, something like echo !pems[%selectedPem%]! can correctly expanded, unless it is placed within a line or block of code in which selectedPem becomes updated before, because then we would require delayed expansion for selectedPem as well. echo !pems[!selectedPem!]! does not work, because it tries to expand variables pems[ and ], and selectedPem is interpreted as a literal string.

Before moving forward, let us first go back to echo !pems[%selectedPem%]!: why is this expandable? Well, the trick is we have two expansion phases here, the normal or immediate expansion (%), followed by the delayed expansion (!). The important thing is the sequence: the inner one of the nested variables (hence the array index) must be expended before the outer one (so the array element). Something like echo %pems[!selectedPem!]% cannot be expanded correctly, because there is (most likely) no variable named pems[!selectedPem!], and after expanding it to an empty string, the two ! disappear and so the delayed expansion phase does never see them.

Now let us go a step further: to expand something similar to echo !pems[%selectedPem%]! inside of a line or block of code that also updates selectedPem, we must avoid immediate expansion. We already learned that delayed expansion cannot be nested, but there are some other expansions:

1. The call command introduces another expansion phase after delayed expansion, which again handles %-signs; so the full sequence is immediate expansion, delayed expansion and another %-expansion. Dismissing the first one, let us make use of the latter two in a way that the inner one of the nested variables becomes expanded before the outer one, meaning: call echo %%pems[!selectedPem!]%%. To skip the immediate expansion phase we simply double the % signs, which become replaced by a literal one each, so we have echo %pems[!selectedPem!]% after immediate expansion. The next step is delayed expansion, then the said next %-expansion.
   You should take notice that the call method is quite slow, so heavy usage could drastically reduce the overall performance of your script.
2. Expansion of for loop variables happens after immediate expansion, but before delayed expansion. So let us wrap a for loop around that iterates once only and returns the value of selectedPem, like this: for %%Z in (!selectedPem!) do echo !pems[%%Z]!. This works, because for does not access the file system unless a wild-card * or ? appears, which should not be used in variable names or (pseudo-)array indexes anyway.
   Instead of a standard for loop, for /F could be used also: for /F %%Z in ("!selectedPem!") do echo !pems[%%Z]! (there is no option string like "delims=" required in case selectedPem is expected to contain just an index number).
   A for /L loop could be used too (for numeric indexes, of course): for /L %%Z in (!selectedPem!,1,!selectedPem!) do echo !pems[%%Z]!.
3. Implicit expansion established by the set /A command, meaning that neither % nor ! is necessary to read variables, can be used too, but only if the array element contains a numeric value (note that /A stands for arithmetics). Since this is a feature specific to set /A, this kind of expansion happens during command execution, which in turn happens after delayed expansion. So we can use that like this: set /A "pem=pems[!selectedPem!]" & echo !pem!.
4. Just for the sake of completeness, here is one more way: set /A, when executed in cmd rather than in batch context, outputs the (last) result on the console; given that this constitutes the index number, we could capture this by for /F and expand the array element like this: for /F %%Z in ('set /A "selectedPem"') do echo !pems[%%Z]!. set /A is executed in a new cmd instance by for /F, so this approach is not the fastest one, of course.

There is a great and comprehensive post concerning this topic which you should go through it detail: Arrays, linked lists and other data structures in cmd.exe (batch) script.

For parsing of command lines and batch scripts as well as variable expansion in general, refer to this awesome thread: How does the Windows Command Interpreter (CMD.EXE) parse scripts?

---

Now let us take a look at your code. There are several issues:

• use cd /D instead of cd in order to change also the drive if necessary; by the way, the interim variable dir is not necessary (let me recommend to not use variable names that equal internal or external commands for the sake of readability), simply use cd on the path immediately;
• you missed using delayed expansion in line set pems[%pemI%]=%%~nxp, it should read set pems[!pemI!]=%%~nxp as pemI becomes updated within the surrounding for loop;
• you either need delayed expansion in line set /a pemI=%pemI%+1 too, or you make use of the implicit variable expansion of set /A, so set /A pemI=pemI+1 would work; this can even be more simplified however: set /A pemI+=1, which is totally equivalent;
• I would use case-insensitive comparison particularly when it comes to user input, like your check for Q, which would be if /I "!selectedPem!"=="q";
• now we come to a line that needs what we have learned above: ECHO !pems[%selectedPem%]! needs to become call echo %%pems[!selectedPem!]%%; an alternative way using for is inserted too in a comment (rem);
• then there is another line with the same problem: set privatekey=!pems[%selectedPem%]! needs to become call set privatekey=%%pems[!selectedPem!]%% (or the approach based on for too);
• once again you missed using delayed expansion, this time in line ECHO You chose %privatekey%, which should read echo You chose !privatekey!;
• finally I improved quotation of your code, in particular that of set commands, which should better be written like set "VAR=Value", so any invisible trailing white-spaces do not become part of the value, and the value itself becomes protected if it contains special characters, but the quotation marks do not become part of the value, which could disturb particularly for contatenation;

So here is the fixed code (with all your original rem remarks removed):

@echo off
setlocal EnableDelayedExpansion 
cd /D "%~dp0"

echo Performing initial search of private and public keys...
set "pemI=0"
set "keyI=0"
for %%p in (*.pem) do (
    set "pems[!pemI!]=%%~nxp"
    set /A "pemI+=1"
)
set /A "totalPems=pemI-1"
if defined pems[0] (
    echo PEM Files Found:
    for /L %%p in (0,1,%totalPems%) do (
        echo %%p^) !pems[%%p]!
    )
    set /P selectedPem="Enter a number or Q: "
    if /I "!selectedPem!"=="q" (
        echo Skipping private key selection.
    ) else (
        echo !selectedPem!
        echo %pems[0]%
        call echo %%pems[!selectedPem!]%%
        rem for %%Z in (!selectedPem!) do echo !pems[%%Z]!
        call set "privatekey=%%pems[!selectedPem!]%%"
        rem for %%Z in (!selectedPem!) do set "privatekey=!pems[%%Z]!"
        echo You chose !privatekey!
    )   
)

I have to admit I did not check the logic of your script in detail though.

Answer 2

With some goto's you can avoid most (code blocks).

In cases where this is not possible use the ! and an alternative delayed expansion with a call and doubled %% properly.

Untested:

@echo off
setlocal enabledelayedexpansion 
set dir=%~dp0
cd %dir%

ECHO Performing initial search of private and public keys...
set pemI=0
set keyI=0
for %%p in (*.pem) do (
    REM echo %%~nxp
    set pems[!pemI!]=%%~nxp
    REM call echo %%pems[!pemI!]%%
    set /a pemI+=1
)
REM echo %pems[0]%
set /a totalPems=%pemI%
REM This is the test selectedPem variable that showed me the variable works 
REM if I set it outside 
REM of the "if defined pems[0]" statement
REM set selectedPem=

if pemI==0 (Echo no *.pem files found & goto :End)

echo PEM Files Found:
for /l %%p in (0,1,%totalPems%) do (
    echo %%p^) !pems[%%p]!
)
ECHO Select a pem file to be used as your private key. Otherwise, press 
Q to not select one.
set /P selectedPem=Enter a number or Q:
IF /I "!selectedPem!"=="q" (ECHO Skipping private key selection.& goto :End

ECHO !selectedPem!
ECHO %pems[0]%
REM The below ECHO only works when the above selectedPem is set 
REM outside of the "IF defined" statement
ECHO !pems[%selectedPem%]!

REM Tried these other implementations:
REM ECHO !pems[!!selectedPem!!]!
REM ECHO %pems[!selectedPem!]%
REM setlocal disabledelayedexpansion
REM ECHO %pems[%%selectedPem%%]%

set privatekey=!pems[%selectedPem%]!
ECHO You chose %privatekey%
)   
:end
REM ddo wahatever
pause