Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node must de-duplicate environment variables before calling CreateProcess (and upon env.extend()) #35129

Closed
asklar opened this issue Sep 9, 2020 · 7 comments
Labels
duplicate Issues and PRs that are duplicates of other issues or PRs. process Issues and PRs related to the process subsystem. windows Issues and PRs related to the Windows platform.

Comments

@asklar
Copy link

asklar commented Sep 9, 2020

  • Version: 12.9.1
  • Platform: x64
  • Subsystem: Windows 10

What steps will reproduce the bug?

yarn sets up some environment variables (like npm_config_cache) and calls execSync. This calls the Win32 API CreateProcess and passes an environment block which is a merge of the existing environment block (i.e. from the calling process) with some additional stuff that includes npm_config_cache among others.

Environment variables in Windows are case-insensitive, but CreateProcess does not seem to sanitize the environment block. This is a problem because if the calling process had set up NPM_CONFIG_CACHE (all caps), then yarn will just set the lowercase variant of the variable, and node will call CreateProcess which will contain the variable twice). This breaks some tools that create dictionaries out of the environment block because they expect to run into each variable exactly once (regardless of casing).

In my case we have a nodejs script that we invoke via yarn, and the nodejs script calls a Windows build utility (msbuild) which crashes when trying to set up options for the compiler when it finds this option twice. We run into this when building in our CI in Azure DevOps pipeline, which sets up NPM_CONFIG_CACHE in the environment.

Related:

How often does it reproduce? Is there a required condition?

100%

What is the expected behavior?

node should de-duplicate variables before passing them to CreateProcess

What do you see instead?

CreateProcess gets an environment block with both variables:
image

Additional information

@bzoz
Copy link
Contributor

bzoz commented Sep 9, 2020

This is basically a duplicate of #34667

When spawn gets the object with the env pairs there is no way for it to know which one is the "true one", there is no record of which was added later. This is a known, documented limitation and unfortunately, there is no way of fixing this in Node.

In this case, yarn needs to be fixed to correctly handle environment variables on Windows.

@bnoordhuis bnoordhuis added duplicate Issues and PRs that are duplicates of other issues or PRs. process Issues and PRs related to the process subsystem. windows Issues and PRs related to the Windows platform. labels Sep 10, 2020
@bnoordhuis
Copy link
Member

Closing as a duplicate.

@asklar
Copy link
Author

asklar commented Sep 10, 2020

@bnoordhuis @bzoz The documentation only mentions PATH as being the special case but in fact all environment variables are susceptible to this problem. Could you please update the documentation to reflect this?

Moreover, the doc says it will sort the variables lexicographically and use the first occurrence of PATH. This is the same behavior that the OS GetEnvironmentVariable will do, so once you get into the duplicate variable case, it doesn't matter which one was added last - the only one that is reachable via GetEnvironmentVariable is going to be the first lexicographically, so spawn could sanitize the env block by doing exactly that, only keep the first instance of each variable

@bzoz
Copy link
Contributor

bzoz commented Sep 10, 2020

Those points make sense, both the doc update and the sanitization.

I think keeping only one entry for each env variable would be a semver-major though.

@asklar
Copy link
Author

asklar commented Sep 15, 2020

@bzoz any clue when we might see this fixed?

@bzoz
Copy link
Contributor

bzoz commented Sep 15, 2020

I've opened a PR with a fix: #35210. I guess this is a semver-major though, so I would expect this to be shipped with the next major release of Node.

@asklar
Copy link
Author

asklar commented Sep 15, 2020

thanks a lot @bzoz. From the release doc it sounds like the next release is October so that would be ok assuming we can get this in soon 🤞

bzoz added a commit to JaneaSystems/node that referenced this issue Sep 24, 2020
On Windows environment variables are case-insensitive. When spawning
child process certain apps can get confused if some of the variables are
duplicated.

This adds a step on Windows to normalizeSpawnArguments that removes such
duplicates, keeping only the first (in lexicographic order) entry in the
env key of options. This is partly already done for the PATH entry.

Fixes: nodejs#35129
@bzoz bzoz closed this as completed in abd8cdf Sep 24, 2020
joesepi pushed a commit to joesepi/node that referenced this issue Jan 8, 2021
On Windows environment variables are case-insensitive. When spawning
child process certain apps can get confused if some of the variables are
duplicated.

This adds a step on Windows to normalizeSpawnArguments that removes such
duplicates, keeping only the first (in lexicographic order) entry in the
env key of options. This is partly already done for the PATH entry.

Fixes: nodejs#35129

PR-URL: nodejs#35210
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate Issues and PRs that are duplicates of other issues or PRs. process Issues and PRs related to the process subsystem. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants
@bnoordhuis @bzoz @asklar and others