Unable to install npm package (kafka-streams)

问题

I am trying to use npm package kafka-streams but getting below error:

PS D:\Projects\POCs\kstreams-poc> npm install kafka-streams

> node-rdkafka@2.7.1 install D:\Projects\POCs\kstreams-poc\node_modules\node-rdkafka
> node-gyp rebuild


D:\Projects\POCs\kstreams-poc\node_modules\node-rdkafka>if not defined npm_config_node_gyp (node "C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
gyp ERR! find VS
gyp ERR! find VS msvs_version not set from command line or npm config
gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt
gyp ERR! find VS could not use PowerShell to find Visual Studio 2017 or newer
gyp ERR! find VS looking for Visual Studio 2015
gyp ERR! find VS - not found
gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8
gyp ERR! find VS
gyp ERR! find VS **************************************************************
gyp ERR! find VS You need to install the latest version of Visual Studio
gyp ERR! find VS including the "Desktop development with C++" workload.
gyp ERR! find VS For more information consult the documentation at:
gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows
gyp ERR! find VS **************************************************************
gyp ERR! find VS
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Visual Studio installation to use
gyp ERR! stack     at VisualStudioFinder.fail (C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:121:47)
gyp ERR! stack     at C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:74:16
gyp ERR! stack     at VisualStudioFinder.findVisualStudio2013 (C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:344:14)
gyp ERR! stack     at C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:70:14
gyp ERR! stack     at C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:365:16
gyp ERR! stack     at C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:54:7
gyp ERR! stack     at C:\Users\virtual\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:33:16
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:302:5)
gyp ERR! stack     at ChildProcess.emit (events.js:203:13)
gyp ERR! stack     at maybeClose (internal/child_process.js:1021:16)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\virtual\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\Projects\POCs\kstreams-poc\node_modules\node-rdkafka
gyp ERR! node -v v12.7.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok
npm WARN kstreams-poc@1.0.0 No description
npm WARN kstreams-poc@1.0.0 No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! node-rdkafka@2.7.1 install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-rdkafka@2.7.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Tried few things like:
1. Installed windows-build-tools
2. Installed python (2.7.16)
3. Updated node (v12.7.0)
4. Installed Visual C++ build tool.

As suggested by Matt I have installed Visual C++ Build tool too. Now I'm getting below error:

gyp info spawn args ]
gyp: binding.gyp not found (cwd: D:\Projects\POCs\kstreams-poc) while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\virtual\AppData\Roaming\npm\node_modules\node-gyp\lib\configure.js:344:16)
gyp ERR! stack     at ChildProcess.emit (events.js:203:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\virtual\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\Projects\POCs\kstreams-poc
gyp ERR! node -v v12.7.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok

Update: 1
Finally, I've been able to build node-gyp successfully:

PS D:\Projects\POCs\test-kstream> node-gyp configure
gyp info it worked if it ends with ok
gyp info using node-gyp@5.0.3
gyp info using node@12.7.0 | win32 | x64
gyp info find Python using Python version 2.7.16 found at "C:\Python27\python.exe"
gyp info find VS using VS2017 (15.9.28307.770) found at:
gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools"
gyp info find VS run with --verbose for detailed information
gyp info spawn C:\Python27\python.exe
gyp info spawn args [## Heading ##
gyp info spawn args   'C:\\Users\\virtual\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'D:\\Projects\\POCs\\test-kstream\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\virtual\\AppData\\Roaming\\npm\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\virtual\\AppData\\Local\\node-gyp\\Cache\\12.7.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\virtual\\AppData\\Local\\node-gyp\\Cache\\12.7.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\virtual\\AppData\\Roaming\\npm\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\Users\\virtual\\AppData\\Local\\node-gyp\\Cache\\12.7.0\\<(target_arch)\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=D:\\Projects\\POCs\\test-kstream',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'D:\\Projects\\POCs\\test-kstream\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
Warning: Missing input files:
D:\Projects\POCs\test-kstream\build\..\build\Release\binding.node
gyp info ok

But still when trying to install kafka-stream getting below error

d:\projects\pocs\test-kstream\node_modules\node-rdkafka\src\workers.cc(909): warning C4996: 'Nan::Callback::Call': was declared deprecated [
D:\Projects\POCs\test-kstream\node_modules\node-rdkafka\build\node-librdkafka.vcxproj]
  d:\projects\pocs\test-kstream\node_modules\nan\nan.h(1740): note: see declaration of 'Nan::Callback::Call'
d:\projects\pocs\test-kstream\node_modules\node-rdkafka\src\workers.h(42): warning C4996: 'Nan::Callback::Call': was declared deprecated (co
mpiling source file ..\src\admin.cc) [D:\Projects\POCs\test-kstream\node_modules\node-rdkafka\build\node-librdkafka.vcxproj]
  d:\projects\pocs\test-kstream\node_modules\nan\nan.h(1740): note: see declaration of 'Nan::Callback::Call' (compiling source file ..\src\a
  dmin.cc)
d:\projects\pocs\test-kstream\node_modules\node-rdkafka\src\admin.cc(93): warning C4996: 'v8::Object::Set': was declared deprecated [D:\Proj
ects\POCs\test-kstream\node_modules\node-rdkafka\build\node-librdkafka.vcxproj]
  c:\users\virtual\.node-gyp\12.7.0\include\node\v8.h(3365): note: see declaration of 'v8::Object::Set'
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:266:23)
gyp ERR! stack     at ChildProcess.emit (events.js:203:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\Projects\POCs\test-kstream\node_modules\node-rdkafka
gyp ERR! node -v v12.7.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

It seems Kafka-streams it trying to use node-gyp version (v3.8.0) but I have

PS D:\Projects\POCs\test-kstream> node-gyp --version
v5.0.3

Is there any way I can make kafka-streams to use node-gyp version (v5.0.3)

Update: 2

Made few changes and now when I'm running npm i kafka-streams it now trying to get node-gyp v5.0.3. Still failing:

gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\aftab.ahmed\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\Projects\POCs\node_modules\node-rdkafka
gyp ERR! node -v v12.7.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open 'D:\Projects\POCs\package.json'
npm WARN POCs No description
npm WARN POCs No repository field.
npm WARN POCs No README data
npm WARN POCs No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! node-rdkafka@2.7.1 install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-rdkafka@2.7.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

回答1:

Looks like its looking for a visual studio version to compile C++. Error: Could not find any Visual Studio installation to use it suggests you go here: https://github.com/nodejs/node-gyp#on-windows

This is the step you're missing:

Install Visual C++ Build Environment: Visual Studio Build Tools (using "Visual C++ build tools" workload) or Visual Studio 2017 Community (using the "Desktop development with C++" workload)

回答2:

kafka-streams package has a dependency on node-rdkafka which wraps the native library librdkafka. Because of all these dependencies, it seems using kafka-streams is a bit tricky. I tried it on multiple windows machine and on one Linux machine too.

After spending some days, somehow I managed to install it using the below steps.

Uninstall node.js (make sure to remove cache C:\Users\aftab.ahmed\AppData\Roaming\npm-cache)
Install latest node.js(v10.16.0). This also installs npm version 6.10.0.
Install node-gyp using npm install -g node-gyp@5.0.3.
Install VC++ / Visual Studio 2010 build tools.
Add the path for the 12.0-version VC++ build tools to your Path environment variable.
Install Python 2.7.16 (3.x or later will not work).
Add the path for Python 2.7 to the Path environment variable.
Download Win32_OpenSSL_1.0.2p.exe and install using all defaults.

After following above steps we can now install kafka-streams