问题
I am trying to catch paramiko exceptions but they are still written to stderr.
Is there a way to stop writing there?
EDIT: It happens even before paramiko gets involved:
import pysftp
try:
pysftp.Connection(host="localhost")
except Exception as e:
print(e)
Results in:
Example with proper SFTP params:
UPDATE:
$ pipenv graph
...
pysftp==0.2.9
- paramiko [required: >=1.17, installed: 2.6.0]
...
$ pipenv run python
Python 3.7.3 (default, Jul 19 2019, 11:21:39)
[Clang 11.0.0 (clang-1100.0.28.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysftp
>>> try:
... pysftp.Connection(host="localhost")
... except Exception as e:
... print(e)
...
No hostkey for host localhost found.
Exception ignored in: <function Connection.__del__ at 0x10f7e8268>
Traceback (most recent call last):
File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 1013, in __del__
self.close()
File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 784, in close
if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
>>>
回答1:
I want to start by pointing out that PySFTP ([PyPI]: PySFTP) has not been maintain for 3+ years (or has been moved to a different location - which so far is kept secret :) ).
I reproduced the problem. Below is a more verbose version of your code.
code00.py:
#!/usr/bin/env python3
import sys
import pysftp
import traceback
def main(argv):
hostname = argv[0] if argv else "localhost"
print("Attempting to connect to {0:s} ...".format(hostname))
try:
print("----------Before conn----------")
conn = pysftp.Connection(host=hostname)
print("----------After conn----------")
except:
print("----------Before exc print----------")
traceback.print_exc()
print("----------After exc print----------")
finally:
print("----------Finally----------")
print("----------After try / except / finally----------")
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
print("pysftp version: {0:s}\n".format(pysftp.__version__))
main(sys.argv[1:])
print("\nDone.")
Output:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 Attempting to connect to localhost ... ----------Before conn---------- e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts. You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None). warnings.warn(wmsg, UserWarning) ----------Before exc print---------- Traceback (most recent call last): File "code00.py", line 13, in main conn = pysftp.Connection(host=hostname) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 132, in __init__ self._tconnect['hostkey'] = self._cnopts.get_hostkey(host) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host localhost found. ----------After exc print---------- Exception ignored in: <function Connection.__del__ at 0x000001CC720C80D0> Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 1013, in __del__ self.close() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 784, in close if self._sftp_live: AttributeError: 'Connection' object has no attribute '_sftp_live' ----------Finally---------- ----------After try / except / finally---------- Done.
It's a PySFTP bug:
- The Connection object is constructed (__new__)
- The initializer (__init__) is called
- Somewhere in the initializer an exception occurs
- The lines after the exception are not executed
- When the object is (automatically) garbage collected (when it goes out of scope, at the end of the except block), in its close method (called by the destructor (__del__)), some attributes are referenced
- But since those attributes initialization occurs after the line (raising the exception) from #2.2., they were never initialized, so their reference raises AttributeError
The fix is straightforward: initialize the attributes to some default values, at the initializer's beginning, so their reference doesn't represent a problem if the above scenario happens.
I noticed that you've already submitted an issue on BitBucket.
Considering that:
- I'm not a BitBucket expert
- I don't have permissions on [BitBucket]: dundeemt/pysftp - pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. (I can neither push nor submit pull requests)
I created my own repo (out of the above one), and pushed the changes at: [BitBucket]: CristiFati0/pysftp - [Issue #144]: Exceptions leaking into stderr (one commit, so far).
Output (after manually applying the fix to the file installed by pip):
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 Attempting to connect to localhost ... ----------Before conn---------- e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts. You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None). warnings.warn(wmsg, UserWarning) ----------Before exc print---------- Traceback (most recent call last): File "code00.py", line 13, in main conn = pysftp.Connection(host=hostname) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 135, in __init__ self._tconnect['hostkey'] = self._cnopts.get_hostkey(host) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host localhost found. ----------After exc print---------- ----------Finally---------- ----------After try / except / finally---------- Done.
Needless to say, other unhandled exceptions might be raised for different scenarios.
@EDIT0
Apparently, ther's more to this than meets the eye. Besides the PySFTP bug described above, there are 2 more things that pollute stderr.
1. The warning
In my case (as I am on Win and don't have any native SSH tool installed) it pops up every time (unless I create / copy some valid known_hosts file in my home dir), but on Nix systems it most likely won't.
Anyway, the fix for this one (if wanted) is easy, simply suppress the UserWarning for example by setting the %PYTHONWARNINGS% env var to ignore::UserWarning (can also be achieved from code - like in next item's case).
2. Paramiko exceptions
I was able to reproduce this by manually modifying transport.py (and raising socket.timeout).
paramiko.Transport which is initialized by pysftp.Connection._start_transport (called by the initializer) does its work in a thread (by subclassing threading.Thread). Any exception raised in that thread, can't be caught by the calling thread (ours). This is a Python limitation scheduled to be addressed in v3.8 ([Python.Bugs]: threading.Thread should have way to catch an exception thrown within).
For this one, there is a (lame) workaround (gainarie): redirecting stderr. Of course there are other workarounds, but they would imply modifying Paramiko, so I'd advice against them.
Below is an example that redirects stderr to stdout (but you can choose any other file - including /dev/null (or nul on Win)). It does it from code (but it can also be done from interpreter command line) so that it only affects the desired (hot) area(s).
code01.py:
#!/usr/bin/env python3
import sys
import pysftp
import paramiko
import traceback
import threading
_sys_stderr = sys.stderr # For restoring purposes
def main(argv):
hostname = argv[0] if argv else "localhost"
print("Attempting to connect to {0:s} ...".format(hostname))
try:
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
print("---------- STATS: {0:s} {1:d} ----------".format(__file__, threading.get_ident()))
print("---------- Before conn ----------")
sys.stderr.write("DUMMY TEXT before sent to stderr\n")
sys.stderr = sys.stdout # @TODO - cfati: decomment so that everything from stderr is redirected to stdout
conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,)
print("---------- After conn ----------")
except:
sys.stderr = _sys_stderr
print("---------- Before exc tb ----------")
traceback.print_exc(file=sys.stdout)
print("---------- After exc tb ----------")
finally:
sys.stderr = _sys_stderr
print("---------- Finally ----------")
sys.stderr.write("DUMMY TEXT after sent to stderr\n")
print("---------- After try / except / finally ----------")
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
print("pysftp version: {0:s}\nparamiko version: {1:s}".format(pysftp.__version__, paramiko.__version__))
main(sys.argv[1:])
print("\nDone.")
Output:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> sopr.bat *** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages *** [prompt]> dir /b code00.py code01.py [prompt]> :: Suppress warning [prompt]> set PYTHONWARNINGS=ignore::UserWarning [prompt]> :: Redirect stdout and stderr to different files, so it is obvious which is which [prompt]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code01.py 1>1.out 2>1.err [prompt]> type 1.err DUMMY TEXT before sent to stderr DUMMY TEXT after sent to stderr [prompt]> type 1.out Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 paramiko version: 2.6.0 Attempting to connect to localhost ... ---------- STATS: code01.py 23016 ---------- ---------- Before conn ---------- ---------- STATS: e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py 45616 ---------- Exception: Error reading SSH protocol banner Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner raise socket.timeout() socket.timeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run self._check_banner() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner "Error reading SSH protocol banner" + str(e) paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ---------- Before exc tb ---------- Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner raise socket.timeout() socket.timeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "code01.py", line 23, in main conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 144, in __init__ self._transport.connect(**self._tconnect) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 1291, in connect self.start_client() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 660, in start_client raise e File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run self._check_banner() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner "Error reading SSH protocol banner" + str(e) paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ---------- After exc tb ---------- ---------- Finally ---------- ---------- After try / except / finally ---------- Done.
And how it looks from PyCharm (had to stretch it to the max to fit the whole thing):
回答2:
I would skip pysftp and just use paramiko as it support SFTP just fine by itself:
import paramiko
host,port = "example.com",22
transport = paramiko.Transport((host,port))
username,password = "bar","foo"
transport.connect(None,username,password)
sftp = paramiko.SFTPClient.from_transport(transport)
Then here you can again try to catch the exception at exactly where it occurs and see how you fare.
Or if you need to authenticate with only password:
client = pk.SSHClient()
client.set_missing_host_key_policy(pk.AutoAddPolicy())
client.connect(hostname=ip, port=22, username=userName, password=pwd)
ftp_client = client.open_sftp()
来源：https://stackoverflow.com/questions/58110732/pysftp-paramiko-exceptions-leaking-into-stderr