SQLAlchemy through Paramiko SSH

问题

I have a database on a server which I need to access through SSH. Right now I deal with the DB by using the command line to get the data.

import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='XX.XX.XX', username='user', password='pass', port = YYY)
query = "mysql -u " + username_sql + " -p" + password_sql +" dbb -e \"" + sql_query + "\""
ssh.exec_command(query.decode('string_escape'))
ssh.close()

Is there a way to do this with SQLAlchemy to be more efficient and so I can work with pandas DataFrames directly?

from sqlalchemy import create_engine
engine = create_engine(
       "mysql://username_sql:password_sql@localhost/dbb")

回答1:

The easiest way to do this would be to run an SSH tunnel to the mysql port on the remote host. For example:

ssh -f user@XX.XX.XX.XX -L 3307:mysql1.example.com:3306 -N

Then connect locally with SQLAlchemy:

engine = create_engine("mysql://username_sql:password_sql@localhost:3307/dbb")

If you really want to use paramiko, try this demo code in the paramiko repo or the sshtunnel module. The ssh command might be the easiest method though.. and you can use autossh to restart the tunnel if it goes down.

回答2:

In case there is anyone who's interested in connecting to a remote Postgresql database via SSH and wants to load data into a pandas DataFrame here is how to do it.

Suppose we have installed a postgresql database on a remote server, to which we can ssh by the following parameters.

SSH parameters:

Server's ip: 10.0.0.101
SSH port: 22 (default port for SSH)
Username: my_username
Password: my_password

Database parameters:

Port: 5432 (postgresql default port)
Database name: db
Database user: postgres_user (default username is postgres)
Database password: postgres_pswd (default password is an empty string)
Table with our data: MY_TABLE

Now, we want to connect to this database on our end and load data into a pandas DataFrame:

from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
import pandas as pd

server = SSHTunnelForwarder(
    ('10.0.0.101', 22),
    ssh_username="my_username",
    ssh_password="my_password",
    remote_bind_address=('127.0.0.1', 5432)
    )

server.start()
local_port = str(server.local_bind_port)
engine = create_engine('postgresql://{}:{}@{}:{}/{}'.format("postgres_user", "postgres_pswd", "127.0.0.1", local_port, "db"))

dataDF = pd.read_sql("SELECT * FROM \"{}\";".format("MY_TABLE"), engine)

server.stop()

回答3:

You could use the SSHTunnel library as follows:

from sshtunnel import SSHTunnelForwarder #Run pip install sshtunnel
from sqlalchemy.orm import sessionmaker #Run pip install sqlalchemy

with SSHTunnelForwarder(
    ('10.160.1.24', 22), #Remote server IP and SSH port
    ssh_username = "<usr>",
    ssh_password = "<pwd>",
    remote_bind_address=('127.0.0.1', 5432)
    ) as server:

    server.start() #start ssh sever
    print 'Server connected via SSH'

    #connect to PostgreSQL
    local_port = str(server.local_bind_port)
    engine = create_engine('postgresql://<db_user>:<db_pwd>@127.0.0.1:' + local_port +'/<db_name>')

    Session = sessionmaker(bind=engine)
    session = Session()

    print 'Database session created'

    #test data retrieval
    test = session.execute("SELECT * FROM <table_name>")

回答4:

Just swap the (host, port) of the server with postgres:

from sshtunnel import SSHTunnelForwarder #Run pip install sshtunnel

server = SSHTunnelForwarder(
    (<'your host'>, <host port>),
    ssh_username=<"os remote username">,
    ssh_pkey=<'path/to/key.pem'>,  # or ssh_password.
    remote_bind_address=(<'postgres db host'>, <'postgres db port'>))

server.start()

connection_data = 'postgresql://{user}:{password}@{host}:{port}/{db}'.format(user=<'postgres user'>,
                                                                             password=<'postgres password'>,
                                                                             host=server.local_bind_host,
                                                                             port=server.local_bind_port,
                                                                             db=<'postgres db name'>)

engine = create_engine(connection_data)

# Do your queries

server.stop()

回答5:

I'm going to piggy back @Matin Kh with a non-postgresql db - MySQL and using Pythonanywhere.com. This code will take a table and convert it to an excel file.

import sshtunnel
import sqlalchemy
import pymysql
import pandas as pd
from pandas import ExcelWriter
import datetime as dt
from sshtunnel import SSHTunnelForwarder

server = SSHTunnelForwarder(
    ('ssh.pythonanywhere.com'),
    ssh_username='username', 
    ssh_password='password',
    remote_bind_address=('username.mysql.pythonanywhere-services.com', 3306) ) 

server.start()
local_port = str(server.local_bind_port)
db = 'username$database'
engine = sqlalchemy.create_engine(f'mysql+pymysql://username:password@127.0.0.1:{local_port}/{db}')

print('Engine Created')

df_read = pd.read_sql_table('tablename',engine)
print('Grabbed Table')
writer = ExcelWriter('excelfile.xlsx')
print('writer created')
df_read.to_excel(writer,'8==D') # '8==D' specifies sheet
print('df to excel')
writer.save()
print('saved')
server.stop()

来源：https://stackoverflow.com/questions/31506958/sqlalchemy-through-paramiko-ssh

标签

python

mysql

ssh

sqlalchemy

paramiko