How to export pandas data to elasticsearch?

Question

It is possible to export a pandas dataframe data to elasticsearch using elasticsearch-py. For example, here is some code:

https://www.analyticsvidhya.com/bl

Answer 1

I'm not aware of any to_elastic method integrated in pandas. You can always raise an issue on the pandas github repo or create a pull request.

However, there is espandas which allows to import a pandas DataFrame to elasticsearch. The following example from the README has been tested with Elasticsearch 6.2.1.

import pandas as pd
import numpy as np
from espandas import Espandas

df = (100 * pd.DataFrame(np.round(np.random.rand(100, 5), 2))).astype(int)
df.columns = ['A', 'B', 'C', 'D', 'E']
df['indexId'] = (df.index + 100).astype(str)

INDEX = 'foo_index'
TYPE = 'bar_type'
esp = Espandas()
esp.es_write(df, INDEX, TYPE)

Retrieving the mappings with GET foo_index/_mappings:

{
  "foo_index": {
    "mappings": {
      "bar_type": {
        "properties": {
          "A": {
            "type": "long"
          },
          "B": {
            "type": "long"
          },
          "C": {
            "type": "long"
          },
          "D": {
            "type": "long"
          },
          "E": {
            "type": "long"
          },
          "indexId": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}