elasticsearch_dsl: Generate multiple buckets in aggregation

Question

I want to generate this:

GET /packets-2017-09-25/_search
{
  "size": 0,
  "query": {
        "match": {
            "transport_protocol": "tcp"
        }
  },
  "aggs": {
    "clients": {
      "terms": {
        "field": "layers.ip.src.keyword",
        "size": 1000,
        "order":{ "num_servers.value":"desc" }
      },
      "aggs": {
        "num_servers": {
          "cardinality": {
            "field": "layers.ip.dst.keyword",
            "precision_threshold" : 40000
          } 
        },
        "server_list": {
          "terms": {
            "field": "layers.ip.dst.keyword"
          }
        }
      }
    }
  }
}

i.e I want two buckets (num_servers) and (server_list) under clients.

I am trying the below piece of code, which errors out:

def get_streams_per_client(proto='tcp', max=40000):
    s = Search(using=client, index="packets-2017-09-25") \
               .query("match", transport_protocol=proto)
    s.aggs.bucket('clients', 'terms', field='layers.ip.src.keyword', size=max, order={"num_servers.value":"desc"})\
    .bucket('num_servers', 'cardinality', field='layers.ip.dst.keyword', precision_threshold=40000)\
    .bucket('server_list', 'terms', field='layers.ip.dst.keyword')
    s = s.execute()
    <snip>

I think I am missing on the right syntax. Appreciate some guidance.

Answer 1

You can always reach existing aggregation using the ["name"] notation if you want to define other sub-aggregations:

s = Search().query('match', transport_protocol='tcp')
s.aggs.bucket('clients', 'terms', field='layers.ip.src.keyword', size=max, order={"num_servers.value":"desc"})
s.aggs['clients'].metric('num_servers', 'cardinality', field=..., precision_threshold=...)
s.aggs['clients'].bucket('server_list', 'terms', ...)

Hope this helps!

Answer 2

Got the answer from Honza on elasticsearch_dsl project page:

  s = Search(using=client, index="packets-2017-09-25").query('match', transport_protocol=proto)
  s.aggs.bucket('clients', 'terms', field='layers.ip.src.keyword', size=max, order={"num_servers.value":"desc"})
  s.aggs['clients'].bucket('num_servers', 'cardinality', field='layers.ip.dst.keyword', precision_threshold=40000)
  s.aggs['clients'].bucket('server_list', 'terms', field='layers.ip.dst.keyword')
  print json.dumps(s.to_dict(), indent=4)
  s = s.execute()