negative lookahead Regexp doesnt work in ES dsl query

99封情书 提交于 2019-12-02 00:44:15

问题


The mapping of my Elastic search looks like below:

{
  "settings": {
    "index": {
      "number_of_shards": "5",
      "number_of_replicas": "1"
    }
  },
  "mappings": {
    "node": {
      "properties": {
        "field1": {
          "type": "keyword"
        },
        "field2": {
          "type": "keyword"
        },
        "query": {
          "properties": {
            "regexp": {
              "properties": {
                "field1": {
                  "type": "keyword"
                },
                "field2": {
                  "type": "keyword"
                }
              }
            }
          }
        }
      }
    }
  }
}

Problem is :

I am forming ES queries using elasticsearch_dsl Q(). It works perfectly fine in most of the cases when my query contains any complex regexp. But it totally fails if it contains regexp character '!' in it. It doesn't give any result when the search term contains '!' in it.

For eg:

1.) Q('regexp', field1 = "^[a-z]{3}.b.*") (works perfectly)

2.) Q('regexp', field1 = "^f04.*") (works perfectly)

3.)Q('regexp', field1 = "f00.*") (works perfectly)

4.) Q('regexp', field1 = "f04baz?") (works perfectly)

Fails in below case:

5.) Q('regexp', field1 = "f04((?!z).)*") (Fails with no results at all)

I tried adding "analyzer":"keyword" along with "type":"keyword" as above in the fields, but in that case nothing works.

In the browser i tried to check how analyzer:keyword will work on the input on the case it fails:

http://localhost:9210/search/_analyze?analyzer=keyword&text=f04((?!z).)*

Seems to look fine here with result:

{
  "tokens": [
    {
      "token": "f04((?!z).)*",
      "start_offset": 0,
      "end_offset": 12,
      "type": "word",
      "position": 0
    }
  ]
}

I'm running my queries like below:

search_obj = Search(using = _conn, index = _index, doc_type = _type).query(Q('regexp', field1 = "f04baz?"))
count = search_obj.count()
response = search_obj[0:count].execute()
logger.debug("total nodes(hits):" + " " + str(response.hits.total))

PLease help, its really a annoying problem as all the regex characters work fine in all the queries except !.

Also, how do i check what analyzer is currently applied with above setting in my mappings?


回答1:


ElasticSearch Lucene regex engine does not support any type of lookarounds. The ES regex documentation is rather ambiguous saying matching everything like .* is very slow as well as using lookaround regular expressions (which is not only ambiguous, but also wrong since lookarounds, when used wisely, may greatly speed up regex matching).

Since you want to match any string that contains f04 and does not contain z, you may actually use

[^z]*fo4[^z]*

Details

  • [^z]* - any 0+ chars other than z
  • fo4 - fo4 substring
  • [^z]* - any 0+ chars other than z.

In case you have a multicharacter string to "exclude" (say, z4 rather than z), you may use your approach using a complement operator:

.*f04.*&~(.*z4.*)

This means almost the same but does not support line breaks:

  • .* - any chars other than newline, as many as possible
  • f04 - f04
  • .* - any chars other than newline, as many as possible
  • & - AND
  • ~(.*z4.*) - any string other than the one having z4


来源:https://stackoverflow.com/questions/51783196/negative-lookahead-regexp-doesnt-work-in-es-dsl-query

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!