Pyspark: filter dataframe by regex with string formatting?

Question

I\'ve read several posts on using the \"like\" operator to filter a spark dataframe by the condition of containing a string/expression, but was wondering if the following is a \

Answer 1

From neeraj's hint, it seems like the correct way to do this in pyspark is:

expr = "Arizona.*hot"
dk = dx.filter(dx["keyword"].rlike(expr))

Note that dx.filter($"keyword" ...) did not work since (my version) of pyspark didn't seem to support the $ nomenclature out of the box.