StackExchange.Redis server.Keys(pattern:“IsVerySlow*”)

Question

I'm new to redis so I'm doing something wrong, I'm sure:

I've stored roughly 16,000 key/values in Azure Redis.

I used the following to write the keys/values

      foreach (var worksheet in wksList)
      {
        var wksYYMM = string.Format("{0}{1:00}", worksheet.ReviewDt.Year, worksheet.ReviewDt.Month);
        var wksKey = string.Format("{0}:{1}:{2}", provCode, wksYYMM, worksheet.AcctNbr);
        string jsonStr = JsonConvert.SerializeObject( MakeWsListDto(worksheet, provCoderList, rvrList));
        cache.StringSet(wksKey, jsonStr);
      }

so my keys look like this: "AP:201401:AZ5798BK"

When I try a lookup like:

    var keys = server.Keys(pattern: "AP:201401:*"); // returns in milliseconds
    var keyAry = keys.ToArray(); // returns in over one minute
    (note: this returns 12 keys)

it takes 1 min 12 seconds to return the keys. Once I have the keys it takes milliseconds to retrieve the values for those. If I iterate over the value of keys and return the values I get a similar result. I did a ToArray() just to isolate the problem.

If I try the same query in redis-cli.exe it comes back in milliseconds.

Am I using this command incorrectly?

Answer 1

server.Keys automatically selects between KEYS and the preferred SCAN based on server version. I suspect what is happening is that you are thus using SCAN with a too-small page-size. There is an optional parameter for page size. Try specifying something considerably larger than the default - hundreds, thousands, etc. If not specified, the page-size uses the Redis SCAN default of 10, which could cause a lot of round-trips to be required.

Answer 2

Do not use KEYS - it is a blocking command that will Render your Redis server unavailable to other request while it is running. Quoting from the command's documentation:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using SCAN or sets.

If you read the warning thoroughly, you'll notice the recommended approaches at the end of the paragraph, namely using SCAN or Redis' sets. SCAN is non-blocking but since your question suggests that that you're into performance, I recommend using sets.

The idea is to maintain a Redis set with all the key names that are associated with that "pattern", so in your example you have to do the StackExchange.Redis-equivalent of SADD AP:201401 AP:201401:AZ5798BK after your call to cache.SetString, e.g.:

cache.SetAdd(wksYYMM, wksKey);

Disclaimer: I am not a C# programmer nor am I too familiar with StackExchange.Redis (sorry Marc ;))

Now, instead of KEYS or SCAN to get your keys, just do SMEMBERS AP:201401 or probably:

var keys = cache.Members(wksYYMM);

Bonus: since you're actually interested in the values of the these keys, you can use Redis' Lua scripting to fetch the keys values based on the set's members, or just use SORT.

Pure Redis:

SORT AP:201401 BY nosort GET *`

C# & StackExchange.Redis:

vals = cache.Sort(wksYYMM, by = "nosort", get = "*");