How can C#'s string.IndexOf perform so fast, 10 times faster than ordinary for loop find?

问题

I have a very long string (60MB in size) in which I need to find how many pairs of '<' and '>' are in there.

I have first tried my own way:

        char pre = '!';
        int match1 = 0;
        for (int j = 0; j < html.Length; j++)
        {
            char c = html[j];
            if (pre == '<' && c == '>') //find a match
            {
                pre = '!';
                match1++;
            }
            else if (pre == '!' && c == '<')
                pre = '<';
        }

The above code runs on my string for roughly 1000 ms.

Then I tried using string.IndexOf

        int match2 = 0;
        int index = -1;
        do
        {
            index = html.IndexOf('<', index + 1);
            if (index != -1) // find a match
            {
                index = html.IndexOf('>', index + 1);
                if (index != -1)
                   match2++;
            }
        } while (index != -1);

The above code runs for only around 150 ms.

I am wondering what is the magic that makes string.IndexOf runs so fast?

Anyone can inspire me?

Edit

Ok, according to @BrokenGlass's answer. I modified my code in the way that they don't check the pairing, instead, they check how many '<' in the string.

        for (int j = 0; j < html.Length; j++)
        {
            char c = html[j];
            if (c == '>')
            {
                match1++;
            }
        }

the above code runs for around 760 ms.

Using IndexOf

        int index = -1;
        do
        {
            index = html.IndexOf('<', index + 1);
            if (index != -1)
            {
                match2++;
            }
        } while (index != -1);

The above code runs for about 132 ms. still very very fast.

Edit 2

After read @Jeffrey Sax comment, I realised that I was running in VS with Debug mode.

Then I built and ran in release mode, ok, IndexOf is still faster, but not that faster any more.

Here is the results:

For the pairing count: 207ms VS 144ms

For the normal one char count: 141ms VS 111ms.

My own codes' performance was really improved.

Lesson learned: when you do the benchmark stuff, do it in release mode!

回答1:

Are you running your timings from within Visual Studio? If so, your code would run significantly slower for that reason alone.

Aside from that, you are, to some degree, comparing apples and oranges. The two algorithms work in a different way.

The IndexOf version alternates between looking for an opening bracket only and a closing bracket only. Your code goes through the whole string and keeps a status flag that indicates whether it is looking for an opening or a closing bracket. This takes more work and is expected to be slower.

Here's some code that does the comparison the same way as your IndexOf method.

int match3 = 0;
for (int j = 0; j < html.Length; j++) {
    if (html[j] == '<') {
        for (; j < html.Length; j++)
            if (html[j] == '>')
                match3++;
    }
}

In my tests this is actually about 3 times faster than the IndexOf method. The reason? Strings are actually not quite as simple as sequences of individual characters. There are markers, accents, etc. String.IndexOf handles all that extra complexity properly, but it comes at a cost.

回答2:

The only thing that comes to my mind is actual implementation of IndexOf iniside string class, that call

callvirt    System.String.IndexOf

which, if we use a power of reflector (as much as it possible) ends up into the

CompareInfo.IndexOf(..)

call, which instead use super fast windows native function FindNLSString:

回答3:

It's a bit of a fallacy to directly compare your managed implementation and the String.IndexOf method. The implementation of IndexOf is largely native code and hence has a different set of performance characteristics than your managed implementation. In particular native code avoid the type safety checks, and their corresponding overhead, which injected by the CLR in the managed algorithm.

One example is the use of the array index

char c = html[j];

In managed code this statement will both verify that j is a valid index into the array html and then return the value. The native code equivalent simply returns a memory offset with no additional checking necessary. This lack of a check gives native code an inherent advantage here because it can avoid an additional branch instruction on every iteration of the loop.

Note that this advantage is not absolute. The JIT could very well examine this loop and decide that j could never be an invalid index and elide the checks in the generated native code (and it does do this in some cases IIRC).

回答4:

I would expect string.IndexOf compared to your first code sample to run at least twice as fast (besides any other optimizations that might be done there) since you check for both start and end character in each of your iterations. Your implementation with string.IndexOf on the other hand will check for the end character only after it has successfully found a start character. This cuts the number of operations on each iteration down significantly (one less comparison).

回答5:

Using a switch statement instead of an if test speeds things up a little also. This code occasionally beats the indexof code on my machine.

        int count = 0;
        bool open = false;
        for (int j = 0; j < testStr.Length; j++)
        {  
            switch (testStr[j])
            {
                case '<':
                    open = true;
                    break;
                case '>':
                    if (open)
                       count++;

                    open = false;
                    break;         
            }
        }

来源：https://stackoverflow.com/questions/10519380/how-can-cs-string-indexof-perform-so-fast-10-times-faster-than-ordinary-for-l

标签

string

indexof