Performance Counter - System.InvalidOperationException: Category does not exist

Question

I have following class that returns number of current Request per Second of IIS. I call RefreshCounters every minute in order to keep Requests per Second value refreshed (be

Answer 1

I just solved this type of error or exception with:

Using,

new PerformanceCounter("Processor Information", "% Processor Time", "_Total");

Instead of,

new PerformanceCounter("Processor", "% Processor Time", "_Total");

Answer 2

I had an issue retrieving requests per second on IIS using code similar to the following

var pc = new PerformanceCounter();
pc.CategoryName = @"W3SVC_W3WP";
pc.InstanceName = @"_Total";
pc.CounterName = @"Requests / Sec";
Console.WriteLine(pc.NextValue());

This would sometimes throw InvalidOperationException and I was able to reproduce the exception by restarting IIS. If I run with a non warmed up IIS, e.g. after a laptop reboot or IIS restart, then I get this exception. Hit the website first, make any http request beforehand, and wait a second or two and I don't get the exception. This smells like the performance counters are cached,and when Idle they get dumped, and take a while to re-cache? (or similar).

Update1: Initially when I manually browse to the website and warm it up, it solves the problem. I've tried programmatically warming up the server with new WebClient().DownloadString(); Thread.Sleep() up to 3000ms and this has not worked? So my results of manually warming up server, might somehow be a false positive. I'm leaving my answer here, because it might be the cause, (i.e. manual warming up), and maybe someone else can elaborate further?

Update2: Ah, ok, here are some unit tests that summarises some learning from further experimenting I did yesterday. (There's not a lot on google on this subject btw.)

As far as I can reason, the following statements might be true; (and I submit the unit tests underneath as evidence.) I may have misinterpreted the results, so please double check ;-D

1. Create a performance counter and calling getValue before the category exists, e.g. querying an IIS counter, while IIS is cold and no process running, will throw InvalidOperation exception "category does not exist". (I assume this is true for all counters, and not just IIS.)

2. From within a Visual Studio unit test, once your counter throws an exception, if you subsequently warm up the server after the first exception, and create a new PerformanceCounter and query again, it will still throw an exception! (this one was a surprise, I assume this is because of some singleton action. My apologies I have not had enough time to decompile the sources to investigate further before posting this reply.)

3. In 2 above, if you mark the unit test with [STAThread] then I was able to create a new PerformanceCounter after one has failed. (This might have something to do with Performance counter possibly being singletons? Needs further testing.)

4. No pause was required for me before creating counter and using it, despite some warnings in MSDN same code documentation, other than the time it takes to create a performance counter itself before calling NextValue().In my case, to warm up the counter and bring the "category" into existance, was for me to fire one shot across the bow of IIS, i.e. make a single GET request, and viola, no longer get "InvalidOperationException", and this seems to be a reliable fix for me, for now. At least when querying IIS performance counters.

CreatingPerformanceCounterBeforeWarmingUpServerThrowsException

[Test, Ignore("Run manually AFTER restarting IIS with 'iisreset' at cmd prompt.")]
public void CreatingPerformanceCounterBeforeWarmingUpServerThrowsException()
{
    Console.WriteLine("Given a webserver that is cold");
    Console.WriteLine("When I create a performance counter and read next value");
    using (var pc1 = new PerformanceCounter())
    {
        pc1.CategoryName = @"W3SVC_W3WP";
        pc1.InstanceName = @"_Total";
        pc1.CounterName = @"Requests / Sec";
        Action action1 = () => pc1.NextValue();
        Console.WriteLine("Then InvalidOperationException will be thrown");
        action1.ShouldThrow<InvalidOperationException>();                
    }
}


[Test, Ignore("Run manually AFTER restarting IIS with 'iisreset' at cmd prompt.")]
public void CreatingPerformanceCounterAfterWarmingUpServerDoesNotThrowException()
{
    Console.WriteLine("Given a webserver that has been Warmed up");
    using (var client = new WebClient())
    {
        client.DownloadString("http://localhost:8082/small1.json");
    }
    Console.WriteLine("When I create a performance counter and read next value");
    using (var pc2 = new PerformanceCounter())
    {
        pc2.CategoryName = @"W3SVC_W3WP";
        pc2.InstanceName = @"_Total";
        pc2.CounterName = @"Requests / Sec";
        float? result = null;
        Action action2 = () => result = pc2.NextValue();
        Console.WriteLine("Then InvalidOperationException will not be thrown");
        action2.ShouldNotThrow();
        Console.WriteLine("And the counter value will be returned");
        result.HasValue.Should().BeTrue();
    }
}

Answer 3

Just out of curiousity, what do you have set for properties in Visual Studio? In VS go to Project Properties, Build, Platform target and change it to AnyCPU. I have seen it before where Performance Counters aren't always retrieved when it is set to x86, and changing it to AnyCPU could fix it.

Answer 4

Antenka has led you in a good direction here. You should not be disposing and re-creating the performance counter on every update/request for value. There is a cost for instantiating the performance counters and the first read can be inaccurate as indicated in the quote below. Also your lock() { ... } statements are very broad (they cover a lot of statements) and will be slow. Its better to have your locks as small as possible. I'm giving Antenka a voteup for the quality reference and good advice!

However, I think I can provide a better answer for you. I have a fair bit of experience with monitoring server performance and understand exactly what you need. One problem your code doesn't take into account is that whatever code is displaying your performance counter (.aspx, .asmx, console app, winform app, etc) could be requesting this statistic at any rate; it could be requested once every 10 seconds, maybe 5 times per second, you don't know and shouldn't care. So you need to separate the PerformanceCounter collection code from that does the monitoring from the code that actually reports the current Requests / Second value. And for performance reasons, I'm also going to show you how to setup the performance counter on first request and then keep it going until nobody has made any requests for 5 seconds, then close/dispose the PerformanceCounter properly.

public class RequestsPerSecondCollector
{
    #region General Declaration
    //Static Stuff for the polling timer
    private static System.Threading.Timer pollingTimer;
    private static int stateCounter = 0;
    private static int lockTimerCounter = 0;

    //Instance Stuff for our performance counter
    private static System.Diagnostics.PerformanceCounter pcReqsPerSec;
    private readonly static object threadLock = new object();
    private static decimal CurrentRequestsPerSecondValue;
    private static int LastRequestTicks;
    #endregion

    #region Singleton Implementation
    /// <summary>
    /// Static members are 'eagerly initialized', that is, 
    /// immediately when class is loaded for the first time.
    /// .NET guarantees thread safety for static initialization.
    /// </summary>
    private static readonly RequestsPerSecondCollector _instance = new RequestsPerSecondCollector();
    #endregion

    #region Constructor/Finalizer
    /// <summary>
    /// Private constructor for static singleton instance construction, you won't be able to instantiate this class outside of itself.
    /// </summary>
    private RequestsPerSecondCollector()
    {
        LastRequestTicks = System.Environment.TickCount;

        // Start things up by making the first request.
        GetRequestsPerSecond();
    }
    #endregion

    #region Getter for current requests per second measure
    public static decimal GetRequestsPerSecond()
    {
        if (pollingTimer == null)
        {
            Console.WriteLine("Starting Poll Timer");

            // Let's check the performance counter every 1 second, and don't do the first time until after 1 second.
            pollingTimer = new System.Threading.Timer(OnTimerCallback, null, 1000, 1000);

            // The first read from a performance counter is notoriously inaccurate, so 
            OnTimerCallback(null);
        }

        LastRequestTicks = System.Environment.TickCount;
        lock (threadLock)
        {
            return CurrentRequestsPerSecondValue;
        }
    }
    #endregion

    #region Polling Timer
    static void OnTimerCallback(object state)
    {
        if (System.Threading.Interlocked.CompareExchange(ref lockTimerCounter, 1, 0) == 0)
        {
            if (pcReqsPerSec == null)
                pcReqsPerSec = new System.Diagnostics.PerformanceCounter("W3SVC_W3WP", "Requests / Sec", "_Total", true);

            if (pcReqsPerSec != null)
            {
                try
                {
                    lock (threadLock)
                    {
                        CurrentRequestsPerSecondValue = Convert.ToDecimal(pcReqsPerSec.NextValue().ToString("N2"));
                    }
                }
                catch (Exception) {
                    // We had problem, just get rid of the performance counter and we'll rebuild it next revision
                    if (pcReqsPerSec != null)
                    {
                        pcReqsPerSec.Close();
                        pcReqsPerSec.Dispose();
                        pcReqsPerSec = null;
                    }
                }
            }

            stateCounter++;

            //Check every 5 seconds or so if anybody is still monitoring the server PerformanceCounter, if not shut down our PerformanceCounter
            if (stateCounter % 5 == 0)
            {
                if (System.Environment.TickCount - LastRequestTicks > 5000)
                {
                    Console.WriteLine("Stopping Poll Timer");

                    pollingTimer.Dispose();
                    pollingTimer = null;

                    if (pcReqsPerSec != null)
                    {
                        pcReqsPerSec.Close();
                        pcReqsPerSec.Dispose();
                        pcReqsPerSec = null;
                    }
                }                                                      
            }

            System.Threading.Interlocked.Add(ref lockTimerCounter, -1);
        }
    }
    #endregion
}

Ok now for some explanation.

1. First you'll notice this class is designed to be a static singleton. You can't load multiple copies of it, it has a private constructor and and eagerly initialized internal instance of itself. This makes sure you don't accidentally create multiple copies of the same PerformanceCounter.
2. Next you'll notice in the private constructor (this will only run once when the class is first accessed) we create both the PerformanceCounter and a timer which will be used to poll the PerformanceCounter.
3. The Timer's callback method will create the PerformanceCounter if needed and get its next value is available. Also every 5 iterations we're going to see how long its been since your last request for the PerformanceCounter's value. If it's been more than 5 seconds, we'll shutdown the polling timer as its unneeded at the moment. We can always start it up again later if we need it again.
4. Now we have a static method called GetRequestsPerSecond() for you to call which will return the current value of the RequestsPerSecond PerformanceCounter.

The benefits of this implementation are that you only create the performance counter once and then keep using until you are finished with it. Its easy to use because you simple call RequestsPerSecondCollector.GetRequestsPerSecond() from wherever you need it (.aspx, .asmx, console app, winforms app, etc). There will always be only one PerformanceCounter and it will always be polled at exactly 1 times per second regardless of how quickly you call RequestsPerSecondCollector.GetRequestsPerSecond(). It will also automatically close and dispose of the PerformanceCounter if you haven't requested its value in more than 5 seconds. Of course you can adjust both the timer interval and the timeout milliseconds to suit your needs. You could poll faster and timeout in say 60 seconds instead of 5. I chose 5 seconds as it proves that it works very quickly while debugging in visual studio. Once you test it and know it works, you might want a longer timeout.

Hopefully this helps you not only better use PerformanceCounters, but also feel safe to reuse this class which is separate from whatever you want to display the statistics in. Reusable code is always a plus!

EDIT: As a follow up question, what if you want to performance some cleanup or babysitting task every 60 seconds while this performance counter is running? Well we already have the timer running every 1 second and a variable tracking our loop iterations called stateCounter which is incremented on each timer callback. So you could add in some code like this:

// Every 60 seconds I want to close/dispose my PerformanceCounter
if (stateCounter % 60 == 0)
{
    if (pcReqsPerSec != null)
    {
        pcReqsPerSec.Close();
        pcReqsPerSec.Dispose();
        pcReqsPerSec = null;
    }
}

I should point out that this performance counter in the example should not "go stale". I believe 'Request / Sec" should be an average and not a moving average statistic. But this sample just illustrates a way you could do any type of cleanup or "babysitting" of your PerformanceCounter on a regular time interval. In this case we are closing and disposing the performance counter which will cause it to be recreated on next timer callback. You could modify this for your use case and according the specific PerformanceCounter you are using. Most people reading this question/answer should not need to do this. Check the documentation for your desired PerformanceCounter to see if it is a continuous count, an average, a moving average, etc... and adjust your implementation appropriately.

Answer 5

I don't know, if this passes you .. I've read article PerformanceCounter.NextValue Method

And there was a comment:

// If the category does not exist, create the category and exit.
// Performance counters should not be created and immediately used.
// There is a latency time to enable the counters, they should be created
// prior to executing the application that uses the counters.
// Execute this sample a second time to use the category.

So, I have a question, which can lead to answer: isn't call to a RequestsPerSecond method happends too early? Also, I would suggest you to to try check if the Category doesn't exists and log the info somewhere, so we can analyze it and determine which conditions we have and how often that happends.