Console.WriteLine slow

Question

I run through millions of records and sometimes I have to debug using Console.WriteLine to see what is going on.

However, Console.WriteLine

Answer 1

I recently did a benchmark battery for this on .NET 4.8. The tests included many of the proposals mentioned on this page, including Async and blocking variants of both BCL and custom code, and then most of those both with and without dedicated threading, and finally scaled across power-of-2 buffer sizes.

The fastest method, now used in my own projects, buffers 64K of wide (Unicode) characters at a time from .NET directly to the Win32 function WriteConsoleW without copying or even hard-pinning. Remainders larger than 64K, after filling and flushing one buffer, are also sent directly, and in-situ as well. The approach deliberately bypasses the Stream/TextWriter paradigm so it can (obviously enough) provide .NET text that is already Unicode to a (native) Unicode API without all the superfluous memory copying/shuffling and byte[] array allocations required for first "decoding" to a byte stream.

If there is interest (perhaps because the buffering logic is slightly intricate), I can provide the source for the above; it's only about 80 lines. However, my tests determined that there's a simpler way to get nearly the same performance, and since it doesn't require any Win32 calls, I'll show this latter technique instead.

The following is way faster than Console.Write:

public static class FastConsole
{
    static readonly BufferedStream str;

    static FastConsole()
    {
        Console.OutputEncoding = Encoding.Unicode;  // crucial

        // avoid special "ShadowBuffer" for hard-coded size 0x14000 in 'BufferedStream' 
        str = new BufferedStream(Console.OpenStandardOutput(), 0x15000);
    }

    public static void WriteLine(String s) => Write(s + "\r\n");

    public static void Write(String s)
    {
        // avoid endless 'GetByteCount' dithering in 'Encoding.Unicode.GetBytes(s)'
        var rgb = new byte[s.Length << 1];
        Encoding.Unicode.GetBytes(s, 0, s.Length, rgb, 0);

        lock (str)   // (optional, can omit if appropriate)
            str.Write(rgb, 0, rgb.Length);
    }

    public static void Flush() { lock (str) str.Flush(); }
};

Note that this is a buffered writer, so you must call Flush() when you have no more text to write.

I should also mention that, as shown, technically this code assumes 16-bit Unicode (UCS-2, as opposed to UTF-16) and thus won't properly handle 4-byte escape surrogates for characters beyond the Basic Multilingual Plane. The point hardly seems important given the more extreme limitations on console text display in general, but could perhaps still matter for piping/redirection.

Usage:

FastConsole.WriteLine("hello world.");
// etc...
FastConsole.Flush();

On my machine, this gets about 77,000 lines/second (mixed-length) versus only 5,200 lines/sec under identical conditions for normal Console.WriteLine. That's a factor of almost 15x speedup.

These are controlled comparison results only; note that absolute measurements of console output performance are highly variable, depending on the console window settings and runtime conditions, including size, layout, fonts, DWM clipping, etc.

Answer 2

Try using the System.Diagnostics Debug class? You can accomplish the same things as using Console.WriteLine.

You can view the available class methods here.

Answer 3

Just a little trick I use sometimes: If you remove focus from the Console window by opening another window over it, and leave it until it completes, it won't redraw the window until you refocus, speeding it up significantly. Just make sure you have the buffer set up high enough that you can scroll back through all of the output.

Answer 4

You can use the OutputDebugString API function to send a string to the debugger. It doesn't wait for anything to redraw and this is probably the fastest thing you can get without digging into the low-level stuff too much. The text you give to this function will go into Visual Studio Output window.

[DllImport("kernel32.dll")]
static extern void OutputDebugString(string lpOutputString);

Then you just call OutputDebugString("Hello world!");

Answer 5

Do something like this:

public static class QueuedConsole
{
    private static StringBuilder _sb = new StringBuilder();
    private static int _lineCount;

    public void WriteLine(string message)
    {
        _sb.AppendLine(message);
        ++_lineCount;
        if (_lineCount >= 10)
           WriteAll();
    }

    public void WriteAll()
    {
        Console.WriteLine(_sb.ToString());
        _lineCount = 0;
        _sb.Clear();
    }
}

QueuedConsole.WriteLine("This message will not be written directly, but with nine other entries to increase performance.");

//after your operations, end with write all to get the last lines.
QueuedConsole.WriteAll();

Here is another example: Does Console.WriteLine block?

Answer 6

Why Console is slow:

• Console output is actually an IO stream that's managed by your operating system. Most IO classes (like FileStream) have async methods but the Console class was never updated so it always blocks the thread when writing.

• Console.WriteLine is backed by SyncTextWriter which uses a global lock to prevent multiple threads from writing partial lines. This is a major bottleneck that forces all threads to wait for each other to finish the write.

• If the console window is visible on screen then there can be significant slowdown because the window needs to be redrawn before the console output is considered flushed.

Solutions:

Wrap the Console stream with a StreamWriter and then use async methods:

var sw = new StreamWriter(Console.OpenStandardOutput());
await sw.WriteLineAsync("...");

You can also set a larger buffer if you need to use sync methods. The call will occasionally block when the buffer gets full and is flushed to the stream.

var sw = new StreamWriter(Console.OpenStandardOutput(), Encoding.UTF8, 8192);
sw.Write("...") // this will block for flushing when the buffer size of 8192 is full

If you want the fastest writes though, you'll need to make your own buffer class that writes to memory and flushes to the console asynchronously in the background using a single thread without locking. The new Channel<T> class in .NET Core 2.1 makes this simple and fast. Plenty of other questions showing that code but comment if you need tips.