When a console application is started from another console application, how does console ownership work?
I see four possibilities:
I think it's spelled out fairly well in the documentation.
My guess is somewhere between 3 and 4. The console is a self-standing object, which has standard input, output and error streams. These streams are attached to the first process that uses the console. Subsequent processes can also inherit these streams if not redirected (e.g. running a command with redirect to a file.)
Normally there is no contention, since parent processes usually wait for their child process to complete, and asynchronous processes typically start their own console (e.g. try "start cmd" in a command prompt) or redirect standard output.
However, there is nothing to stop both processes writing to the output stream at the same time - the streams are shared. This can be a problem when using some runtime libraries since writes to standard output/error may not be immediately flushed, leading to mixed garbled output. In general, having to processes actively writing to the same output stream is usually not a good idea, unless you take measures to co-ordinate their output through concurrency primitives like Mutexes, Events and the like.
CMD 'owns' the console. When it creates a process for an app, that app inherits handles to the console. It can read and write those. When the process goes away, CMD continues ownership.
Note: I'm not entirely sure that 'ownership' is the right word here. Windows will close the Console when CMD exits, but that may be a simple setting.
None of your four possibilities is actually the case, and the answer to your follow-on question, "Which of the two processes is responsible for managing the window?", is that neither process is responsible. TUI programs don't have to know anything about windows at all, and, under the covers, aren't necessarily even plumbed in to the GUI.
Consoles are objects, accessed via handles just like files, directories, pipes, processes, and threads. A single process doesn't "own" a console via its handle to it any more than a process "owns" any file that it has an open handle to. Handles to consoles are inherited by child processes from their parents in the same way that all other (inheritable) handles are. Your TUI application, spawned by CMD, simply inherits the standard handles that CMD said that it should inherit, when it called CreateProcess()
— which are usually going to be CMD's standard input, output, and error (unless the command-line told CMD to use some other handles as the child's standard input, output, and error).
Consoles aren't dependent upon CMD. They exist as long as there are (a) any open handles to the console's input or output buffers or (b) any processes otherwise "attached" to the console. So in your example you could kill CMD, but only when you terminated the child process too would the console actually be destroyed.
The process that is in charge of displaying the GUI windows in which consoles are presented is, in Windows NT prior to version 6.1, CSRSS, the Client-Server Runtime SubSystem. The window handling code is in WINSRV.DLL, which contains the "console server" that — under the covers — Win32 programs performing console I/O make LPC calls to. In Windows NT 6.1, this functionality, for reasons covered by Raymond Chen, moved out of CSRSS into a less-privileged process that CSRSS spawns.
The way the SDK talks about it strongly resembles 1. It is an option with CreateProcess, described as follows:
CREATE_NEW_CONSOLE
The new process has a new console, instead of inheriting its parent's console (the default). For more information, see Creation of a Console.
Output however happens through handles, you'd get one with GetStdHandle(). Passing STD_OUTPUT_HANDLE returns the console handle, assuming output isn't redirected. Actual output is done through WriteFile() or WriteConsole/Output(). If both processes keep writing output to the handle then their output will be randomly intermingled. This is otherwise indistinguishable from what would happen when two programs write to the same file handle.
Logically, there's a screen buffer associated with a console. You can tinker with it with SetConsoleScreenBufferXxx(). From that point of view you could call it shared memory. The actual implementation is undiscoverable, handles abstract them away, like any Win32 API. It is sure to have changed considerably in Vista with the new conhost.exe process.
Each application will run in it's own AppDomain. Each AppDomain should be running it's own console.
Ah, you're right. I was thinking about running executables within a process and forgot they start their own process - I didn't drill down far enough.