And if so why? What does X do for me beyond piping my rendering commands to the graphics card driver?
I'm not clear on the relationship X - OpenGL. I've searched the internet but couldn't find a concise answer.
If it matters, assuming a minimal modern distribution, like a headless Ubuntu 13 machine.
With the current drivers: Yes.
And if so why?
Because the X server is the host for the actual graphics driver talking to the GPU. At the moment Linux GPU drivers require a X server that gives them an environment to live in and a channel to the kernel interfaces to talk through with the GPU.
On the DRI/DRM/Gallium front a new driver model has been created that allows to use the GPU without an X server, for example using the EGL-API. However only a small range of GPUs is supported by this right now; most Intel and AMD; none NVidia.
I'm not clear on the relationship X - OpenGL
I covered that in detail in the SO answers found at https://stackoverflow.com/a/7967211/524368 and https://stackoverflow.com/a/8777891/524368
In short the X server acts like a "proxy" to the GPU. You send the X server commands like "open a window" or "draw a line there". And there's an extension to the X protocol called "GLX", where each OpenGL command gets translated into a stream of GLX/X opcodes and the X server executes those commands on the GPU on behalf of the calling client. Also most OpenGL/GLX implementations provide a mechanism to bypass the X server if the client process could actually talk directly to the GPU (because it runs on the same machine as the X server and has permissions to access the kernel API); that is called Direct Rendering. It however still requires the X server for opening the window, creating the context and to general housekeeping.
Update due to comment
Also if you can live without GPU acceleration, you can use Mesa3D using the osmesa (off-screen mesa) mode and the LLVMpipe software rasterizer.
With Linux 3.12: Not any more.
Offscreen rendering is what DRM render nodes are for, according to the commit. See the developer's blog for a better explanation.
TLDR:
A render node (/dev/dri/renderD<num>) appears as a GPU with no screens attached.
As for how exactly one is supposed to make use of this, the (kernel) developer only has very general advice for userspace infrastructure. Nevertheless, it is fair to assume the feature to be nothing short of a show-enabler for Wayland and Mir, as clients won't be able to render on-screen any more.
The wikipedia entry has some more pointers.
来源：https://stackoverflow.com/questions/19046855/in-linux-do-i-need-an-x-server-to-do-off-screen-rendering