I have large data sets (10 Hz data, so 864k points per 24 Hours) which I need to plot in real time. The idea is the user can zoom and pan into highly detailed scatter plots.
I don't know how Matlab does it, but I'd start with Quadtrees.
Dump all your data points into the quadtree, then to render at a given zoom level, you walk down the quadtree (starting with the areas that overlap what you're viewing) until you reach areas which are comparable to the size of a pixel. Stick a pixel in the middle of that area.
added: Doing your drawing with OpenGL/JOGL will also help you get faster drawing. Especially if you can predict panning, and build up the points to show in a display list or something, so that you don't have to do any CPU work for the new frames.