Java tcp can only retrieve image once

Question

I am making a program that captures a screenshot from a tcp server. It works but after one screenshot i get this error: java.lang.IllegalArgumentException: image == nu

Answer 1

Normally your server would accept an incoming connection and spawn a new thread to handle that new socket connection.

That thread would then continue in a loop (reading/writing) until the client disconnected.

Now you could open a new connection, download the image and close the connection each time the user clicks the "grab" button, but the establishment of the connection can be a time consuming process.

So. The basic idea is, when the user clicks the "grab" button, the client will send a request to the server to "grab" a screen shot. The server will generate the screen shot and write it back to the client. The client will then read the screen shot and display it...simple...

There is just one little problem. ImageIO doesn't behave quite the way you (and I) think it should. You can't simply write the image out to the socket's OutputStream and read it using the socket's InputStream. It seems the ImageIO needs the stream to be closed so it can be "finalised" ... or something.

Instead, I had to write it to a ByteArrayOutputStream, then write the resulting byte array to the socket's OutputStream. Conversely, I had to read the byte array into a ByteArrayOutputStream and the dump it into a ByteArrayInputStream to be read by ImageIO...fun stuff ;)

Server

import java.awt.AWTException;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.text.NumberFormat;
import java.util.Iterator;
import javax.imageio.ImageIO;
import javax.imageio.ImageWriter;
import javax.imageio.event.IIOWriteProgressListener;

public class Server {

    public static void main(String args[]) {
        try {
            ServerSocket welcomeSocket = new ServerSocket(6789);

            while (true) {
                System.out.println("Get next client...");
                Socket skt = welcomeSocket.accept();
                // Hand of the processing to the socket handler...
                new Thread(new SocketHandler(skt)).start();
            }
        } catch (IOException ex) {
        }
    }

    // Reads a request from the client
    // All requests must be terminated with a new line (\n)
    protected static String readRequest(InputStream is) throws IOException {
        StringBuilder sb = new StringBuilder(128);
        int in = -1;
        while ((in = is.read()) != '\n') {
            sb.append((char) in);
        }
        return sb.toString();
    }

    // Grabs the screen shot and writes to the supplied output stream
    // This will first write the byte size of the following byte array and
    // writes the byte array of the image.  Clients should expect a 
    // int value terminated by a new line character (\n)
    protected static void grabScreen(OutputStream os) throws AWTException, IOException {
        System.out.println("Grab screen shot");
        Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
        BufferedImage capture = new Robot().createScreenCapture(screenRect);

        System.out.println("Writing image to buffer...");
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ImageIO.write(capture, "jpg", baos);
        baos.close();
        System.out.println("Write byte size = " + baos.size());
        os.write((Integer.toString(baos.size()) + "\n").getBytes());
        System.out.println("Write byte stream");
        os.write(baos.toByteArray());
        System.out.println("Image sent");
    }

    // Handler for an individual client socket...
    public static class SocketHandler implements Runnable {

        private Socket socket;

        public SocketHandler(Socket socket) {
            this.socket = socket;
        }

        @Override
        public void run() {
            String request = null;
            InputStream is = null;
            OutputStream os = null;
            try {
                System.out.println("Processing client requests");
                is = socket.getInputStream();
                os = socket.getOutputStream();
                do {
                    System.out.println("Waiting for next request");
                    request = readRequest(is);
                    System.out.println("Request = " + request);
                    if ("grab".equalsIgnoreCase(request)) {
                        grabScreen(os);
                    }
                } while (!"done".equalsIgnoreCase(request) && !"shutdown".equalsIgnoreCase(request));
                System.out.println("Client has closed");
            } catch (IOException | AWTException exp) {
                exp.printStackTrace();
            } finally {
                try {
                    socket.close();
                } catch (Exception e) {
                }
            }
            // Special command to stop the server...
            if ("shutdown".equalsIgnoreCase(request)) {
                System.exit(0);
            }
        }
    }
}

Client

import java.awt.BorderLayout;
import java.awt.Dimension;
import java.awt.EventQueue;
import java.awt.Image;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.text.NumberFormat;
import java.util.Iterator;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.imageio.ImageIO;
import javax.imageio.ImageReader;
import javax.imageio.event.IIOReadProgressListener;
import javax.imageio.stream.ImageInputStream;
import javax.swing.ImageIcon;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.UIManager;
import javax.swing.UnsupportedLookAndFeelException;

public class Client {

    public static void main(String[] args) {
        new Client();
    }

    public Client() {
        EventQueue.invokeLater(new Runnable() {
            @Override
            public void run() {
                try {
                    UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
                } catch (ClassNotFoundException | InstantiationException | IllegalAccessException | UnsupportedLookAndFeelException ex) {
                }

                final CapturePane capturePane = new CapturePane();

                JFrame frame = new JFrame("Testing");
                frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                frame.add(capturePane);
                frame.pack();
                frame.setLocationRelativeTo(null);
                frame.addWindowListener(new WindowAdapter() {
                    @Override
                    public void windowClosing(WindowEvent e) {
                        try {
                            capturePane.close();
                        } catch (IOException ex) {
                            ex.printStackTrace();
                        }
                    }
                });
                frame.setVisible(true);
            }
        });
    }

    public class CapturePane extends JPanel {

        private Socket socket;
        private ScreenPane screenPane;
        private JButton grabButton;

        public CapturePane() {
            setLayout(new BorderLayout());
            screenPane = new ScreenPane();
            grabButton = new JButton("Grab");
            try {
                socket = new Socket("localhost", 6789);
            } catch (IOException ex) {
                grabButton.setEnabled(false);
                ex.printStackTrace();
            }
            add(screenPane);
            add(grabButton, BorderLayout.SOUTH);

            grabButton.addActionListener(new ActionListener() {
                @Override
                public void actionPerformed(ActionEvent e) {
                    if (socket != null) {
                        InputStream is = null;
                        OutputStream os = null;

                        ByteArrayOutputStream baos = null;
                        ByteArrayInputStream bais = null;

                        try {
                            is = socket.getInputStream();
                            os = socket.getOutputStream();
                            // Send the "grab" request...
                            writeRequest(os, "grab");
                            System.out.println("Reading image...");
                            // Read back the expected byte size of the image
                            String size = readResponse(is);

                            int expectedByteCount = Integer.parseInt(size);
                            System.out.println("Expecting " + expectedByteCount);
                            // Create a buffer for the image bytes...
                            baos = new ByteArrayOutputStream(expectedByteCount);
                            byte[] buffer = new byte[1024];
                            int bytesRead = 0;
                            int bytesIn = 0;
                            // Read the image from the server...
                            while (bytesRead < expectedByteCount) {
                                bytesIn = is.read(buffer);
                                bytesRead += bytesIn;
                                baos.write(buffer, 0, bytesIn);
                            }
                            System.out.println("Read " + bytesRead);
                            baos.close();
                            // Wrap the result in an InputStream
                            bais = new ByteArrayInputStream(baos.toByteArray());

                            // Read the image...
                            BufferedImage image = ImageIO.read(bais);
                            System.out.println("Got image...");
                            screenPane.setImage(image);
                            bais.close();
                        } catch (IOException exp) {
                            exp.printStackTrace();
                        } finally {
                            try {
                                bais.close();
                            } catch (Exception exp) {
                            }
                            try {
                                baos.close();
                            } catch (Exception exp) {
                            }
                        }
                    }
                }

                protected String readResponse(InputStream is) throws IOException {
                    StringBuilder sb = new StringBuilder(128);
                    int in = -1;
                    while ((in = is.read()) != '\n') {
                        sb.append((char) in);
                    }
                    return sb.toString();
                }

            });
        }

        protected void writeRequest(OutputStream os, String request) throws IOException {
            os.write((request + "\n").getBytes());
            os.flush();
        }

        public void close() throws IOException {
            try {
                try {
                    System.out.println("Write done...");
                    writeRequest(socket.getOutputStream(), "shutdown");
                } finally {
                    try {
                        System.out.println("Close outputstream");
                        socket.getOutputStream().close();
                    } finally {
                        try {
                            System.out.println("Close inputStream");
                            socket.getInputStream().close();
                        } finally {
                            System.out.println("Close socket");
                            socket.close();
                        }
                    }
                }
            } finally {
                socket = null;
            }
        }

    }

    public class ScreenPane extends JPanel {

        private JLabel background;

        public ScreenPane() {
            setLayout(new BorderLayout());
            background = new JLabel();
            add(new JScrollPane(background));
        }

        @Override
        public Dimension getPreferredSize() {
            return new Dimension(400, 400);
        }

        public void setImage(BufferedImage img) {
            if (img != null) {
                ImageIcon icon = null;
                if (getWidth() > getHeight()) {
                    icon = new ImageIcon(img.getScaledInstance(getWidth(), -1, Image.SCALE_SMOOTH));
                } else {
                    icon = new ImageIcon(img.getScaledInstance(-1, getHeight(), Image.SCALE_SMOOTH));
                }
                background.setIcon(icon);
            } else {
                background.setIcon(null);
            }
            repaint();
        }

    }

}

Problems, issues and topics not convered

1. The current implementation will only capture the default monitor. If you have more than one screen, this will not capture it. There's a choice here. Capture the entire virtual desktop or allow the client to specify the screen to capture...
2. Requesting the screen from the server within the Event Dispatching Thread is going to cause the client to "pause". This might not be an issue if running locally, but is going to be a massive issue if your trying to do this remotely. A better solution would be to grab and load the image in a background process. SwingWorker is perfect for this. Check out Concurrency in Swing for more details