How to improve the behavior of ConnectionRequest?

问题

I wrote this code:

public static Slider downloadStorageFile(String url, OnComplete<Integer> percentageCallback, OnComplete<String> filesavedCallback) {

        final String filename = getNewStorageFilename();

        ConnectionRequest cr = Rest.get(url).fetchAsBytes(response -> CN.callSerially(() -> filesavedCallback.completed(filename)));
        cr.setDestinationStorage(filename);

        Slider slider = new Slider();
        slider.addDataChangedListener((int type, int index) -> {
            CN.callSerially(() -> percentageCallback.completed(slider.getProgress()));
        });
        sliderBridgeMap.put(cr, slider);
        SliderBridge.bindProgress(cr, slider);

        return slider;
    }

Basically, as you can guess, it asynchronously downloads a file, it offers two callbacks that will run on the EDT and it immediately returns a Slider that keep track of the download progress.

On Android, the download continues and ends even when the app is in background. On iOS, on the other hand, the app must remain active and foregrounded at all time.

Is it possible to complete the download even on iOS when the app goes to the background?
Alternatively, can I get ConnectionRequest.retry(), which is automatically invoked by my network error handler when the app returns to foreground on iOS, to restart where the download arrived at, rather than restart it from scratch?
Even better, can I get both points 1 and 2?

回答1:

This is problematic on Android too as the OS may suddenly kill your ongoing download when the system is constrained. E.g. in the developer tools just turn on the kill activities immediately and see your download die the moment you minimize the app.

Most devices don't do that but if a device is running in battery saving mode that might happen.

The solution is to use background fetch when going into the background. The problem is that this doesn't provide the nice sort of UI you would get with the regular foreground download so you need to pick and choose which one to use.

See the JavaDoc for the class here: https://www.codenameone.com/javadoc/com/codename1/background/BackgroundFetch.html

And the slightly out of date blog post on the subject: https://www.codenameone.com/blog/background-fetch.html

回答2:

I would like to design a system to download small and large files that is very robust, i.e. resistant to network errors and capable of resume the download as soon as network conditions allow and in a completely transparent way for the user.

However, this conflicts with certain limits imposed by a cross-platform approach. I'm not sure that background fetch is the most appropriate solution to download heavy multimedia content and I don't know if the network errors in the background fetch are captured by the general error handler. Maybe I'll look into it.

I've elaborated a solution that has pros and cons, and it circumvents the problem.

Pros: always allows you to complete downloads, even if very heavy (e.g. 100MB), even if the connection is unstable (network errors) and even if the app goes temporarily in the background

Cons: since my idea is based on splitting the download into small parts, this approach causes many GET requests that slightly slow down the download and cause more traffic than normally needed.

Prerequisite 1: in global network error handling, there must be an automatic .retry() as in this code: Distinguish between server-side errors and connection problems

Prerequisite 2: for the getFileSizeWithoutDownload(String url) implementation and the Wrapper implementation, see: https://stackoverflow.com/a/62130371/1277576

Explanation: the code should be self-explanatory. Basically it downloads 512kbytes at a time and then merges it with the output. If a network error happens (and if on iOS the app goes in the background) everything that has already been downloaded is not lost (at most only the last 512kbyte fragment is lost). As each fragment is downloaded, ConnectionRequest calls itself, changing the header for partial download. The filesavedCallback callback is only called when all the download is finished.

Code:

    public static void downloadToStorage(String url, OnComplete<Integer> percentageCallback, OnComplete<String> filesavedCallback) throws IOException {

        final String output = getNewStorageFilename(); // get a new random available Storage file name
        final long fileSize = getFileSizeWithoutDownload(url); // total expected download size
        final int splittingSize = 512 * 1024; // 512 kbyte, size of each small download
        Wrapper<Integer> downloadedTotalBytes = new Wrapper<>(0);
        OutputStream out = Storage.getInstance().createOutputStream(output); // leave it open to append partial downloads
        Wrapper<Integer> completedPartialDownload = new Wrapper<>(0);

        ConnectionRequest cr = new GZConnectionRequest();
        cr.setUrl(url);
        cr.setPost(false);
        if (fileSize > splittingSize) {
            // Which byte should the download start from?
            cr.addRequestHeader("Range", "bytes=0-"  + splittingSize);
            cr.setDestinationStorage("split-" + output);
        } else {
            Util.cleanup(out);
            cr.setDestinationStorage(output);
        }
        cr.addResponseListener(a -> {
            CN.callSerially(() -> {
                try {
                    // We append the just saved partial download to the output, if it exists
                    if (Storage.getInstance().exists("split-" + output)) {
                        InputStream in = Storage.getInstance().createInputStream("split-" + output);
                        Util.copyNoClose(in, out, 8192);
                        Util.cleanup(in);
                        Storage.getInstance().deleteStorageFile("split-" + output);
                        completedPartialDownload.set(completedPartialDownload.get() + 1);
                    }
                    // Is the download finished?
                    if (fileSize <= 0 || completedPartialDownload.get() * splittingSize >= fileSize || downloadedTotalBytes.get() >= fileSize) {
                        // yes, download finished
                        Util.cleanup(out);
                        filesavedCallback.completed(output);
                    } else {
                        // no, it's not finished, we repeat the request after updating the "Range" header
                        cr.addRequestHeader("Range", "bytes=" + downloadedTotalBytes.get() + "-" + (downloadedTotalBytes.get() + splittingSize));
                        NetworkManager.getInstance().addToQueue(cr);
                    }
                } catch (IOException ex) {
                    Log.p("Error in appending splitted file to output file", Log.ERROR);
                    Log.e(ex);
                    Server.sendLogAsync();
                }
            });
        });
        NetworkManager.getInstance().addToQueue(cr);
        NetworkManager.getInstance().addProgressListener((NetworkEvent evt) -> {
            if (cr == evt.getConnectionRequest() && fileSize > 0) {
                downloadedTotalBytes.set(completedPartialDownload.get() * splittingSize + evt.getSentReceived());
                // the following casting to long is necessary when the file is bigger than 21MB, otherwise the result of the calculation is wrong
                percentageCallback.completed((int) ((long) downloadedTotalBytes.get() * 100 / fileSize));
            }
        });
    }

I tried this solution in the Simulator, on Android and iOS, in different network conditions, with a 100MB download and occasionally moving the app in the background (or letting it go automatically). In all cases the app succeeds in completing the download. However, the differences between Android and iOS remain when the app is in the background.

I hope this is useful. If someone wants to further improve this code, they can add another answer :)

来源：https://stackoverflow.com/questions/62093943/how-to-improve-the-behavior-of-connectionrequest

标签

codenameone