How to download a file in Lua, but write to a local file as it works

Question

I'm trying to make an updater so when my Lua application is out-of-date it will use LuaSocket to download the newer .exe file (which can run my Lua code).

Inside this updater, I want it to show how much has been downloaded so far. However, with the following HTTP request, it blocks the application until downloaded fully:

local b, c, h = http.request("https://www.example.com/Download/Example.exe?from="..Game.Version)

I am using threads to download it, however I still cannot write to the file until the download is complete inside of the thread, thus the progress bar will go 0%, 100%, without anything in between.

Is there anything I can do to download a remote file, but save it to a local file as it downloads?

cURL can do this. I don't know if LuaSocket or anything else for Lua can. :(

Answer 1

You are right - cURL can do it. LuaSocket does not have this functionality. You can create a LTN12 sink which will report progress made, but you won't know the total size of the file until you have downloaded it completely, so it is kind of useless. Why not use luacurl instead?

local curl = require "luacurl"
local c = curl.new()

function GET(url)
    c:setopt(curl.OPT_URL, url)
    local t = {} -- this will collect resulting chunks
    c:setopt(curl.OPT_WRITEFUNCTION, function (param, buf)
        table.insert(t, buf) -- store a chunk of data received
        return #buf
    end)
    c:setopt(curl.OPT_PROGRESSFUNCTION, function(param, dltotal, dlnow)
        print('%', url, dltotal, dlnow) -- do your fancy reporting here
    end)
    c:setopt(curl.OPT_NOPROGRESS, false) -- use this to activate progress
    assert(c:perform())
    return table.concat(t) -- return the whole data as a string
end

local s = GET 'http://www.lua.org/'
print(s)

Answer 2

You can save yourself the dependency on cURL by making a HEAD request and getting the filesize from the Content-Length header:

require "socket.http"

local resp, stat, hdr = socket.http.request{
  url     = "http://www.lua.org/ftp/lua-5.2.1.tar.gz",
  method  = "HEAD",
}

print(hdr["content-length"])
-- 249882

That said, if this is all you'd be using LuaSocket for, then cURL is arguably a better choice.