Python Flask date update real-time

Question

I am building a web app with Python Flask with JavaScript. I am a beginner of Javascript.

The process I do now:

In Flask Python code,
1. I get data by scrapp

Answer 1

Well... in an ideal world you would separate this into 2 applications:

• Scrape & Parse Data From the Web (doesn't need to be exposed to the web)

• Deliver data to a user via a web app

You could use some sort of CI tool (e.g. Jenkins) to monitor and add the external data into a database and then your Flask app to serve this pre-processed data to your users.

If Steps 1-6 are relatively quick what you could do is setup an XHR endpoint within Flask and then use a setInterval() javascript function to call it on a interval to tell your application to update the data. E.g.:

setInterval(function() {
    var req = new XMLHttpRequest();
    req.open('GET', '/refresh_data?time=234', true);
    req.onreadystatechange = function(e) {
        if(req.readyState !== 4) {
            return;
        }
        if ([200, 304].indexOf(req.status) === -1) {
            console.warn('Error! XHR failed.');
        }
        else {
            data = JSON.parse(e.target.responseText);
            processData();
            updateChart();
        }
    };
    req.send();
}, 10000);  // time in milliseconds (e.g. every 10 seconds)

And have a flask endpoint like:

@app.route('/refresh_data')
def refresh_data():
    time = requests.args.get('time', type=int)
    if not time:
        return jsonify({status: 'error: incorrect parameters'})
    data = requests.get('http://api.getmoredata.com?time=%s' % time)
    # process the results...
    return jsonify({status: 'success', data: data})

Ideally you'd have some sort of a chart that has a refresh() method where you can pass it new data and just keep adding to it... (e.g. I'm a fan of D3 for stuff like this).

I'd probably remove this from you're init.py code, but it might be acceptable for the "first load". a flask app (e.g. any web app) responds to HTTP requests and nothing persists on the server between requests so a time.sleep() would not do much for you... if you want to run persistent code on your server you'd need to look into something like celery to manage background tasks.