Python Flask date update real-time

Question

I am building a web app with Python Flask with JavaScript. I am a beginner of Javascript.

The process I do now:

In Flask Python code,
1. I get data by scrapp

Answer 1

This is a classic "push" update problem. Your web server is updated with information that you would like to frequently update (or "push") to your web client front-end.

As PJ Santoro suggests, you can repeatedly poll your server (every 10 seconds say) along the lines of "Is there new data? No. Okay. Is there new data? No. Okay. Is there new data? Yes, here it is! Great! Is there new data? ...." It is rather inefficient, but for low data volumes you might never see that problem.

A more efficient update system would have the server send an update signal only when new data is ready. There are two major ways to do that:

1. Long polling, also known as reverse AJAX and Comet. Your web page opens an AJAX request on a server URL such as /update_data with a really long timeout (say 30 minutes or an hour). When the server has data, it sends it through the waiting AJAX connection. There are complications with this, including managing concurrency on the server side (not historically Python's strong suit) and the fact that even the maximum timeout (~1 hour last I used this technique) might expire without data, and some error handling is needed to manage that. If you wan to try long polling / reverse AJAX, here is an example using Flask.

2. WebSockets. WebSockets are a much more recent approach to interactive and "push" updates between servers and clients. They're very well-used; projects like the Jupyter Notebook depend on them extensively. They're extremely efficient and well-fit for this kind of incremental update of client state, and the code is often much less Byzantine and confusing than retrofits like reverse AJAX. But...there are complexities with WebSockets too. Managing concurrency in the Flask server side is still a significant issue, for example. If you'd like to consider WebSockets as your update mechanism, here is an example of how to use them with Flask.

Either approach you use, if your data communicated over time grows, you will also need to structure the data transmitted on each update so that it's an incremental update. No mater how good the plumbing, if you transmit thousands of data points every update, it's not going to be fast. But, long polling and WebSockets plumbing should at least get you a great distance toward a legitimate real-time update capability.