Long-running / blocking operations in boost asio handlers

Question

Current Situation

I implemented a TCP server using boost.asio which currently uses a single io_service object on which I call the run method f

Answer 1

The approaches are not explicitly mutually exclusive. I often see a combination of the first and second:

• One or more thread are processing network I/O in one io_service.
• Long running or blocking tasks are posted into a different io_service. This io_service functions as a thread pool that will not interfere with threads handling network I/O. Alternatively, one could spawn a detached thread every time a long running or blocking task is needed; however, the overhead of thread creation/destruction may a noticeable impact.

This answer that provides a thread pool implementation. Additionally, here is a basic example that tries to emphasize the interaction between two io_services.

#include <iostream>

#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/chrono.hpp>
#include <boost/optional.hpp>
#include <boost/thread.hpp>

/// @brief Background service will function as a thread-pool where
///        long-standing blocking operations may occur without affecting
///        the network event loop.
boost::asio::io_service background_service;

/// @brief The main io_service will handle network operations.
boost::asio::io_service io_service;

boost::optional<boost::asio::io_service::work> work;

/// @brief ODBC blocking operation.
///
/// @brief data Data to use for query.
/// @brief handler Handler to invoke upon completion of operation.
template <typename Handler>
void query_odbc(unsigned int data,
                Handler handler)
{
  std::cout << "in background service, start querying odbc\n";
  std::cout.flush();
  // Mimic busy work.
  boost::this_thread::sleep_for(boost::chrono::seconds(5));

  std::cout << "in background service, posting odbc result to main service\n";
  std::cout.flush();
  io_service.post(boost::bind(handler, data * 2));
}

/// @brief Functions as a continuation for handle_read, that will be
///        invoked with results from ODBC.
void handle_read_odbc(unsigned int result)
{
  std::stringstream stream;
  stream << "in main service, got " << result << " from odbc.\n";
  std::cout << stream.str();
  std::cout.flush();

  // Allow io_service to stop in this example.
  work = boost::none;
}

/// @brief Mocked up read handler that will post work into a background
///        service.
void handle_read(const boost::system::error_code& error,
                 std::size_t bytes_transferred)
{
  std::cout << "in main service, need to query odbc" << std::endl;
  typedef void (*handler_type)(unsigned int);
  background_service.post(boost::bind(&query_odbc<handler_type>,
    21,                // data
    &handle_read_odbc) // handler
  );

  // Keep io_service event loop running in this example.
  work = boost::in_place(boost::ref(io_service));
} 

/// @brief Loop to show concurrency.
void print_loop(unsigned int iteration)
{
  if (!iteration) return;

  std::cout << "  in main service, doing work.\n";
  std::cout.flush();
  boost::this_thread::sleep_for(boost::chrono::seconds(1));
  io_service.post(boost::bind(&print_loop, --iteration));  
}

int main()
{
  boost::optional<boost::asio::io_service::work> background_work(
      boost::in_place(boost::ref(background_service)));

  // Dedicate 3 threads to performing long-standing blocking operations.
  boost::thread_group background_threads;
  for (std::size_t i = 0; i < 3; ++i)
    background_threads.create_thread(
      boost::bind(&boost::asio::io_service::run, &background_service));

  // Post a mocked up 'handle read' handler into the main io_service.
  io_service.post(boost::bind(&handle_read,
    make_error_code(boost::system::errc::success), 0));

  // Post a mockup loop into the io_service to show concurrency.
  io_service.post(boost::bind(&print_loop, 5));  

  // Run the main io_service.
  io_service.run();

  // Cleanup background.
  background_work = boost::none;
  background_threads.join_all();
}

And the output:

in main service, need to query odbc
  in main service, doing work.
in background service, start querying odbc
  in main service, doing work.
  in main service, doing work.
  in main service, doing work.
  in main service, doing work.
in background service, posting odbc result to main service
in main service, got 42 from odbc.

Note that the single thread processing the main io_service posts work into the background_service, and then continues to process its event loop while the background_service blocks. Once the background_service gets a result, it posts a handler into the main io_service.

Answer 2

We have same long-running tasks in our server (a legacy protocol with storages). So our server is running 200 threads to avoid blocking service (yes, 200 threads is running io_service::run). Its not too great thing, but works well for now.

The only problem we had is asio::strand which uses so-called "implementations" which gets locked when hadler is currently called. Solved this via increase this strands butckets and "deattaching" task via io_service::post without strand wrap.

Some tasks may run seconds or even minutes and this does work without issues at the moment.