问题
My code is allocating memory and never freeing it, even though it should (at least in my opinion).
The header looks like this:
typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> sslSocket_t;
class Object {
boost::asio::io_service ioService_;
boost::asio::ip::tcp::acceptor acceptor_;
boost::asio::ssl::context context_;
void functionOne();
void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& error)
}
And my source like this:
void Object::functionOne() {
for (int i = 0; i < 10; i++) {
shared_ptr<sslSocket_t> sslSocket(new sslSocket_t(ioService_, context_));
acceptor_.async_accept(sslSocket->lowest_layer(),
boost::bind(&Object::functionTwo, this, sslSocket, boost::asio::placeholders::error));
}
acceptor_.cancel();
boost::asio::io_service::work work(ioService_);
ioService_.run();
}
void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& err) {
// Do nothing
}
So when i call Object.functionOne(), memory is getting allocated to the Object.ioService_ object, in order to be able to call the bound asynchronous method. Then after the loop, all pending asynchronous actions on the acceptor are getting canceled. The appropriate handler is getting invoked as soon as Object.ioService_.run() is called (i've been testing that). BUT for some reason, the allocated memory does not get freed. So can someone please explain, why the memory is not getting deallocated and give me a hint how to free it?
Btw.: I'm working on debian and looking at /proc/self/status -> VmRSS to whatch the used memory.
@Vinnie Falco
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/bind.hpp>
#include <iostream>
#include <memory>
typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> sslSocket_t;
using namespace std;
struct T {
boost::asio::io_service ioService_;
boost::asio::ip::tcp::acceptor acceptor_;
boost::asio::ssl::context context_;
void functionOne() {
for (int i = 0; i < 10; i++) {
shared_ptr<sslSocket_t> sslSocket(new sslSocket_t(ioService_, context_));
acceptor_.async_accept(sslSocket->lowest_layer(),
boost::bind(&T::functionTwo, this, sslSocket, boost::asio::placeholders::error));
}
acceptor_.cancel();
boost::asio::io_service::work work(ioService_);
ioService_.run();
}
void functionTwo(shared_ptr<sslSocket_t>& sslSocket, const boost::system::error_code& err) {
// Do nothing
}
T() : acceptor_(ioService_,
boost::asio::ip::tcp::endpoint(boost::asio::ip::tcp::v4(), 443)),
context_(boost::asio::ssl::context::sslv23_server) {
}
~T() {
}
};
int main() {
try {
T t;
t.functionOne();
} catch (std::exception& e) {
cout << "Exception: " << e.what() << endl;
}
}
My question is not, if and why the destructor of T is called, this works as supposed to. But the behaviour concerning the used memory is strange. So if you increase the limit in the for loop, you will observe, that a lot of memory is getting reserved by the program, even though it should be released after all asnchronous handlers have been invoked. But the sslSocket objects are not getting deallocated, which is what my question is about: Why is the memory (specifically the memory allocated for the sslSocket), bound to the functor functionTwo, not deallocated, even after the asynchronous method fucntionTwo has been invoked and no reference to the sslSocket is left?
My final approach to explain my concern (edit 28 April)
Alright, i made a runnable example, that shows my concern: My Problem in an example
Output:
Before leaking call: 6984 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running: 460244 kB
Memory after ioService is stopped: 460244 kB
What's even more crazy is, that in my own local implementation I get the following output:
Memory leaking call: 8352 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running: 471932 kB
Memory after ioService is stopped: 8436 kB
So it can clearly be seen: the memory is not freed, even after all asynchronous operations have been invoked.
Summary and understood(?) behaviour (last edit)
As some of you might have missunderstood, i'm not thinking that there is some kind of a leak in my code. I named the structure in my code example Leak, which might have confused you, but my question is not if and where a memory leak occurs in my example. It's about the memory allocation in combination with the ioService object. First I thought, that the claimed memory is increasing infinitly. I made a last approach to understand this behaviour and came to the conclusion, that the memory management is fine. Memory is not reclaimed by the OS, but the program's memory allocation is converging to a limit, which is fine with me. So this problem is out of my way.
Example: Converging memory consumption
What disturbed me most was that my local implemention showed a slightly different behaviour. There the memory was reclaimed by the OS, when the ioService object was done with its jobs and reset, which satisfied my expectations.
So to sum up all observations:
The allocated memory is managed by the C++ Runtime and the OS. It's quite difficult (if not even impossible?) to directly observe the allocation procedure, since it's optimized to reduce the amount of requests for new memory pages, which means that allocated and freed memory might not be immidiatly reallocated by the OS.
To point out the critical point to me with this behaviour, i want to describe the usage of my program: I'm developing a server application, which means, that the program should run an infinit amount of time. If the program claims a lot of peak memory at some time, it's totaly fine, but it needs to release the claimed memory at some point in runtime, not after the runtime. So to me, there is only one question remaining:
Will the claimed (but unused) memory be reclaimed by the OS at some time? Or do I have to manage the memory on my own (with new and delete) in runtime?
回答1:
I'm not sure what the problem is but I think you're doing something wrong. Could you provide a self-contained example that exhibits the problem? This sample program compiles and runs, and the destructor is called:
#include <boost/asio.hpp>
#include <functional>
#include <iostream>
#include <memory>
struct T
{
T()
{
std::cerr << "T::T()\n";
}
~T()
{
std::cerr << "T::~T()\n";
}
};
void f(std::shared_ptr<T>&)
{
}
int main()
{
using namespace boost::asio;
io_service ios;
ios.post(std::bind(&f, std::make_shared<T>()));
ios.run();
}
You can see the output here: http://melpon.org/wandbox/permlink/0fkIAnoMXDOeedx7
The output:
T::T()
T::~T()
回答2:
Taking your selfcontained example and running it under valgrind shows that exactly nothing is leaked
==14098==
==14098== HEAP SUMMARY:
==14098== in use at exit: 73,696 bytes in 7 blocks
==14098== total heap usage: 163,524 allocs, 163,517 frees, 733,133,505 bytes allocated
==14098==
==14098== LEAK SUMMARY:
==14098== definitely lost: 0 bytes in 0 blocks
==14098== indirectly lost: 0 bytes in 0 blocks
==14098== possibly lost: 0 bytes in 0 blocks
==14098== still reachable: 73,696 bytes in 7 blocks
==14098== suppressed: 0 bytes in 0 blocks
==14098== Rerun with --leak-check=full to see details of leaked memory
If you supply valgrind --show-leak-kinds=all --leak-check=full ./test you'll find that the "leaked" (left-over reachables) are the usual static stuff allocated from libssl/libcrypto. They get allocated even if you only do 1 iteration. No change for 10000 iterations.
==14214== Memcheck, a memory error detector
==14214== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==14214== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==14214== Command: ./test 10000
==14214==
Before leaking call: 50056 kB
Asynchronous calls of functionTwo: 10000
Memory while ioService is still running: 265592 kB
Memory after ioService is stopped: 265592 kB
==14214==
==14214== HEAP SUMMARY:
==14214== in use at exit: 73,696 bytes in 7 blocks
==14214== total heap usage: 163,524 allocs, 163,517 frees, 733,133,505 bytes allocated
==14214==
==14214== 24 bytes in 1 blocks are still reachable in loss record 1 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53BF315: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C1863: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x40C9CA: context (context.ipp:70)
==14214== by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214== by 0x403E13: main (test.cpp:86)
==14214==
==14214== 32 bytes in 1 blocks are still reachable in loss record 2 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53BE7AE: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x507FD69: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x5081E68: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x5087532: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x40B9A8: do_init (openssl_init.ipp:39)
==14214== by 0x40B9A8: boost::asio::ssl::detail::openssl_init_base::instance() (openssl_init.ipp:131)
==14214== by 0x403C3C: openssl_init (openssl_init.hpp:60)
==14214== by 0x403C3C: __static_initialization_and_destruction_0 (openssl_init.hpp:90)
==14214== by 0x403C3C: _GLOBAL__sub_I_count (test.cpp:96)
==14214== by 0x40FE1C: __libc_csu_init (in /home/sehe/Projects/stackoverflow/test)
==14214== by 0x5EC09CE: (below main) (libc-start.c:245)
==14214==
==14214== 32 bytes in 1 blocks are still reachable in loss record 3 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53BE7CC: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x507FD69: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x5081E68: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x5087532: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==14214== by 0x40B9A8: do_init (openssl_init.ipp:39)
==14214== by 0x40B9A8: boost::asio::ssl::detail::openssl_init_base::instance() (openssl_init.ipp:131)
==14214== by 0x403C3C: openssl_init (openssl_init.hpp:60)
==14214== by 0x403C3C: __static_initialization_and_destruction_0 (openssl_init.hpp:90)
==14214== by 0x403C3C: _GLOBAL__sub_I_count (test.cpp:96)
==14214== by 0x40FE1C: __libc_csu_init (in /home/sehe/Projects/stackoverflow/test)
==14214== by 0x5EC09CE: (below main) (libc-start.c:245)
==14214==
==14214== 128 bytes in 1 blocks are still reachable in loss record 4 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53BEFE1: lh_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C1512: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C182F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x40C9CA: context (context.ipp:70)
==14214== by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214== by 0x403E13: main (test.cpp:86)
==14214==
==14214== 176 bytes in 1 blocks are still reachable in loss record 5 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53BEFBF: lh_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C1512: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C182F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C245D: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x40C9CA: context (context.ipp:70)
==14214== by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214== by 0x403E13: main (test.cpp:86)
==14214==
==14214== 600 bytes in 1 blocks are still reachable in loss record 6 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x5307E77: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C23F5: ERR_get_state (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x53C25EE: ERR_clear_error (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==14214== by 0x40C9CA: context (context.ipp:70)
==14214== by 0x40C9CA: Leak::Leak() (test.cpp:77)
==14214== by 0x403E13: main (test.cpp:86)
==14214==
==14214== 72,704 bytes in 1 blocks are still reachable in loss record 7 of 7
==14214== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14214== by 0x57731FF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14214== by 0x4010609: call_init.part.0 (dl-init.c:72)
==14214== by 0x401071A: call_init (dl-init.c:30)
==14214== by 0x401071A: _dl_init (dl-init.c:120)
==14214== by 0x4000D09: ??? (in /lib/x86_64-linux-gnu/ld-2.21.so)
==14214== by 0x1: ???
==14214== by 0xFFEFFFF76: ???
==14214== by 0xFFEFFFF7D: ???
==14214==
==14214== LEAK SUMMARY:
==14214== definitely lost: 0 bytes in 0 blocks
==14214== indirectly lost: 0 bytes in 0 blocks
==14214== possibly lost: 0 bytes in 0 blocks
==14214== still reachable: 73,696 bytes in 7 blocks
==14214== suppressed: 0 bytes in 0 blocks
==14214==
==14214== For counts of detected and suppressed errors, rerun with: -v
==14214== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Your way of measuring memory usage is not sound.
回答3:
It took me a while, but i finally managed to work it out by myself.
So to clearify things, let's make sure, that the root of my problem is understood: I'm developing a server application, which is meant to run for an infinit amount of time. This application must be able to handle a lot of concurrent incomming connections. At some point in time, there may be a peak in load, leading to a lot of claimed memory to my application. Then after a while, most incomming requests have been processed, causing a lot of objects to be freed in runtime. Since the OS is in no need for memory (My application is the only huge memory consumer on the server), all freed memory stays with my application and can be reused at another point in time. This would be absolutely fine with me, but some customers and administrators might misinterpret the greater amount of constantly claimed memory as a memory leaking application. To avoid this, i wanted to hand some of the unused memory back to the OS manually. In my example, the bound handlers to the ioService (e.g. accepting a new connection) would be freed in runtime, but the appropriate memory won't be reclaimed by the OS. So to do this manually, i found the following solution:
Release unused heap memory under Linux in C/C++: int malloc_trim(size_t pad)
The documentation can be found here. Simplified explained, this method releases unused memory from the heap to the OS, which is exactly what i've been searching for. I'm aware that under memory optimization aspects, the manual use of this function maybe dangerous, but since i only want to release the memory every few minutes, this performance issue is acceptable to me.
Thank you all for your efforts and patience!
回答4:
I think your approach is broken. You should never interleave async operations with asio. If your do all sorts of undefined crap will happen. The way you normally implement async accept is as follows:
void do_accept() {
shared_ptr<sslSocket_t> socket(new sslSocket_t(ioService_, context_));
// Queue an async accept operation
acceptor_.async_accept(socket->lowest_layer(), [this, socket](auto ec) {
if (!ec) {
// Handle the socket
}
// If not shutting down
this->do_accept(); // next accept
});
}
来源：https://stackoverflow.com/questions/36913157/boostasio-and-boostbind-functor-memory-is-never-released