Allocating largest buffer without using swap

Question

In C/C++ under Linux, I need to allocate a large (several gigabyte) block of memory, in order to store real-time data from a sensor connected to the ethernet port and streaming

Answer 1

What is the best way to determine how much memory to allocate?

Due to how virtual memory is used, non-swappable kernel memory, it is nearly impossible to identify how much of installed memory can be accessed by an application.

Best I can come up with is to allow user to configure how much memory to use for buffering.

Am I limited to just allocating a slightly smaller block than the reported free memory,

Reported free memory is not really "free physical memory." Unfortunately.

or can I interface more directly with the linux virtual memory manager?

That can be done by using a custom device driver, allocating memory directly in kernel space and providing access to it via mmap(). Generally not recommended, yet would works in specialized cases such as yours.

However, I also need to make sure that there will be no disk-swapping

At pace of the Linux kernel development, knowledge becomes obsolete quite fast, so take with grain of salt what I'm saying here. You can try to play with the following:

1. SysV shared memory. It is generally not swapped. See man shmget.

2. tmpfs - in-memory file system. The memory was pinned to RAM at least in early 2.6 kernels and thus was not swappable. To use it as memory, create a file on tmpfs, write() something into the file (to force the memory being actually allocated) and then mmap() the file.