How is the Linux kernel tested ?

Question

How do the Linux kernel developers test their code locally and after they have it committed? Do they use some kind of unit testing, build automation? test plans?

Answer 1

There also are:

MMTests which is collection of benchmarks and scripts to analyze the results

https://github.com/gormanm/mmtests

Trinity which is Linux system call fuzz tester

http://codemonkey.org.uk/projects/trinity/

Also the LTP pages at the sourceforge are quite outdated and the project has moved to GitHub https://github.com/linux-test-project/ltp

Answer 2

LTP and Memtests are generally preferred tools.

Answer 3

How do the Linux kernel developers test their code locally and after they have it committed?

Do they use some kind of unit testing, build automation?

In classic sense of words, no.

E. g. Ingo Molnar is running the following workload: 1. build new kernel with random set of config options 2. boot into it 3. goto 1

Every build fail, boot fail, BUG or runtime warning is dealt with. 24/7. Multiply by several boxes, and one can uncover quite a lot of problems.

test plans?

No.

There may be misunderstanding that there is central testing facility, there is none. Everyone does what he wants.

Answer 4

The linux kernel has a heavy emphasis on community testing.

Typically any developer will test their own code before submitting, and quite often they will be using a development version of the kernel from Linus, or one of the other unstable/development trees for a project relevant to their work. This means they are often testing both their changes and other people's changes.

There tend not to be much in the way of formal test plans, but extra testing may be asked for before features are merged into upstream trees.

As Dean pointed out, there's also some automated testing, the linux test project and the kernel autotest (good overview).

Developers will often also write automated tests targetted to test their change, but I'm not sure there's a (often used) mechanism to centrally collect these adhoc tests.

It depends a lot on which area of the kernel is being changed of course - the testing you'd do for a new network driver is quite different to the testing you'd do when replacing the core scheduling algorithm.

Answer 5

In addition to above/below points, which emphasis more on the functionality testing, hardware certification testing and performance testing the Linux kernel.

A lot a testing actually happen through, actually scripts, static code analysis tools, code reviews etc. which is very efficient in catching bugs, which would otherwise break something in the application.

Sparse – An open-source tool designed to find faults in the Linux kernel.

Coccinelle is another program does matching and transformation engine which provides the language SmPL (Semantic Patch Language) for specifying desired matches and transformations in C code.

checkpatch.pl and other scripts - coding style issues can be found in the file Documentation/CodingStyle in the kernel source tree. The important thing to remember when reading it is not that this style is somehow better than any other style, just that it is consistent. this helps developers easily find and fix coding style issues, the script scripts/checkpatch.pl in the kernel source tree has been developed. This script can point out problems easily, and should always be run by a developer on their changes, instead of having a reviewer waste their time by pointing out problems later on.

Answer 6

Naturally, the kernel itself and its parts are tested prior to the release, but these tests cover only the basic functionality. There are some testing systems which perform testing of Linux Kernel:

The Linux Test Project (LTP) delivers test suites to the open source community that validate the reliability and stability of Linux. The LTP test suite contains a collection of tools for testing the Linux kernel and related features. https://github.com/linux-test-project/ltp

Autotest -- a framework for fully automated testing. It is designed primarily to test the Linux kernel, though it is useful for many other purposes such as qualifying new hardware, virtualization testing, and other general user space program testing under Linux platforms. It's an open-source project under the GPL and is used and developed by a number of organizations, including Google, IBM, Red Hat, and many others. http://autotest.github.io/

Also there are certification systems developed by some major GNU/Linux distribution companies. These systems usually check complete GNU/Linux distributions for compatibility with hardware. There are certification systems developed by Novell, Red Hat, Oracle, Canonical, Google.

There are also systems for dynamic analysis of Linux kernel:

Kmemleak is a memory leak detector included in the Linux kernel. It provides a way of detecting possible kernel memory leaks in a way similar to a tracing garbage collector with the difference that the orphan objects are not freed but only reported via /sys/kernel/debug/kmemleak.

Kmemcheck traps every read and write to memory that was allocated dynamically (i.e. with kmalloc()). If a memory address is read that has not previously been written to, a message is printed to the kernel log. Also is a part of Linux Kernel

Fault Injection Framework (included in Linux kernel) allows for infusing errors and exceptions into an application's logic to achieve a higher coverage and fault tolerance of the system.