How is the Linux kernel tested ?

Question

How do the Linux kernel developers test their code locally and after they have it committed? Do they use some kind of unit testing, build automation? test plans?

Answer 1

I had done linux kernel compilation and done some Modifications for android(Marshmallow and Nougat) in which I use linux version 3. I cross-compiled it in linux system, debug the errors manually and then run its boot image file in Android and check if it was going in loop-hole or not. If it runs perfect then it means it is compiled perfectly according to system requirements.
For MotoG kernel Compilation

NOTE:- Linux Kernel will change according to requirements which depend on System Hardware

Answer 2

In-tree tools

A good way to find test tools in the kernel is to:

• make help and read all targets
• look under tools/testing

In v4.0, this leads me to:

• kselftest under tools/testing/selftests. Run with make kselftest. Must be running built kernel already. See also: Documentation/kselftest.txt , https://kselftest.wiki.kernel.org/

• ktest under tools/testing/ktest. See also: http://elinux.org/Ktest , http://www.slideshare.net/satorutakeuchi18/kernel-auto-testbyktest

• Static analysers section of make help, which contains targets like:

  • checkstack: Perl: what does checkstack.pl in linux source do?
  • coccicheck for Coccinelle (mentioned by askb)

Kernel CI

https://kernelci.org/ is a project that aims to make kernel testing more automated and visible.

It appears to do only build and boot tests (TODO how to test automatically that boot worked Source should be at https://github.com/kernelci/).

Linaro seems to be the main maintainer of the project, with contributions from many big companies: https://kernelci.org/sponsors/

Linaro Lava

http://www.linaro.org/initiatives/lava/ looks like a CI system with focus on development board bringup and the Linux kernel.

ARM LISA

https://github.com/ARM-software/lisa

Not sure what it does in detail, but it is by ARM and Apache Licensed, so likely worth a look.

Demo: https://www.youtube.com/watch?v=yXZzzUEngiU

Step debuggers

Not really unit testing, but may help once your tests start failing:

• QEMU + GDB: https://stackoverflow.com/a/42316607/895245
• KGDB: https://stackoverflow.com/a/44226360/895245

My own QEMU + Buildroot + Python setup

I also started a setup focused on ease of development, but I ended up adding some simple testing capabilities to it as well: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/8217e5508782827320209644dcbaf9a6b3141724#test-this-repo

I haven't analyzed all the other setups in great detail, and they likely do much more than mine, however I believe that my setup is very easy to get started with quickly because it has a lot of documentation and automation.

Answer 3

adobriyan mentioned Ingo's loop of random config build testing. That is pretty much now covered by the 0-day test bot (aka kbuild test bot). A nice article about the infrastructure is presented here:Kernel Build/boot testing

The idea behind this set-up is to notify the developers ASAP so that they can rectify the errors soon enough. (before the patches make it into Linus' tree in some cases as the kbuild infrastructure also tests against maintainer's subsystem trees)

Answer 4

I would imagine they use virtualization to do quick tests, something like QEMU, VirtualBox or Xen, and some scripts to perform configurations and automated tests.

Automated testing is probably done by trying either many random configurations or a few specific ones (if they are working with a specific issue). Linux has a lot of low-level tools (such as dmesg) to monitor and log debug data from the kernel, so I imagine that is used as well.

Answer 5

Once after contributers submit their patch files & after making merge request linux gatekeepers are checking the patch by integrating & review it.Once if it successs they will merge the patch into relevant branch & make new version release. Linux Test Project (https://github.com/linux-test-project/ltp) is the main source which provides test scenarios(Test Cases) to run against kernel after applying patches. This may take around 2 ~ 4 hours & depends. Please note regarding the file system of the Selected kernel is going to test against. Ex:Ext4 generates different results against EXT3 & so on.

Kernel Testing procedure.

1. Get latest kernel source from the repository.(https://www.kernel.org/ or Github.com)
2. Apply patch file(Using Diff tool)
3. Build new kernel.
4. Test against test procedures in LTP(https://github.com/linux-test-project/ltp)

Answer 6

Its not very easy to automate kernel testing. Most Linux developers do the testing on their own, much like adobriyan mentioned.

However, there are a few things that help with debugging the Linux Kernel:

• kexec: A system call that allows you to put another kernel into memory and reboot without going back to the BIOS, and if it fails, reboot back.
• dmesg: Definitely the place to look for information about what happened during the kernel boot and whether it works/doesn't work.
• Kernel Instrumentation: In addition to printk's (and an option called 'CONFIG_PRINTK_TIME' which allows you to see (to microsecond accuracy) when the kernel output what), the kernel configuration allows you to turn on a LOT of tracers that enable them to debug what is happening.

Then, developers usually have others review their patches. Once the patches are reviewed locally and seen not to interfere with anything else, and the patches are tested to work with the latest kernel from Linus without breaking anything, the patches are pushed upstream.

Edit: Here's a nice video detailing the process a patch goes through before it is integrated into the kernel.