Convert large core files to “minicore” files

问题

How do I reduce core files to just the threads' stacks?

I want to be able to run gdb thread apply all bt on the mini core and no more

I'm dealing with large (>4GB) multi-threaded Linux ELF core files that are too big to get back for analysis.

I've see google-breakpad which is meant to create a "minidump" when a process crashes. In google-breakpad there are two utilities core2md and minidump-2-core which at first glance should be able to convert a core file to minidump format, and then back to a core file from the minidump with just the stack information. The problem with this is that core2md requires the process information from under /proc/$PID/ and that's not something I have.

回答1:

Probably your best bet is to integrate google breakpad into your Linux process so that when a crash occurs it is google breakpad recording the crash instead of the OS creating a core file. This will get you the information that you want, and then you can either use minidump_stackwalk to dump call stacks (handy for summarizing lots of crashes) or you can use minidump-2-core to create a core file.

It is unfortunate that Linux does not seem to have an effective way to create minimal core files with just the contents of the threads' stacks. These files (minidumps in the Windows world) are extremely useful -- they have a very high information-to-size ratio which allows easy collection of millions of crash dumps.

回答2:

You can probably write a simple program which remove much of the data from the core file.

The core file is made of PT_LOAD program header entries representing the different VMAs:

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  NOTE           0x0000000000004b80 0x0000000000000000 0x0000000000000000
                 0x0000000000009064 0x0000000000000000  R      1
  LOAD           0x000000000000dbe4 0x0000000000400000 0x0000000000000000
                 0x0000000000000000 0x000000000009d000  R E    1
  LOAD           0x000000000000dbe4 0x000000000069c000 0x0000000000000000
                 0x0000000000004000 0x0000000000004000  RW     1
  LOAD           0x0000000000011be4 0x00000000006a0000 0x0000000000000000
                 0x0000000000004000 0x0000000000004000  RW     1
  LOAD           0x0000000000015be4 0x0000000001872000 0x0000000000000000
                 0x0000000000ed4000 0x0000000000ed4000  RW     1
  LOAD           0x0000000000ee9be4 0x00007f248c000000 0x0000000000000000
                 0x0000000000021000 0x0000000000021000  RW     1
  LOAD           0x0000000000f0abe4 0x00007f2490885000 0x0000000000000000
                 0x000000000001c000 0x000000000001c000  R      1
  LOAD           0x0000000000f26be4 0x00007f24908a1000 0x0000000000000000
                 0x000000000001c000 0x000000000001c000  R      1
[...]

The PT_NOTE entry contains (among other things) informations about the thread state:

Displaying notes found at file offset 0x00004b80 with length 0x00009064:
  Owner                 Data size       Description
  CORE                 0x00000088       NT_PRPSINFO (prpsinfo structure)

  [Thread #1]
  CORE                 0x00000150       NT_PRSTATUS (prstatus structure)
  CORE                 0x00000200       NT_FPREGSET (floating point registers)
  LINUX                0x00000440       NT_X86_XSTATE (x86 XSAVE extended state)
  CORE                 0x00000080       NT_SIGINFO (siginfo_t data)

  [other threads ...]

PT_NOTE handling

You want to keep this PT_NOTE program header entry.

Moreover you can extract the stack pointers from the prstatus structures:

struct elf_prstatus {
  [...]
  elf_gregset_t pr_reg;   /* GP registers */
  [...]
};

PT_LOAD handling

Once you have extracted all the stack pointers from all threads, you can process the PT_LOAD entries:

if FileSize == 0, this program header does not consume any memory in the core file and can be ignored;
if the stack pointer of some thread is in this region of virtual memory, this is probably a stack and you might need to keep it;
otherwise, you might be able to remove it from the core file (replace it with a program header entry with FileSize == 0).

Alternatively, you might be able to completely remove the program header entries of all the non-stack regions.

Creating you own core dumper

Another solution would be to write a core dumper which generated this directly and register in /proc/sys/kernel/core_pattern (man core).

来源：https://stackoverflow.com/questions/12737025/convert-large-core-files-to-minicore-files

标签

c++

Linux

debugging

gdb