Core Dump Generation
How to write a crashing C++ program?
vi a.cpp
#include <string.h>
#include <chrono>
#include <thread>
int main() {
std::this_thread::sleep_for(std::chrono::seconds(10)); // 10 or 120 char* p = NULL; strcpy(p, "copy string to null pointer and cause crash"); return 0; } // g++ -g a.cpp -o a.out -std=c++11Segmentation fault (core dumped)
ulimit -c
0
ulimit -a | grep core
core file size (blocks, -c) 0
How to enable core file size?
ulimit -c unlimited
ulimit -a | grep corecore file size (blocks, -c) unlimited
What is the file size of the core dump?
417KB
./a.out
ll core.*
rw-------. 1 ricky ricky 417792 Mar 28 11:06 core.1258
Background of systemd
What is init?
init is the default PID 1 in CentOS 6
cat /etc/*release
CentOS release 6.6
ps -ef | head -2
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 2016 ? 00:01:03 /sbin/init
What is systemd?
systemd is the default PID 1 in CentOS 7
cat /etc/*release
CentOS Linux release 7.2.1511
ps -ef | head -2
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 2016 ? 00:09:54 /usr/lib/systemd/systemd --system --deserialize 26
How to create a daemon?
systemctl daemon-reexec
How to make a b.service that requires a.service?
sudo vim /etc/systemd/system/b.service
[Unit]
Description=service b.service here
Requires=a.service
[Service]
Type=simple
Restart=always
WorkingDirectory=/home/ricky/b/
ExecStart=/home/ricky/b/b.out
ExecStartPre=/usr/bin/date
ExecStartPost=/usr/bin/date
ExecReload=/usr/bin/date
ExecStop=/usr/bin/date
ExecStopPost=/usr/bin/date
[Install]
WantedBy=multi-user.target
sudo systemctl stop a
sudo systemctl stop b
sudo systemctl start a
sudo systemctl start b
sudo systemctl status -l a
sudo systemctl status -l b
[Unit]
Description=sleeping service
[Service]
Type=simple
User=ricky
Restart=always
WorkingDirectory=/home/ricky/
ExecStart=/usr/bin/sleep 10
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload && sudo systemctl restart sleeping && sudo systemctl status sleeping
abcdefghijklmnopqrstuwxyz
Core Dump Analysis
How to setup dbg for debugging?
sudo debuginfo-install glibc-2.17-106.el7_2.8.x86_64 libgcc-4.8.5-11.el7.x86_64 libstdc++-4.8.5-11.el7.x86_64
How to analyse the core dump file?
gdb a.out core.1258
bt
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/ricky/crash/a.out...done.
[New LWP 1258]
Core was generated by `./a.out'.
Program terminated with signal 11, Segmentation fault.
#0 0x00000000004005cb in main () at crashing.cpp:6
6 strcpy(p, "copy string to null pointer and cause crash");
(gdb) print p
$1 = 0x0
What is init?
init is the default PID 1 in CentOS 6
cat /etc/*release
CentOS release 6.6
ps -ef | head -2
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 2016 ? 00:01:03 /sbin/init
What is systemd?
systemd is the default PID 1 in CentOS 7
cat /etc/*release
CentOS Linux release 7.2.1511
ps -ef | head -2
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 2016 ? 00:09:54 /usr/lib/systemd/systemd --system --deserialize 26
Restarting the crashing program with systemd
How to create a daemon?
sudo vim /etc/systemd/system/a.service
[Unit]
Description=service a.service here
[Unit]
Description=service a.service here
[Service]
Type=simple
Restart=always
WorkingDirectory=/home/ricky/a/
ExecStart=/home/ricky/a/a.out
[Install]
WantedBy=multi-user.target
How to start a daemon?
sudo systemctl daemon-reload
sudo systemctl status -l a
● a.service
Loaded: loaded (/etc/systemd/system/a.service; disabled; vendor preset: disabled)
Active: inactive (dead)
sudo systemctl start a
sudo systemctl status -l a
● a.service
Loaded: loaded (/etc/systemd/system/a.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2017-03-28 11:46:36 HKT; 108ms ago
Main PID: 2056 (a.out)
CGroup: /system.slice/a.service
└─2056 /home/ricky/a/a.out
ps -ef | grep a.out
How to enable core dump for systemd daemon?
sudo vi /etc/systemd/system.conf
DefaultLimitCORE=infinitysystemctl daemon-reexec
How to make a b.service that requires a.service?
sudo vim /etc/systemd/system/b.service
[Unit]
Description=service b.service here
Requires=a.service
[Service]
Type=simple
Restart=always
WorkingDirectory=/home/ricky/b/
ExecStart=/home/ricky/b/b.out
ExecStartPre=/usr/bin/date
ExecStartPost=/usr/bin/date
ExecReload=/usr/bin/date
ExecStop=/usr/bin/date
ExecStopPost=/usr/bin/date
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl status -l b
sudo systemctl status -l a
sudo systemctl start b
sudo systemctl status -l a
sudo systemctl status -l b
ls && ps -ef | grep b.out
ls && ps -ef | grep b.out
sudo systemctl stop a
sudo systemctl stop b
sudo systemctl start a
sudo systemctl start b
sudo systemctl status -l a
sudo systemctl status -l b
ExecStartPre=/bin/bash -c 'pwd; ulimit -c unlimited; rm ./*.log; python -c "print(123)" '?
Process: 23069 ExecStartPre=/bin/bash -c pwd; ulimit -c unlimited; rm ./*.log; python -c "print(123)" (code=exited, status=0/SUCCESS)
Apr 07 12:41:05 pc009 bash[23069]: /home/ricky/b
Apr 07 12:41:05 pc009 bash[23069]: rm: cannot remove ‘./*.log’: No such file or directory
Apr 07 12:41:05 pc009 bash[23069]: 123
ExecStartPre=/usr/bin/sleep 5?
Yes. It will be exited with status 0/SUCCESS.
Process: 20924 ExecStartPre=/usr/bin/sleep 5 (code=exited, status=0/SUCCESS)
ExecStartPre=/usr/bin/sleep 1; /usr/bin/sleep 1; /usr/bin/sleep 3?
No. Multiple commands not allowed.
Process: 21025 ExecStartPre=/usr/bin/sleep 1; /usr/bin/sleep 1; /usr/bin/sleep 3 (code=exited, status=1/FAILURE)
ExecStartPre=rm /home/ricky/b/md/*.con ?
No. It is because rm is not absolute. You will see below error.
[/etc/systemd/system/b.service:8] Executable path is not absolute, ignoring: rm /home/ricky/b/md/*.con
ExecStartPre=/usr/bin/rm /home/ricky/b/md/*.con ?
ExecStartPre=/usr/bin/rm /home/ricky/b/md/*.con ?
No. It will be exited with status 1/FAILURE
Process: 20827 ExecStartPre=/usr/bin/rm /home/ricky/b/md/*.con (code=exited, status=1/FAILURE)
How to make a sleeping service?
sudo vim /etc/systemd/system/sleeping.service[Unit]
Description=sleeping service
[Service]
Type=simple
User=ricky
Restart=always
WorkingDirectory=/home/ricky/
ExecStart=/usr/bin/sleep 10
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload && sudo systemctl restart sleeping && sudo systemctl status sleeping
● sleeping.service - sleeping service
Loaded: loaded (/etc/systemd/system/sleeping.service; disabled; vendor preset: disabled)
Active: active (running) since Fri 2017-04-21 16:43:19 HKT; 31ms ago
Main PID: 28266 (sleep)
CGroup: /system.slice/sleeping.service
└─28266 /usr/bin/sleep 10
Apr 21 16:43:19 pc009 systemd[1]: Started sleeping service.
Apr 21 16:43:19 pc009 systemd[1]: Starting sleeping service...
abcdefghijklmnopqrstuwxyz
abcdefghijklmnopqrstuwxyz
abcdefghijklmnopqrstuwxyz
abcdefghijklmnopqrstuwxyz
abcdefghijklmnopqrstuwxyz
abcdefghijklmnopqrstuwxyz