EuroBSDcon 2019
Author: Kamil Rytarowski
E-mail: kamil@netbsd.org
Speaker: Maciej Grochowski
Date: September 21st 2018
Place: Lillehammer, Norway
Sanitizer - is an open source programming tool that detects program bugs. Sanitizer is based on compiler instrumentation and runtime library.
MKSANITIZER is a distinct feature of NetBSD that allows the whole distribution sanitization.
See: MeetBSDCa 2018 slides.
Sanitizers:
Additionally:
MKSANITIZER merged into NetBSD-current (and NetBSD-9).
LLVM sanitizers pass as of now 95% of the upstream tests on NetBSD.
There are hundreds of issues left in programs, libraries and the sanitizer runtime before executing all ATF regression tests. There are likely some issues to be fixed on the kernel/userland level.
Prerequisites:
Step 1: Build the distribution with MKSANITIZER and libFuzzer code coverage.
1 ./build.sh \
2 -V MKCOMPAT=no \ # No 32-bit support in libFuzzer
3 -V MKDEBUGLIB=yes -V MKDEBUG=yes \ # Add debug symbols
4 -V MKSANITIZER=yes \ # Enable sanitization of the userland
5 -V USE_SANITIZER=fuzzer-no-link \ # libFuzzer code coverage
6 -V MKLLVM=yes -V MKGCC=no -V HAVE_LLVM=yes \ # Clang/LLVM distro
7 -O /public/netbsd.fuzzer \ # Example destination work dir
8 distribution
GCC MKSANITIZER is possible, but currently there was no work on it.
Step 2: Install devel/honggfuzz from pkgsrc.
honggfuzz is code coverage aided evolutionary fuzzer, with ptrace(2) based monitoring.
honggfuzz supports persistent fuzzing mode (without forks, execs).
Step 3: Mount useful directories into destdir
mount -t null /dev /public/netbsd.fuzzer/destdir.amd64/dev
mount -t null /dev/pts /public/netbsd.fuzzer/destdir.amd64/dev/pts
mount -t null /tmp /public/netbsd.fuzzer/destdir.amd64/tmp
mkdir -p /public/netbsd.fuzzer/destdir.amd64/usr/pkg
mount -t null /usr/pkg /public/netbsd.fuzzer/destdir.amd64/usr/pkg
Step 4: chroot into the new environment.
chroot /public/netbsd.fuzzer/destdir.amd64
cd /tmp
Step 5: Write rumpkernel code.
Example: Testing mount(2) + unmount(2) of FFS.
http://netbsd.org/~kamil/rump/rump_pub_etfs_register_buffer.c
Step 6: Start the fuzzer.
1 hfuzz-clang -lrumpvfs -lrump -lrumpfs_ffs -lrumpuser -lrumpdev_disk \
2 -lrumpdev -pthread rump_pub_etfs_register_buffer.c
3 mkdir corpus
4 newfs -F -s 10000 corpus/ffs.img
5 honggfuzz -P -f corpus/ -- ./a.out
Step 7: Observe crashes...
1 ------------------------[ 0 days 01 hrs 00 mins 00 secs ]-------
2 Iterations : 367,977 [367.98k]
3 Mode : [2/2] Feedback Driven Mode
4 Target : ./a.out
5 Threads : 4, CPUs: 8, CPU%: 0% [0%/CPU]
6 Speed : 25/sec [avg: 102]
7 Crashes : 7817 [unique: 7817, blacklist: 0, verified: 0]
8 Timeouts : 13 [10 sec]
9 Corpus Size : 203, max size: 5,120,000 bytes, init dir: 361 files
10 Cov Update : 0 days 00 hrs 00 mins 29 secs ago
11 Coverage : edge: 28 pc: 206 cmp: 104,650
12 ---------------------------------- [ LOGS ] ----/ honggfuzz 1.7 /-
Status after 60 minutes of fuzzing.
Step 8: Observe crashes...
1 Persistent mode: Launched new persistent PID: 18581
2 [ 1.0400090] panic: kernel diagnostic assertion "newsize != VSIZ
3 ENOTSET && newsize >= 0" failed: file "/usr/src/lib/librumpvfs/../
4 ../sys/rump/../uvm/uvm_vnode.c", line 353
5 [ 1.0400090] rump kernel halting...
6 halted
7 [ 1.0300090] ufs_inactive: unlinked ino 2 on "/mnt" has non zero
8 size 0 or blocks 817b2ac6 with allerror 0
9 [ 1.0300090] panic: ufs_inactive: dirty filesystem?
10 [ 1.0300090] rump kernel halting...
11 halted
12 Crash: saved as './SIGABRT.PC.0.STACK.badbad183b661ac0.CODE.-5.ADD
13 R.0x0.INSTR.jb_0x7f7ff4d79acd.2019-08-31.01:57:47.18384.fuzz'
14 [2019-08-31T01:57:47+0200][W][5] arch_checkWait():230 Persistent m
15 ode: PID 18384 exited with status: SIGNALED, signal: 6 (Abort trap)
16 Persistent mode: Launched new persistent PID: 10281
Step 9: Log rumpkernel crashes into a file.
1 [ 1.0900090] panic: kernel diagnostic assertion "size + offset <
2 = flen" failed: file "/usr/src/lib/librumpvfs/../../sys/rump/libru
3 mp/rumpvfs/rumpblk.c", line 292
4 [ 1.0900090] rump kernel halting...
5 halted
6 [ 1.4300090] /devdisk: file system not clean (fs_clean=0x4); ple
7 ase fsck(8)
8 [ 1.4300090] /devdisk: lost blocks 0 files 0
9 [ 1.4500090] /devdisk: file system not clean (fs_clean=0x8); ple
10 ase fsck(8)
11 [ 1.4500090] /devdisk: lost blocks 0 files 0
12 [ 1.6300090] ufs_inactive: unlinked ino 2 on "/mnt" has non zero
13 size 0 or blocks 1b8b53de with allerror 0
14 [ 1.6300090] panic: ufs_inactive: dirty filesystem?
15 [ 1.6300090] rump kernel halting...
16 halted
17 [ 1.4200090] panic: kernel diagnostic assertion "newsize != VSIZ
18 ENOTSET && newsize >= 0" failed: file "/usr/src/lib/librumpvfs/../
19 ../sys/rump/../uvm/uvm_vnode.c", line 353
Parse thousands of similar lines.
Step 10: Fix bugs, repeat.
There is enough work before need to make the fuzzing process quicker.
NetBSD supports right now:
Sanitizers still not merged into mainline:
Credit: Maxime Villard (maxv@)
Syzkaller + syzbot 24/7 fuzzing with KCOV assisted coverage driver
http://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd http://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd http://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd1
GSoC 2019 by Siddharth Muralee
http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1 http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part2
GSoC 2019 by Akul Pillai
1 Modified Files:
2 src/etc: MAKEDEV.tmpl
3 src/sys/arch/amd64/conf: GENERIC
4 src/sys/conf: files majors
5 Added Files:
6 src/sys/dev/usb: vhci.c
7
8 Log Message:
9 Add vHCI, a driver which allows to send and receive USB packets
10 directly from userland via /dev/vhci. Using this, it becomes
11 possible to test and fuzz the USB stack and all the USB drivers
12 without having the associated hardware.
13
14 The vHCI device has four ports independently addressable.
15
16 For each xfer on each port, we create two packets: a setup packet
17 (which indicates mostly the type of request) and a data packet
18 (which contains the raw data). These packets are processed by read
19 and write operations on /dev/vhci: userland poll-reads it to fetch
20 usb_device_request_t structures, and dispatches the requests
21 depending on bRequest and bmRequestType.
vHCI added by Maxime Villard on 2019-09-14.
Hardware assisted virtualization is now available for NetBSD/amd64 through NVMM and HAXM.
A prerequisite for Qemu-assisted fuzzing (Syzkaller, ...).
Old school Xen for old school use-cases.
Table of Contents | t |
---|---|
Exposé | ESC |
Full screen slides | e |
Presenter View | p |
Source Files | s |
Slide Numbers | n |
Toggle screen blanking | b |
Show/hide slide context | c |
Notes | 2 |
Help | h |