Coverage of signal routines in the kernel in the context of ptrace(2)
During the past month I have been working on coverage of various corner cases in the signal subsystem in the kernel. I have also spent some time on improvements in the land of sanitizers. As a mentor I was able to, thanks to the fulltime focus on NetBSD work, actively help three Google Summer of Code students. Not every question would be answered by myself without code reading but at least I am available for active collaboration, especially when it's to improve code that I have already authored, like sanitizers. At the end of the month we have managed to catch two uninitialized memory reads in the top(1) utility, using the Memory Sanitizer feature and rebuilt part of the basesystem (i.e. library dependencies: libterminfo, libkvm, libutil) with dedicated sanitization flags.
I am actively working on handling of processes, forks/vforks, signals and threads that is reliable and fully functional under a debugger. This is a process and the situation is actively improving. For the end-user this means that we are achieving the state when a developer will be able to trace an application like Firefox using modern tools and save time detecting the issues quickly.
I am using the Test-Driven Development approach in my work. I keep extending the Automatic Test Framework with new tests, covering sets of scenarios handled by debuggers and related code. This is followed by kernel fixes. Thanks to the tests, I can more confidently introduce changes to critical routines inside the Operating System, test new changes quickly for regressions and keep covering new verifable scenarios.
Titles of the merged commits with the main tree of NetBSD:
e_tracesig used to be implemented for Darwin compat. Nowadays the Darwin compatiblity layer is gone and there are no other users.
In traceme* tests after validate_status_stopped() include additional check the verify the received signal with PT_GET_SIGINFO.
New functions:
raise(SIGSTOP) is now handled correctly by the kernel, in a child that vfork(2)ed and called PT_TRACE_ME.
Adapt the test to be independent from the software breakpoint trap behavior, whether the Program Counter is moved or not. Just kill the process after catching the expected signal, instead of pretending to resume it.
Note that SIGCHLD is not just a child exit signal. Note that SIGIOT is PDP-11 specific signal.
Note that SIGCHLD covers process continued event.
Introduce:
Added tests:
The purpose of this function is to detect whether a tracer can write to the .text section of its tracee.
This functionality now works.
Don't allow to PT_ATTACH from a vfork(2)ed child (before exec(3)/_exit(3)) to its parent. Return error with EPERM errno.
This scenario does not have a purpose and there is no clear picture how to route signals.
No need to check p_pid to compare whether two processes are the same.
I have helped the GSoC student to prepare for LLVM libfuzzer integration with the NetBSD base system. We have managed to get down to the following results for the test target in the upstream repository:
$ check-fuzzer-default Expected Passes : 105 Unsupported Tests : 8 Unexpected Failures: 2 $ check-fuzzer Expected Passes : 105 Unsupported Tests : 8 Unexpected Failures: 2 $ check-fuzzer-unit Expected Passes : 35
The remaining two failures appear to be false positives and specific to the differences between the NetBSD setup difference and other supported Operating Systems (including Linux). I have decided not to investigate them and instead to move on to more urgent tasks.
While there, I have been working on restoring a good state to userland LLVM sanitizers in the upstream repository, in order ship them in the NetBSD distribution along with the libfuzzer utility. While there, I've been working on restoring a good state with userland LLVM sanitizers in the upstream repository, in order ship them into the NetBSD distribution, along the libfuzzer utility.
A number of patches were merged upstream:
There is also at least a single pending upstream patch that is worth to note: Introduce CheckASLR() in sanitizers
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD. Introduce a generic CheckASLR() routine, that implements a check for the current process. This flag depends on the global or per-process settings. There is no simple way to disable ASLR in the build process from the level of a sanitizer or during the runtime execution. With ASLR enabled sanitizers that operate over the process virtual address space can misbehave usually breaking with cryptic messages. This check is dummy for !NetBSD.
The current results for test targets in the compiler-rt features are as follows:
$ make check-builtins Expected Passes : 343 Expected Failures : 4 Unsupported Tests : 36 Unexpected Failures: 5 $ check-interception -- Testing: 0 tests, 0 threads -- $ check-lsan Expected Passes : 6 Unsupported Tests : 60 Unexpected Failures: 106 $ check-ubsan Expected Passes : 229 Expected Failures : 1 Unsupported Tests : 32 Unexpected Failures: 2 $ check-cfi Unsupported Tests : 232 $ check-cfi-and-supported BaseException: Tests unsupported $ make check-sanitizer Expected Passes : 576 Expected Failures : 13 Unsupported Tests : 206 Unexpected Failures: 31 $ check-asan Expected Passes : 852 Expected Failures : 4 Unsupported Tests : 440 Unexpected Failures: 16 $ check-asan-dynamic Expected Passes : 394 Expected Failures : 3 Unsupported Tests : 440 Unexpected Passes : 1 Unexpected Failures: 222 $ check-msan Expected Passes : 102 Expected Failures : 1 Unsupported Tests : 30 Unexpected Failures: 4 $ check-tsan Expected Passes : 288 Expected Failures : 1 Unsupported Tests : 84 Unexpected Failures: 8 $ check-safestack Expected Passes : 7 Unsupported Tests : 1 $ check-scudo Expected Passes : 14 Unexpected Failures: 28 $ check-ubsan-minimal Expected Passes : 6 Unsupported Tests : 2 $ check-profile Unsupported Tests : 116 $ check-xray Expected Passes : 21 Unsupported Tests : 1 Unexpected Failures: 21 $ check-shadowcallstack Unsupported Tests : 4
I am helping to setup the process for shipping a NetBSD userland that is prebuilt with a desired sanitizer. This involves consulting the Google Summer of Code student, fixing known issues, reviewing patches etc.
There were two new uninitialized memory read bugs detected in the top(1) program:
Fix unitialized signal mask passed to sigaction(2) in top(1) Detected with Memory Sanitizer during the integration of sanitizers with the NetBSD basesystem. Reported by <Yang Zheng>
Fix read of uni[ni]tialized array elements in top(1) The cp_old array is allocated with malloc(3) and its pointer is passed to percentages64(). In this function there happens a calculation of total_change, which value depends on the value inside the unitialized cp_old[] array. ==26662==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x268a2c in percentages64 /usr/src/external/bsd/top/bin/../dist/machine/m_netbsd.c:1341:6 #1 0x26748b in get_system_info /usr/src/external/bsd/top/bin/../dist/machine/m_netbsd.c:478:6 #2 0x25518e in do_display /usr/src/external/bsd/top/bin/../dist/top.c:507:5 #3 0x253038 in main /usr/src/external/bsd/top/bin/../dist/top.c:975:2 #4 0x21cad1 in ___start (/usr/bin/top+0x1cad1) SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/src/external/bsd/top/bin/../dist/machine/m_netbsd.c:1341:6 in percentages64 Exiting Fix this issue by chang[]ing malloc(3) with calloc(3). Detected with Memory Sanitizer during the integration of sanitizers with the NetBSD basesystem. Reported by <Yang Zheng>
As similar process happens with two kernel sanitizer GSoC tasks: kernel-ubsan and kernel-asan.
Thanks to the involvement to The NetBSD Foundation tasks, I can be reachable for students (although not always in all cases) for active feedback and collaboration.
The number of ATF ptrace(2) tests cases has been significantly incremented, however there is still a susbstantial amount of work to be done and a number of serious bugs to be resolved.
With fixes and addition of new test cases, as of today we are passing 1,206 (last month: 961) ptrace(2) tests and skipping 1 (out of 1,256 total; last month: 1,018 total). No counted here tests that appeared outside the ptrace(2) context.
Cover with regression tests remaining elementary scenarios of handling crash signals. Fix known bugs in the NetBSD kernel.
Follow up the process with the remaining fork(2) and vfork(2) scenarios.
The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: