LLVM santizers and GDB regression test suite
<p>
As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration.
This work is now accomplished and with MKLLVM=yes build option (by default off),
the distribution will be populated with LLVM files for
ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.
<p>
I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.
<p>
<h1>NetBSD distribution changes</h1>
<p>
I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible.
Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state.
I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).
<p>
The following changes were committed to the sources:
<ul>
<li>ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers</li>
<li>966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build</li>
<li>8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data</li>
<li>fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data</li>
<li>40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER</li>
<li>b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER</li>
<li>c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER</li>
<li>030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER</li>
<li>fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER</li>
<li>5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data</li>
<li>5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot</li>
<li>d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk</li>
<li>2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER</li>
<li>ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7</li>
<li>7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7</li>
<li>d8671fba7a78 Set NODEBUG for LLVM sanitizers</li>
<li>242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER</li>
<li>5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers</li>
<li>e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers</li>
<li>231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers</li>
<li>8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers</li>
<li>81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers</li>
<li>a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c</li>
<li>7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer</li>
<li>921ddc9bc97c Set NOSANITIZER in i386 ramdisk image</li>
<li>64361771c78d Enhance MKSANITIZER support</li>
<li>3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests</li>
<li>c27f4619d513 Avoids signedness bit shift in db_get_value()</li>
<li>680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)</li>
<li>4ecfbbba2f2a Rework the LLVM compiler_rt build rules</li>
<li>748813da5547 Correct the build rules of LLVM sanitizers</li>
<li>20e223156dee Enhance the support of LLVM sanitizers</li>
<li>0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files</li>
</ul>
<p>
Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.
<p>
As a demo, I have crafted a writing on combining RUMPKERNEL, MKSANITIZER with the honggfuzz fuzzer:
<a href="//netbsd.org/~kamil/rump/rump_pub_etfs_register_buffer.c">Rumpkernel assisted fuzzing of the NetBSD file system kernel code in userland</a>.
<p>
<h1>GDB</h1>
I've merged NetBSD distribution downstream GDB patches into my local GDB tree and executed the regression tests (check-gdb):
<p>
<pre>
[...]
Test run by kamil on Mon Sep  2 12:36:03 2019
Native configuration is x86_64-unknown-netbsd9.99

                === gdb tests ===

Schedule of variations:
    unix

[...]
                === gdb Summary ===

# of expected passes            54591
# of unexpected failures        3267
# of expected failures          35
# of unknown successes          3
# of known failures             59
# of unresolved testcases       29
# of untested testcases         141
# of unsupported tests          399
</pre>
<p>
Full log is
<a href="http://netbsd.org/~kamil/gdb/test-gdb.patched-8.3.50.20190716-git.gdb.sum">here</a>.
<p>
This means that there are a lot of more tests and known failures than in 2017-09-05:
<pre>
$ uname -a
NetBSD chieftec 8.99.2 NetBSD 8.99.2 (GENERIC) #0: Sat Sep  2 22:55:29 CEST 2017  root@chieftec:/public/netbsd-root/sys/arch/amd64/compile/GENERIC amd64

Test run by kamil on Tue Sep  5 17:06:28 2017
Native configuration is x86_64--netbsd

                === gdb tests ===

Schedule of variations:
    unix

[...]
                === gdb Summary ===

# of expected passes            16453
# of unexpected failures        483
# of expected failures          9
# of known failures             28
# of unresolved testcases       17
# of untested testcases         41
# of unsupported tests          25
</pre>
<p>
There are actually some regressions and a set of tests that fails probably due to environment differences like lack of gfortran at hand.
<p>
Full log is
<a href="http://netbsd.org/~kamil/gdb/test-gdb.base-8.0.50.20170905-git.gdb.sum">here</a>
<p>
<h1>GSoC Mentoring</h1>
<p>
The Google Summer of Code programme reached the end.
My mentees wrote successfully their final reports:
<ul>
<li><a href="//blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd">
Enchancing Syzkaller Support for NetBSD, Part 3</a></li>
<li><a href="//blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part2">
Adapting TriforceAFL for NetBSD, Part 3</a></li>
</ul>
<p>
I'm also mentoring the AFL+KCOV work by Maciej Grochowski.
Maciej will visit EuroBSDCon-2019 and
<a href="https://2019.eurobsdcon.org/talk-speakers/#AFL">speak</a> about his work.
<p>
<h1>Add methods for setting and getting the thread name</h1>
<p>
I've reached out to the people from standards bodies and I'm working on defining the standard approach for setting and getting the thread name.
I have received a proper ID of my proposal and I'm now supposted to submit the text in either PDF or HTML format.
<p>
This change will allow to manage the thread name with an uniform interface on all comforming Operating Systems.
<p>
<h1>Plan for the next milestone</h1>
<p>
Keep enhancing GDB support.
Keep detecting ptrace(2) bugs and addressing them.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any 
donations to help us continue funding projects and services to the 
open-source community. Please consider visiting the following URL to 
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>