NetBSD version 9

Presenter Notes

What's new in store?

Presenter Notes

netbsd

Google Summer of Code 2019 Mentor Summit

Author: Kamil Rytarowski

E-mail: kamil@netbsd.org

Date: October 19th 2019

Place: Munchen Marriott Hotel, Munchen, Germany

Presenter Notes

Bio

Kamil Rytarowski (born 1987)

Krakow, Poland

NetBSD user since 6.1.

The NetBSD Foundation member (== developer) since 2015.

Work areas: kernel, userland, pkgsrc.

Interest: NetBSD on desktop and in particular NetBSD as a workstation.

The current activity in 3rd party software:

  • LLVM committer.
  • GDB & binutils committer.
  • NetBSD maintainer in qemu.

The first time mentor during GSoC 2018.

Presenter Notes

Topics

  • Lightning introduction to NetBSD
  • NetBSD and Google Summer of Code
  • Previous releases of NetBSD

NetBSD 9

  • hardware platforms
  • virtualization
  • quality improvement tools (sanitizers, fuzzers, debuggers, static code analysis)
  • security hardening
  • graphics
  • filesystems
  • networking stack and packet filters
  • other improvements, removals etc...

Presenter Notes

Lightning introduction to NetBSD

netbsd

NetBSD is a free, fast, secure, and highly portable Unix-like Open Source operating system.

It is available for a wide range of platforms, from large-scale servers and powerful desktop systems to handheld and embedded devices.

Presenter Notes

Lightning introduction to NetBSD

How does NetBSD compare to Linux?

  • Independent kernel and userland
  • Whole codebase maintained in a single source tree
  • BSD licensed

How does NetBSD compare to FreeBSD and OpenBSD?

  • Focus on portability to as many hardware platforms (ports) as possible
  • Clean code and portability over x86/x86_64 and performance driven development [FreeBSD]
  • Usability, features and rational security hardening over security driven development [OpenBSD]

Presenter Notes

Lightning introduction to NetBSD

PowerPC, Alpha, SPARC, MIPS, SH3, ARM, amd64, i386, m68k, VAX, RISCV ...

Of course it runs NetBSD.

Presenter Notes

Lightning introduction to NetBSD

Cross-building is possible from most UNIX-like operating systems.

1 ./build.sh \
2     distribution

Additional build information available in the BUILDING file.

Presenter Notes

Lightning introduction to NetBSD

Binaries.

Presenter Notes

Lightning introduction to NetBSD

Testing your NetBSD system with Automated Test Framework (ATF).

1 cd /usr/tests; atf-run | atf-report

Presenter Notes

Lightning introduction to NetBSD

Community support.

Presenter Notes

Lightning introduction to NetBSD

Fetch the latest sources.

To fetch the main CVS repository:

1 cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P src

GitHub mirror:

1 git clone https://github.com/netbsd/src

Alternatively use snapshots, Mercurial or Fossil mirrors.

Presenter Notes

Lightning introduction to NetBSD

Who uses NetBSD?

  • Commercial vendors
  • Researchers
  • Hobbyists

Presenter Notes

Lightning introduction to NetBSD

For additional introduction check The NetBSD Guide.

https://www.netbsd.org/docs/guide/en/

Presenter Notes

NetBSD and Google Summer of Code

Presenter Notes

NetBSD and GSoC

NetBSD participated successfully in the following Google's Summer of Code programs:

  • 2005-2013

and

  • 2016-2019.

Presenter Notes

NetBSD and GSoC

In 2019 there were 7 succesful projects for The NetBSD Foundation.

Check https://blog.netbsd.org/ for their reports.

I've mentored 2 students this year.

  • Siddharth Muralee (India) "Kernel fuzzing with syzkaller"
  • Akul Pillai (India) "Kernel fuzzing with TriforceAFL"

Presenter Notes

Kernel Fuzers - Syzkaller

Syzkaller + syzbot 24/7 fuzzing with KCOV assisted coverage driver

syzbot

http://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd http://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd http://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd1

GSoC 2019 by Siddharth Muralee

Presenter Notes

Kernel Fuzers - TriforceAFL

triforceafl_2019_06_26.png

http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1 http://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part2

GSoC 2019 by Akul Pillai

Presenter Notes

Previous releases of NetBSD

Presenter Notes

Previous releases

Prehistory

  • [1965] Multics
  • [1969] UNIX First Edition
  • [1977] 1BSD
  • [1993] BSD/386 1.0
  • [1993] NetBSD 0.8
  • [1994] NetBSD 1.0
  • [1995] 4.4BSD Lite Release 2
  • [1995] NetBSD 1.1

-- /usr/share/misc/bsd-family-tree

Presenter Notes

Previous releases

Modern times

  • [2009] NetBSD 5.0
  • [2012] NetBSD 6.0
  • [2015] NetBSD 7.0
  • [2018] NetBSD 8.0
  • [2019] NetBSD 8.1

-- /usr/share/misc/bsd-family-tree

Presenter Notes

NetBSD 8.0

  • Released July 17 2018

https://www.netbsd.org/releases/formal-8/NetBSD-8.0.html

  • 1410 resolved pullups to netbsd-8 as of Oct 18th 2019

http://releng.netbsd.org/cgi-bin/req-8.cgi

Presenter Notes

NetBSD 9.0

NetBSD 9.0 release process has started on July 31st 2019

  • 350 resolved pullups to netbsd-9 as of Oct 18th 2019

http://releng.netbsd.org/cgi-bin/req-9.cgi

The NetBSD Foundation hires now a release engineering developer (martin@).

Presenter Notes

Hardware platforms

New AArch64 (ARM64) architecture support:

  • Symmetric and asymmetrical multiprocessing support (aka big.LITTLE)
  • Support for running 32-bit binaries
  • UEFI and ACPI support
  • Support for SBSA/SBBR (server-class) hardware.

The FDT-ization of many ARM boards:

  • the 32-bit GENERIC kernel lists 129 different DTS configurations
  • the 64-bit GENERIC64 kernel lists 74 different DTS configurations
  • All supported by a single kernel, without requiring per-board configurations.

Presenter Notes

Hardware platforms

Selection of new boards:

  • Olimex A10-OLinuXino-LIME
  • Olimex A13-OLinuXino/-MICRO
  • C.H.I.P./Pro/Pocket
  • Pine64 / PineBook / Its / Sophine / Rock64 / ROCKPro64
  • FriendlyARM NanoPi NEO2 / NEO Pluse2
  • Orange Pi One Plus, Pine H64
  • SoftIron Overdrive 1000
  • nvidia TXI
  • ODROID-XU3/XU4
  • Exynos 5422
  • many others....

Presenter Notes

Hardware platforms

RISCV getting closer, boots in emulator.

Still no real hardware support.

Presenter Notes

Hardware platforms

New device drivers for various devices on older hardware, such as:

  • cpuctl for NetBSD/sparc and NetBSD/sparc64,
  • Milan for NetBSD/atari,
  • better Mac G5 NetBSD/macppc support,
  • VAXstation 4000 TURBOchannel support

Presenter Notes

Virtualization

Oldschool NetBSD/Xen is still maintained.

NetBSD was the first Operating System shipping with Xen support.

New approached with kernel backends (Linux KVM-style) for qemu-alike frontend:

  • NVMM (Intel and AMD x86_64)
  • HAXM (Intel x86_64 and potentially i386)

Presenter Notes

Wine64

WoW (Wine32 + Wine64) improvements.

Presenter Notes

Sanitizers

Sanitizer is a programming tool that detects computer program bugs such as:

  • buffer overflows,
  • signed integer overflow,
  • uninitialized memory read,
  • data races etc.

Presenter Notes

Types of sanitizers

The fundamental five types of sanitizers:

  • Address Sanitizer (Asan) - Finds invalid address usage bugs.
  • Undefined Behavior Sanitizer (UBSan) - Finds unspecified code semantics bugs.
  • Thread Sanitizer (TSan) - Finds threading bugs.
  • Memory Sanitizer (MSan) - Finds uninitialized memory read.
  • Leak Sanitizer (LSan) - Finds memory leaks.

Additionally:

  • libFuzzer - Library fuzzer
  • SafeStack - Stack hardening
  • XRay - Function call tracer

Presenter Notes

Types of sanitizers in NetBSD

All of them main five userland sanitizers are supported on NetBSD.

  • ASan (amd64, i386, other ports in various degree)
  • UBSan (all ports)
  • MSan (amd64)
  • TSan (amd64)
  • LSan (amd64, i386, other ports in various degree)

NetBSD pass 95% of the upstream LLVM tests.

There are sanitizers available in the NetBSD kernel.

  • kASan (amd64, aarch64)
  • kUBSan (all ports)
  • kMSan (amd64 - finished, still not merged, waiting for LLVM upgrade)
  • kTSan is WIP
  • KLEAK (amd64)

Presenter Notes

MKSANITIZER

MKSANITIZER is a distinct feature of NetBSD that allows the whole distribution sanitization.

Presenter Notes

MKSANITIZER fuzzing

rumpkernel is a NetBSD kernel code as a library.

rumpkernel code fuzzing with MKSANITIZER and honggfuzz.

 1 ------------------------[  0 days 01 hrs 00 mins 00 secs ]-------
 2   Iterations : 367,977 [367.98k]
 3         Mode : [2/2] Feedback Driven Mode
 4       Target : ./a.out
 5      Threads : 4, CPUs: 8, CPU%: 0% [0%/CPU]
 6        Speed : 25/sec [avg: 102]
 7      Crashes : 7817 [unique: 7817, blacklist: 0, verified: 0]
 8     Timeouts : 13 [10 sec]
 9  Corpus Size : 203, max size: 5,120,000 bytes, init dir: 361 files
10   Cov Update : 0 days 00 hrs 00 mins 29 secs ago
11     Coverage : edge: 28 pc: 206 cmp: 104,650
12 ---------------------------------- [ LOGS ] ----/ honggfuzz 1.7 /-

Status after 60 minutes of fuzzing.

Presenter Notes

Debuggers

Kernel correctness and completeness of features in ptrace(2) is highly improved for debuggers (lldb, gdb), syscall tracers (strace-like) and sanitizers (lsan).

The LLVM LLDB support is getting closer to being accomplished and fully featured.

GNU GDB support is improving (gdbserver is now available, many bugs are gone).

Presenter Notes

Static code analysis

  • LGTM bot

Presenter Notes

Security hardening

  • Intel CPU bugs mitigation
  • Kernel ASLR

Presenter Notes

Graphics

Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices.

Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU.

Presenter Notes

Filesystems

ZFS has been updated to a modern version and seen many bugfixes.

Presenter Notes

Networking stack and packet filters

Further SMP-ification of the codebase.

NPF is now the recommended packet filter for new users and IPF/PF are deprecated.

NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default

The NetBSD Foundation funded project to port FreeBSD wifi stack (not merged yet).

Presenter Notes

Other improvements

  • Upgrades to GCC 7.x, LLVM 7.x
  • Native Command Queuing (NCQ) support
  • Installer improvements (GPT, UEFI)
  • Performance Metric Counters (PMC) for amd64 and aarch64
  • Modularization of compat modules
  • DTrace update and bugfixes
  • new syscall tracers (picotrace, NetBSD truss, singlestepper)
  • ...

Presenter Notes

Removals

  • NetBSD/acorn26
  • ipkgdb - IP-based kernel debugger
  • vm86
  • isdn4bsd
  • NDIS wraper
  • EtherIP - Ethernet over IP
  • ...

Presenter Notes

Q/A

Presenter Notes