Problem solved!

The problem described below has been resolved by this commit on August 9, 2018. The fix has been pulled up to the netbsd-8 branch on the same day and will be in NetBSD 8.1.

Investigating the performance of a build cluster slave

While building a netbsd-7 alpha release on a build cluster slave wtih build.sh -j 24 we took several 1 minute dtrace probes with 997 Hz sample frequency and converted the kernel stacks recorded to flamegraphs. The names of the graphs are time of the sample in UTC.

The build slave is running a 8.0_STABLE kernel from 20180804 and 8.0 userland. Initial tests hint at -current having the same overal timings, while netbsd-7 is significantly faster.

Most graphs look nearly identical.

Here is a generic sample:

17:16

The same graph with the right side (everything above the idle loop) removed:

17:16 w/o idle

Other graphs for reference: