Misc developer notes

About this file

If some nugget of information is useful enough for NetBSD developers to know - list it here.

Logs when bumping a kernel version number or an shlib_version file

When one bumps either the kernel version number or an shlib_version file, please include both the interface changed and the new version number in the log message. Otherwise it's hard to figure out what a particular version bump means by reading the cvs log.

Sharing userland and MACHINE_ARCH

Userland should be shareable between machines of the same MACHINE_ARCH. Some obvious issues:

  • MAXPARTITIONS: Should be determined at runtime using sysctl(3).
  • Machine specific binaries, keymaps, etc.: Should fail gracefully on different MACHINE types, and be included on all MACHINE_ARCH systems.
  • Differing memory architectures: In the case where the memory architectures differs sufficiently to need different libkvm routines, they should all be included in kvm(3) and the version to use determined at runtime.
  • disklabels and boot blocks: This is the largest outstanding area to be addressed. Volunteers welcome.

Adding a new library

The following is intended to be a brief list of items that need to be addressed when adding a new library to the NetBSD tree. In the following example pathnames, foo is the name of the library, 0 is the library's major number and 0 is the library's minor number. A new library would normally have 0 for both the major and minor number.

  • Place the source files (including header files to be installed, Makefile and shlib_version) into a libfoo directory in the CVS repository, using cvs add or cvs import, depending on which is more appropriate e.g. due to external maintenance of the product.
  • Add the libfoo subdirectory to the Makefile's SUBDIRS list one level up in the hierarchy.
  • Add the full shared object name (./usr/lib/libfoo.so.0.0) to src/distrib/sets/lists/base/shl.mi.
  • Add the PIC library (./usr/lib/libfoo_pic.a) to src/distrib/sets/lists/comp/shl.mi.
  • Add the ELF shared object names with no suffix (./usr/lib/libfoo.so) and a single numeric suffix (./usr/lib/libfoo.so.0) to src/distrib/sets/lists/base/shl.elf.
  • Add the lint library (./usr/libdata/lint/llib-lfoo.ln) to src/distrib/sets/lists/comp/lint.mi.
  • Add the normal library (./usr/lib/libfoo.a), profiled library (./usr/lib/libfoo_p.a) and any headers and manual pages to src/distrib/sets/lists/comp/mi.
  • Add 
       LIBFOO?= ${DESTDIR}/usr/lib/libfoo.a
    to src/share/mk/bsd.prog.mk and to the DPADD section of src/share/mk/bsd.README.
  • Add FOO to the __bsd_prog_mk_system_libs variable in src/share/mk/bsd.hostprog.mk.
  • Update the DPADD list in src/share/mk/bsd.README
  • Add 
       .if "\\$1"libfoo" \&Foo Library (libfoo, -lfoo)
    to src/share/tmac/doc2html.
  • Add an entry to the .Lb libfoo macro in src/share/tmac/doc-syms.
  • Make sure that all files have $NetBSD$ RCS Ids.
  • Update src/doc/CHANGES.

Changing the version number of a library

The minor number of a library should be incremented when it is possible for external programs to call any new functions or variables added.

The major number should be incremented and the minor set to zero when an externally visible structure changed size, or function changed arguments. This can cause significant disruption and version renaming is frequently used to avoid this.

With the current C ABIs changing functions (previously void) to return a value of either integer, pointer, or floating-point type does not necessitate incrementing the major number since return values are always passed in (unconditionally) caller-saved registers. In such a case the minor number should be incremented to note the "additions".

If you bump the major of a library, you should:

  1. Bump the major of any library that depends on it.
  2. Send mail to warning about it, so that if anybody's built a shared library that depends on it they can know to do the right thing.
  3. Note it in the install notes, for people upgrading to the next release (or, presumably, to -current via binary snapshots).

Notes on syscall passing

Christos Zoulas writes: I'd like to mention for future reference, that our syscall argument passing mechanism does not handle passing structs or unions properly on all architectures, so syscall implementors need to convert those to pointers and provide stubs in userland that &struct or &union before calling the kernel stub.

Writing a manpage

  1. The mdoc.samples(7) and mdoc(7) manpages are excellent for reference (if a little lengthy).
  2. Work from the template in /usr/share/misc/mdoc.template, and refer to an existing manpage (for example who(1) for a command, and err(3) for a function).
  3. When done, remember to add the formatted (/usr/share/man/mansection/name.section) and formatted (/usr/share/man/catsection/name.0) to src/distrib/sets/lists/man/mi.

Renaming or removing installed binaries

If your change filename, or remove any of files installed to DESTDIR, you will need to do something like the following. You need to decide which list file to update, based on the file you will be removing.

  1. Find the old filename (or removed filename) from src/distrib/sets/lists/base/mi (or relevant file).
  2. Add keyword obsolete to the old filename (or removed filename) as third keyword. This is for upgrade installations.
  3. And of course, don't forget to add new filename to src/distrib/sets/lists/base/mi.

Cleaning your tree

Perry E. Metzger writes: At periodic intervals, I find that my checked out tree gets filled with garbage that cvs update normally doesn't "?" because it is in the ignored list -- stuff like .orig files from patching runs or .o files for things that don't exist any more. Since CVS ignores them, one never finds out one's tree has the junk in it.

I'm sure I'm the last person to discover this trick, but in case I'm not, I have found that doing 

 cvs update -I \! -I CVS

nicely yields lots of good complaints about left-over garbage like this.

When to add weak-symbol to library functions?

In short, to quote Klaus Klein:

  • the identifier is to be used internally by libc, and
  • the identifier is not a feature of ANSI C resp. C90.

The weak symbol reference has two goals:

  • Users should be allowed to define a function "foo", if "foo" is not defined in ANSI C, and the name is used in any of OS-supplied library.
  • A library function may call other library functions. In this case, the call must be made safe against name collision with user-supplied functions. Therefore, we need to use "_foo", not "foo", to refer other library functions from within the library function. If we call "foo", "foo" can be overridden by user-supplied function and may result in strange behavior.

What files need updating on each Jan 1st?

htdocs/layout.xml needs to be updated for the new year. It also needs to have a <new year> entry added for donations and a <prior year> entry added for changes. It's a good idea to update this file before making other changes to htdocs as this file is referenced by everything else.

On the first of each year, the following files need to be updated to list the new year in the Copyright strings

  • src/sys/conf/copyright
    • this should be pulled up to all active branches
  • localsrc/admin/copyright/templates/*
  • htdocs/index.html
  • htdocs/about/disclaimer.xml
  • htdocs/docs/guide/en/netbsd.xml
  • pkgsrc/doc/guide/files/pkgsrc.xml

In htdocs/changes, you need to copy index.xml to \<prior year\>.xml. In \<prior year\>.xml adjust webpage id attribute to changes-\<prior year\> instead of changes-index. Then in index.xml add a link to \<prior year\>.html and delete the prior year items. Then add the prior year to Makefile.

In htdocs/donations, you need to add new files for the new year and update index.xml and Makefile to build and display the new year .html file.

In htdocs/docs/software/index.xml, you need to add the new year to the "Changes to the Packages Collection" item.

In pkgsrc/doc, you need to add a new file for CHANGES-YYYY.

On the main wiki, in the archives directory, a new directory and files need to be added for the new year. On both the main wiki and the pkgsrc wiki, templates/page.tmpl needs to be updated for the new copyright year.

Update footer.part and header.part in localsrc/releng/www/htdocs-proto. These will be automatically copied to the active website.

What is the job of a mailing list moderator?

A short summary by Herb Peyerl: You have to accept and read all bounce messages from the lists you own to make sure they're not legitimate postings. You then forward all legitimate postings that have bounced to the list in question using the majordomo 'approve' script. You also have to do any subscribes or unsubscribes from people whose email addresses didn't match exactly the email address they subscribed with.

In the case of moderated lists (-announce, -news) you need to make sure the postings match the charter of the list(s) and approve them if they are appropriate, or reply to the author and explain why you can't approve their postings.

You'll also get keyword bounces which are usually spam (that you have to read) and if they're legitimate postings, determine where the keyword match was and decide whether it was legitimate and forward it. If not, communicate back to the author to ask him/her to rephrase their posting.

The above task becomes fairly onerous during virus season.