Chapter 14. Buildlink methodology

Table of Contents

14.1. Converting packages to use buildlink3
14.2. Writing files
14.2.1. Anatomy of a file
14.2.2. Updating BUILDLINK_API_DEPENDS.pkg and BUILDLINK_ABI_DEPENDS.pkg in files
14.3. Writing files
14.3.1. Anatomy of a file
14.3.2. Global preferences for native or pkgsrc software

Buildlink is a framework in pkgsrc that controls what headers and libraries are seen by a package's configure and build processes. This is implemented in a two step process:

  1. Symlink headers and libraries for dependencies into BUILDLINK_DIR, which by default is a subdirectory of WRKDIR.

  2. Create wrapper scripts that are used in place of the normal compiler tools that translate -I${LOCALBASE}/include and -L${LOCALBASE}/lib into references to BUILDLINK_DIR. The wrapper scripts also make native compiler on some operating systems look like GCC, so that packages that expect GCC won't require modifications to build with those native compilers.

This normalizes the environment in which a package is built so that the package may be built consistently despite what other software may be installed. Please note that the normal system header and library paths, e.g. /usr/include, /usr/lib, etc., are always searched -- buildlink3 is designed to insulate the package build from non-system-supplied software.

14.1. Converting packages to use buildlink3

The process of converting packages to use the buildlink3 framework (bl3ifying) is fairly straightforward. The things to keep in mind are:

  1. Ensure that the build always calls the wrapper scripts instead of the actual toolchain. Some packages are tricky, and the only way to know for sure is the check ${WRKDIR}/.work.log to see if the wrappers are being invoked.

  2. Don't override PREFIX from within the package Makefile, e.g. Java VMs, standalone shells, etc., because the code to symlink files into ${BUILDLINK_DIR} looks for files relative to pkg_info -qp pkgname.

  3. Remember that only the files that you list in a package's Makefile are added as dependencies for that package.

If a dependency on a particular package is required for its libraries and headers, then we replace:

DEPENDS+=       foo>=1.1.0:../../category/foo


.include "../../category/foo/"

The files usually define the required dependencies. If you need a newer version of the dependency when using files, then you can define it in your Makefile; for example:   foo>=1.1.0
.include "../../category/foo/"

There are several files in pkgsrc/mk that handle special package issues:

  • chooses either the native or a pkgsrc Berkeley DB implementation based on the values of BDB_ACCEPTED and BDB_DEFAULT.

  • If the system comes with neither Curses nor NCurses, this will take care to install the devel/ncurses package.

  • uses the value of KRB5_ACCEPTED to choose between adding a dependency on Heimdal or MIT-krb5 for packages that require a Kerberos 5 implementation.

  • checks for a system-provided Motif installation or adds a dependency on x11/lesstif, x11/motif or x11/openmotif. The user can set MOTIF_TYPE to dt, lesstif, motif or openmotif to choose which Motif version will be used.

  • checks for a system-provided GNU readline or editline (libedit) installation, or adds a dependency on devel/readline, devel/editline. The user can set READLINE_DEFAULT to choose readline implementation. If your package really needs GNU readline library, its Makefile should include devel/readline/ instead of

  • defines several variables that may be used by packages that use the Open Sound System (OSS) API.

  • will accept any of the Postgres versions in the variable PGSQL_VERSIONS_ACCEPTED and default to the version PGSQL_VERSION_DEFAULT. See the file for more information.

  • uses the value of PTHREAD_OPTS and checks for native pthreads or adds a dependency on devel/pth as needed.

  • uses the value of XAW_TYPE to choose a particular Athena widgets library.

The comments in those files provide a more complete description of how to use them properly.

14.2. Writing files

A package's file is included by Makefiles to indicate the need to compile and link against header files and libraries provided by the package. A file should always provide enough information to add the correct type of dependency relationship and include any other files that it needs to find headers and libraries that it needs in turn.

To generate an initial file for further editing, Rene Hexel's pkgtools/createbuildlink package is highly recommended. For most packages, the following command will generate a good starting point for files:

% cd pkgsrc/category/pkgdir
% createbuildlink >

14.2.1. Anatomy of a file

The following real-life example is taken from pkgsrc/graphics/tiff:

# $NetBSD: buildlink.html,v 1.121 2017/01/09 07:24:47 sevan Exp $

BUILDLINK_TREE+=        tiff

.if !defined(TIFF_BUILDLINK3_MK)

BUILDLINK_API_DEPENDS.tiff+=    tiff>=3.6.1
BUILDLINK_ABI_DEPENDS.tiff+=    tiff>=3.7.2nb1
BUILDLINK_PKGSRCDIR.tiff?=      ../../graphics/tiff

.include "../../devel/zlib/"
.include "../../graphics/jpeg/"

BUILDLINK_TREE+=        -tiff

The header and footer manipulate BUILDLINK_TREE, which is common across all files and is used to track the dependency tree.

The main section is protected from multiple inclusion and controls how the dependency on pkg is added. Several important variables are set in the section:

  • BUILDLINK_API_DEPENDS.pkg is the actual dependency recorded in the installed package; this should always be set using += to ensure that we're appending to any pre-existing list of values. This variable should be set to the first version of the package that had an backwards-incompatible API change.

  • BUILDLINK_PKGSRCDIR.pkg is the location of the pkg pkgsrc directory.

  • BUILDLINK_DEPMETHOD.pkg (not shown above) controls whether we use BUILD_DEPENDS or DEPENDS to add the dependency on pkg. The build dependency is selected by setting BUILDLINK_DEPMETHOD.pkg to build. By default, the full dependency is used.

  • BUILDLINK_INCDIRS.pkg and BUILDLINK_LIBDIRS.pkg (not shown above) are lists of subdirectories of ${BUILDLINK_PREFIX.pkg} to add to the header and library search paths. These default to include and lib respectively.

  • BUILDLINK_CPPFLAGS.pkg (not shown above) is the list of preprocessor flags to add to CPPFLAGS, which are passed on to the configure and build phases. The -I option should be avoided and instead be handled using BUILDLINK_INCDIRS.pkg as above.

The following variables are all optionally defined within this second section (protected against multiple inclusion) and control which package files are symlinked into ${BUILDLINK_DIR} and how their names are transformed during the symlinking:

  • BUILDLINK_FILES.pkg (not shown above) is a shell glob pattern relative to ${BUILDLINK_PREFIX.pkg} to be symlinked into ${BUILDLINK_DIR}, e.g. include/*.h.

  • BUILDLINK_FILES_CMD.pkg (not shown above) is a shell pipeline that outputs to stdout a list of files relative to ${BUILDLINK_PREFIX.pkg}. The resulting files are to be symlinked into ${BUILDLINK_DIR}. By default, this takes the +CONTENTS of a pkg and filters it through ${BUILDLINK_CONTENTS_FILTER.pkg}.

  • BUILDLINK_CONTENTS_FILTER.pkg (not shown above) is a filter command that filters +CONTENTS input into a list of files relative to ${BUILDLINK_PREFIX.pkg} on stdout. By default, BUILDLINK_CONTENTS_FILTER.pkg outputs the contents of the include and lib directories in the package +CONTENTS.

  • BUILDLINK_FNAME_TRANSFORM.pkg (not shown above) is a list of sed arguments used to transform the name of the source filename into a destination filename, e.g. -e "s|/curses.h|/ncurses.h|g".

This section can additionally include any needed for pkg's library dependencies. Including these files means that the headers and libraries for these dependencies are also symlinked into ${BUILDLINK_DIR} whenever the pkg file is included. Dependencies are only added for directly include files.

When providing a and including other files in it, please only add necessary ones, i.e., those whose libraries or header files are automatically exposed when the package is use.

In particular, if only an executable (bin/foo) is linked against a library, that library does not need to be propagated in the file.

The following steps should help you decide if a file needs to be included:

  • Look at the installed header files: What headers do they include? The packages providing these files must be buildlinked.

  • Run ldd on all installed libraries and look against what other libraries they link. Some of the packages providing these probably need to be buildlinked; however, it's not automatic, since e.g. GTK on some systems pulls in the X libraries, so they will show up in the ldd output, while on others (like OS X) it won't. ldd output can thus only be used as a hint.

14.2.2. Updating BUILDLINK_API_DEPENDS.pkg and BUILDLINK_ABI_DEPENDS.pkg in files

These two variables differ in that one describes source compatibility (API) and the other binary compatibility (ABI). The difference is that a change in the API breaks compilation of programs while changes in the ABI stop compiled programs from running.

Changes to the BUILDLINK_API_DEPENDS.pkg variable in a file happen very rarely. One possible reason is that all packages depending on this already need a newer version. In case it is bumped see the description below.

The most common example of an ABI change is that the major version of a shared library is increased. In this case, BUILDLINK_ABI_DEPENDS.pkg should be adjusted to require at least the new package version. Then the packages that depend on this package need their PKGREVISIONs increased and, if they have files, their BUILDLINK_ABI_DEPENDS.pkg adjusted, too. This is needed so pkgsrc will require the correct package dependency and not settle for an older one when building the source.

See Section 19.1.5, “Handling dependencies” for more information about dependencies on other packages, including the BUILDLINK_ABI_DEPENDS and ABI_DEPENDS definitions.

Please take careful consideration before adjusting BUILDLINK_API_DEPENDS.pkg or BUILDLINK_ABI_DEPENDS.pkg as we don't want to cause unneeded package deletions and rebuilds. In many cases, new versions of packages work just fine with older dependencies.

Also it is not needed to set BUILDLINK_ABI_DEPENDS.pkg when it is identical to BUILDLINK_API_DEPENDS.pkg.

14.3. Writing files

Some packages in pkgsrc install headers and libraries that coincide with headers and libraries present in the base system. Aside from a file, these packages should also include a file that includes the necessary checks to decide whether using the built-in software or the pkgsrc software is appropriate.

The only requirements of a file for pkg are:

  1. It should set USE_BUILTIN.pkg to either yes or no after it is included.

  2. It should not override any USE_BUILTIN.pkg which is already set before the file is included.

  3. It should be written to allow multiple inclusion. This is very important and takes careful attention to Makefile coding.

14.3.1. Anatomy of a file

The following is the recommended template for files:

.if !defined(
# is set to "yes" or "no" depending on whether "foo"
# genuinely exists in the system or not.
#        no

# should be set here if "foo" is built-in and its package
# version can be determined.
.  if !empty([yY][eE][sS])       foo-1.0
.  endif
.endif  #

.if !defined(       ${}
.  if defined(
.    for _depend_ in ${}
.      if !empty([yY][eE][sS])!=                                                       \
        ${PKG_ADMIN} pmatch '${_depend_}' ${}            \
        && ${ECHO} "yes" || ${ECHO} "no"
.      endif
.    endfor
.  endif
.endif  #     no
.if !empty([nN][oO])
# Here we place code that depends on whether is set to
# "yes" or "no".
.endif  #

The first section sets IS_BUILTIN.pkg depending on if pkg really exists in the base system. This should not be a base system software with similar functionality to pkg; it should only be yes if the actual package is included as part of the base system. This variable is only used internally within the file.

The second section sets BUILTIN_PKG.pkg to the version of pkg in the base system if it exists (if IS_BUILTIN.pkg is yes). This variable is only used internally within the file.

The third section sets USE_BUILTIN.pkg and is required in all files. The code in this section must make the determination whether the built-in software is adequate to satisfy the dependencies listed in BUILDLINK_API_DEPENDS.pkg. This is typically done by comparing BUILTIN_PKG.pkg against each of the dependencies in BUILDLINK_API_DEPENDS.pkg. USE_BUILTIN.pkg must be set to the correct value by the end of the file. Note that USE_BUILTIN.pkg may be yes even if IS_BUILTIN.pkg is no because we may make the determination that the built-in version of the software is similar enough to be used as a replacement.

The last section is guarded by CHECK_BUILTIN.pkg, and includes code that uses the value of USE_BUILTIN.pkg set in the previous section. This typically includes, e.g., adding additional dependency restrictions and listing additional files to symlink into ${BUILDLINK_DIR} (via BUILDLINK_FILES.pkg).

14.3.2. Global preferences for native or pkgsrc software

When building packages, it's possible to choose whether to set a global preference for using either the built-in (native) version or the pkgsrc version of software to satisfy a dependency. This is controlled by setting PREFER_PKGSRC and PREFER_NATIVE. These variables take values of either yes, no, or a list of packages. PREFER_PKGSRC tells pkgsrc to use the pkgsrc versions of software, while PREFER_NATIVE tells pkgsrc to use the built-in versions. Preferences are determined by the most specific instance of the package in either PREFER_PKGSRC or PREFER_NATIVE. If a package is specified in neither or in both variables, then PREFER_PKGSRC has precedence over PREFER_NATIVE. For example, to require using pkgsrc versions of software for all but the most basic bits on a NetBSD system, you can set:

PREFER_NATIVE=  getopt skey tcp_wrappers

A package must have a file to be listed in PREFER_NATIVE, otherwise it is simply ignored in that list.

Setting PREFER_NATIVE should be performed straight after bootstrap and PREFER_PKGSRC during bootstrap. Switching between settings globally at a later date can introduce complications with dependency resolution. This is caused by packages built with the opposite preference being installed alongside each other.

# ./bootstrap --prefer-pkgsrc yes