NetBSD/cobalt Restore CD HOWTO

This document attempts to explain how to create and use a NetBSD Restore CD for Cobalt Qube/Raq devices. It was originally written by Alex Pelts <alexp@broadcom.com> and posted to the port-cobalt mailinglist, then updated by Izumi Tsutsui <tsutsui@NetBSD.org> for the newer 5.0 restorecd script.

Introduction

Disclaimer (top)

The author does not accept any liability for your use of this document. It is possible that your use of this document will render your computer useless, damage your hardware, software, break your window or your relationship with a loved one. Use the contents of this document at your own risk. This document is just the authors opinion on how to use hardware and software mentioned herein.

You have been warned.

Copyright Information (top)

This document is copyrighted (c) 2005 Alex Pelts and also copyrighted (c) 2008, 2009 Izumi Tsutsui, and distributed under the terms of the OpenContent License (OPL). The full text of the license can be found at http://www.opencontent.org/opl.shtml .

What is a Cobalt Qube/Raq and what is a Restore CD?

What is a Cobalt Qube/Raq? (top)

The Cobalt Qube/Raq is a server appliance. To put it simply, they are just computers without keyboard and monitor and without the ability to attach one. There are several versions of Qubes and Raqs in existence, older MIPS based and newer AMD-K6 based. This document covers only MIPS based server appliances. Further in the document both appliances will be referred to as "Qube". There are differences between the Qube and the Raq but for the purpose of this document they are irrelevant.

What is a Restore CD? (top)

Because the Qube does not have an easy way to attach a monitor, a keyboard and most importantly a CDROM drive, there is no easy way to install a new operating system on it. This is where the Restore CD comes in. The Qube has the ability to boot an operating system over the network. A Restore CD provides the environment allowing the Qube to boot an operating system and perform an unattended install of the operating system onto hard drive. An original Linux Restore CD was provided with the device. ISO image of this CD can still be downloaded from Sun's web site.

Based on the original Cobalt Restore CD, Dennis Chernoivanov created a NetBSD Restore CD for versions 1.6 and 1.6.1. He then published the scripts he used to create CD and he also updated the script for 3.0 release and 3.99.xx -current. After that his restorecd script has been updated and improved by Izumi Tsutsui for 4.0 and 5.0_BETA.

This document describes how to create a Restore CD for version 5.0 (and later) from scratch and how to use it.

Hardware and Software Requirements

Hardware Requirements (top)

To create and use a Restore CD you will need following hardware:

MIPS based Qube or RaQ with at least 1GB hard drive installed.
x86 based PC or Laptop with NetBSD supported Ethernet adapter and bootable CDROM drive.
Crossover Ethernet cable or hub with two straight cables.
RS232 Serial cable could be very helpful to diagnose problems.
CD Recorder to create your final CD.

Software Requirements (top)

To create a Restore CD you will need following software:

POSIX-compliant systems which are capable to build NetBSD from sources. Any of the following should work - NetBSD, FreeBSD, Linux, Cygwin etc.
wget or rsync to retrieve all new installation binary and source sets.
Typical unix tools including awk, basename, cat, chmod, cp, dirname, echo, gzip, ln, mkdir, mv, patch, pwd, rm, sed, sh, tar, and touch etc. These usually come with your OS.
More commands to build NetBSD toolchains, including make, cc (or gcc), as, ld, and cmp (required by configure scripts) etc. Usually these also come with your OS.

Steps to create a new Restore CD

Prepare fetch command (top)

The restorecd script requires wget or rsync to fetch necessary NetBSD release files for restorecd. If you are using a NetBSD host you can install them via Packages Collection by the following commands:

# PKG_PATH=ftp://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/i386/5.0/All	# use appropriate port and version for your build host
# export PKG_PATH
# pkg_add -v wget
# pkg_add -v rsync

Download restorecd script (top)

Download the NetBSD Restore CD scirpt and extract it. It is located on the NetBSD ftp site.

Edit Makefile (top)

The restorecd script includes a simple Makefile to create a Restore CD, but there is no automated “configure” mechanism so you might have to modify the Makefile for your host system and environments.

Modify WGET= or RSYNC= lines per where these commands are installed on your host:

WGET?=		/usr/pkg/bin/wget
#WGET=		/usr/bin/wget
#WGET=		/usr/local/bin/wget

RSYNC?= 	/usr/pkg/bin/rsync
#RSYNC= 	/usr/bin/rsync
#RSYNC= 	/usr/local/bin/rsync

If you will use daily snapshot binaries to build restorecd, check Summary of daily snapshot builds page and modify DAILY_DIR?= line to point an appropriate daily directory:
```
DAILY_DIR?=	200904280000Z
```
If you use mirror sites rather than ftp.NetBSD.org or rsync.NetBSD.org, set your mirror host to FTP_HOST= line for wget:
```
FTP_HOST?=	ftp.NetBSD.org
#FTP_HOST=	ftp.jp.NetBSD.org
#FTP_HOST=	ftp5.jp.NetBSD.org 
```
or RSYNC_HOST= line for rsync:
```
RSYNC_HOST?=	rsync.NetBSD.org
#RSYNC_HOST=	rsync.jp.NetBSD.org
RSYNC_PREFIX?=

#RSYNC_HOST=	rsync3.jp.NetBSD.org
#RSYNC_PREFIX=	pub/ 
```
If your rsync mirror have different module names from rsync.NetBSD.org (i.e. its rsync URL has trailing directory path after the hostname), you also have to adjust RSYNC_PREFIX= line.
If you use rsync, you have to change the following lines:
```
${DONE_FETCH}:
	${MKDIR} -p ${DOWNLOADDIR}
#	${MAKE} fetch_wget
	${MAKE} fetch_rsync
	${TOUCH} ${DONE_FETCH}
```
i.e. comment out “${MAKE} fetch_wget” line and uncomment “${MAKE} fetch_rsync” line.

Creating your new Restore CD (top)

Once your fetch command and Makefile are prepared, just type “make”.

This will take some time to execute and after it is finished you will have a file “cd.tmp/restorecd.iso” which is your new Restore CD image.

You can burn this image onto a CD using any software that can record ISO images. On UNIX you can use cdrecord, on Windows use Nero Burning Rom or any other ISO image compatible software.

Congratulations! Enjoy your custom NetBSD Restore CD.

Installation with the Restore CD

Installation Overview (top)

The installation instructions below will in order perform the following steps:

Boot the assisting PC from the Restore CD. This will enable a dhcp server on the PC and other services which the server appliance will need during installation.
(re)boot the Cobalt server appliance in a state where it is prepared to retreive the restore CD contents via network
Re-initialize the Cobalt server appliance from the assisting PC's restore CD

How to Use Your CD (top)

Warning: THIS INSTALLATION WILL DESTROY THE EXISTING CONTENTS OF YOUR COBALT HARDDISK DRIVE. Ensure you have sufficient backups before starting the installation if you need.

Note: on the other hand, restorecd doesn't use harddisk on the i386 server at all (it works even if the server has no hard drive).

Connect your i386 server to your cobalt using ether a crossover cable or hub/switch.

Note: for security reason, it's recommended not to coneect any other machines than your i386 server and cobalt appliance on the same network.
Check your i386 server's manuals to see how to boot from its CDROM drive, then boot it from the Restore CD. An instruction document will be shown by less command during boot process, so use space or return to read and type “q” to quit it. If “login:” prompt appeares the i386 server is ready for installation and no further settings are required on the server.

Note: if your machine has some trouble during boot around ACPI, try alternative kernel settings as shown on the boot menu.
Turn on the cobalt appliance while simultaneously pressing and holding down both the left (<) and right (>) arrow buttons on the LCD console to boot cobalt via network.
Continue holding down both the left (<) and right (>) arrow buttons until “Net booting” appears on the LCD display,
```
Net booting
          
```
then release the buttons.
Wait until the box boots up. Note most boot processes are shown on the LCD display:
```
NetBSD/cobalt
Bootloader
```
```
Loading:
nfs:netbsd
```
```
NetBSD/cobalt
Starting up...
```
```
[Starting up]
  Create /dev/*
```
```
[Starting up]
  > network
```
If it's ready, “restorecd ready” message and IP address assigned by the i386 server appear on the LCD screen:
```
restorecd ready
[10.0.0.160]
```
Note: if you need remote login access for recovery without restoring, you can login to your cobalt from the i386 server via network at this point.
To start restorecd operation, push and hold the “Select” button two seconds to enable paneld menu. Once the menu appears on the LCD display, select “Restore” menu by the Select button,
```
select:
  Restore
```
select “Y” (which is default) by the arrow buttons on the confirm menu,
```
Start restore?
  [Y]es / [N]o
```
and then push the “Enter” button to start installation.

Warning: again, whole disk contents on the cobalt appliance will be destroyed once installation is started.

During installation, whole procedure will appear on the LCD display and serial console.
```
Disk Setup:
  Reset disk
```
```
Disk Setup:
  Format root
```
```
System install:
  base.tgz
```
Warning: please note that the installation process may take a while. Do not interrupt it until it is finished.
Once all part of installation are complete, the cobalt box will be rebooted automatically.
```
Installed OK:
  Rebooting...
```
Wait until the cobalt appliance boots up from the hard drive.
```
NetBSD/cobalt
Bootloader
```
```
Loading:
wd0a:netbsd
```
```
NetBSD/cobalt
Starting up...
```
```
[Starting up]
  > dhclient
```
Finally, the box host name and IP address (that are assigned by the i386 server running restorecd) should appear on the LCD screen.
```
q-160.netbsd.org 
  [10.0.0.160]
```
This indicates installation success.
BE AWARE that a newly installed box is left in a highly insecure state.

All ttys are open by default and allow root access over the network. Also, telnet and rlogin ports are open. This is done to allow you to login into the fresh box in order to make necessary configuration changes.

Please do not forget to bring your cobalt appliance to a secure state before connecting it to an open network:
- set root password by passwd(1)
- remove all pseudo terminal (tty[pqrs]?) lines from /etc/ttys, or remove “secure” field from such pseudo terminals
- comment out telnet and rlogin lines in /etc/inetd.conf if you don't really want to enable those services by default (it might be better to use sshd(8))
- add new users with user(8) etc.
- put users who need root access via su(1) in the “wheel” group by editing the /etc/group file
See the afterboot(8) man page for more details.
Check afterboot(8) for further settings. Some of the information is not applicable to the cobalt, but will give you some background information on configuring NetBSD in general. The NetBSD Guide also provides helpful information.

Warning: a freshly installed cobalt box is configured as a DHCP client, i.e. it tries to obtain its IP address over the network from a DHCP server, for initial setup with the i386 server running restorecd. To disable this after your own settings are complete, change /etc/rc.conf to contain “dhclient=NO” line.

Troubleshooting (top)

If you have any trouble and your cobalt doesn't boot up to the multi user, you can still use the restorecd for recovery. As mentioned in the previous section, netboot environment with the restorcd provides remote login capability without reinstalling, so you can check and fix contents in your hard drive etc. without serial console.

To debug any problems with the Restore CD operation itself you will need a serial cable. Cobalt machines use standard DB9 male connector and NetBSD Serial Port Primer page might help to prepare it. Attach this cable to your Qube's serial port and to your computer. Now you can run a terminal program and monitor the installation progress.

Qube2700 does not have serial port so there is not much you can do. About the only thing is to extract hard disk from Qube, attach it to a PC or a newer Qube2. Examine the failed install and try to figure out what went wrong. If you are brave enough, you could also try to build optional serial interface.

Upgrading installed system (top)

Unfortunately Restore CD doesn't support automated upgrading installed system, but you can still try to update your system manually as described in the installing a current snapshot section in the Tracking NetBSD-current document.

If you have a working serial console, you can also try to boot sysinst(8) based RAMDISK kernel in the distribution for menu based upgrading. See Upgrading NetBSD section in the NetBSD guide for details.

If you have any trouble on upgrading, you can still use Restore CD as a rescue disk as mentioned in the previous section.

Restore CD Internals

What restorecd script does? (top)

Makefile in the restorecd script does the following ops:

fetch release binary sets (both for i386 and cobalt) and source sets (necessary files are listed in restorecd-fetch.lst file)
extract whole source sets
build toolchains for NetBSD/cobalt, to prepare makefs(8) command (required to create ISO image) and to rebuild paneld binary and its manual pages
extract cobalt binary sets and build paneld binary and man pages
invoke “restorecd” script to build the actual Restore CD ISO image

The script of “restorecd” performs the following ops:

check if the system has necessary commands
prepare files for i386 server (which will appear in root of the ISO image) by extracting binary sets with misc local modifications
prepare files for cobalt client (which will appear in /nfsroot in the ISO image) from binary sets, rebuilt paneld binary and some local files for restorecd operations
create a bootable ISO image by makefs(8) in the NetBSD toolchain

See the Makefile and restorecd script for more details.

Note several settings used in the restorecd script depend on the default behaviors and settings of each NetBSD command and kernel, so they might have to be updated for the future NetBSD-current.

How does the Restore CD work? (top)

The i386 server uses ISO9660 file system for root file system. The GENERIC kernel now recognizes CDROM drive as a booted device so there is no special settings for it.
Of cource ISO9660 file system on CDROM is read-only, so several directories which require write permission are prepared using MFS.
- Device nodes in /dev is prepared by init(8) as well as RAMDISK kernels on installation media.
- All files under the /var directory are stored in /altvar in the CDROM, and they are copied to MFS mounted /var directory once it's ready. It's handled by the custom /etc/rc.d/mfsmount rc.d(8) script.
- The /etc directory is a bit tricky. Some files (like /etc/rc) are required right after root file system is mounted so we can't use the same way with the /var directory. Instead, restorecd uses “union” option of mount(8) command to write some necessary files in /etc dir. For named(8), chroot settings are used for some write ops. Some more other rc.d(8) scripts are modified to handle this read-only quirk.
The cobalt client uses NFS for root file system during installation. The NFS exported directory is /nfsroot on the i386 server and of cource it's also read-only, so the similar mechanism with the i386 server mentioned above is used on the cobalt client.
The i386 server searches available network interfaces from output of ifconfig(8) command with -l option. If there are multiple interfaces the first one is used. You have to use userconf(4) at the boot to disable unwanted interfaces, or modify the custom data/i386/etc/rc.d/netconfig rc.d(8) script on building restorecd.
Actual installation operations are described in the install-core.sh found in /nfsroot/cobalt/install/script directory, or data/cobalt/install/script directory in the restorecd script tree. They are not same procedure with the default sysinst(8) installation.
The installer create the following partitions:
- two times as physical memory for swap
- 20% (10% if disk size is smaller then 2Gbytes) of the available disk space for /var
- 5% of the available disk space (or 500Mbytes if it's larger than that) for /tmp
- 10Mbytes for ext2fs partition to store bootloader
- rest for the NetBSD root partition
The installer also put custom files from data/cobalt/install/files directory in the restorecd script tree for the following operations:
- fstab(5) file for the above default partitions
- paneld(8) binary and its config files under /etc
- patched ttys(5) and inetd.conf(5) for initial root login
- patched rc(8) and rc.d(8) scripts to show boot process to the LCD display

Credits

Authors (top)

This document was originally written and contributed to the NetBSD Project by Alex Pelts <alexp@broadcom.com>. The Create and Install sections are updated for 5.0_BETA restorecd and the Internals section is added by Izumi Tsutsui <tsutsui@NetBSD.org>.

Other credits (top)

Thanks to Dennis Chernoivanov for creating the original NetBSD Restore CD upon which this work is based and the original instructions.txt document which the Install section is based on, and also thanks to Havard Eidnes for improving the instructions.txt document. Of course many thanks to numerous people from the port-cobalt mailing list, who helped me to test this CD.

Document revision history (top)

2005-03-11 -- original version announced on the port-cobalt mailing list.
2005-03-13 -- original import into NetBSD's htdocs CVS module with minor english and grammar fixes.
2008-11-18 -- updated for newer 5.0_BETA restorecd script.
2009-04-20 -- Add some note about upgrading.
2009-04-30 -- updated for 5.0 release.

All other changes can be seen from the cvsweb interface.