LiveCPD README 2010-11-01 ================= Table of Contents ================= Overview Installation and Operation Basic Usage Graphical User Interface Accessing secondary removable storage devices: General Notes Readme File Command-line operations Text Editing System Configuration Editing Configuration Files: Editing the CPD configuration file Configuring the network: Firewall configuration Applying Configuration Changes Advanced Override Device Management To change the running crontab file: To use the hard disk as the override device. To set up symbolic links to modifying special system control files To access old archived data To intialize a new bootable USB device: To create an ISO (CD) image that can be burned to make a bootable CD: To update the root image on a bootable USB device If the graphical interface fails to start Switching to CPD2 data acquisition software Hardware Requirements ======== OVERVIEW ======== LiveCPD is a self-contained aerosol data acquisition system developed by the Aerosol Group (AERO) in the Global Monitoring Division of NOAA's Earth System Research Laboratory. The name "LiveCPD" derives from the name of the AERO data acquisition program 'cpd' and the term "Live CD", which used to denote an operating system that runs from a bootable CD. The 'cpd' software automates the operation of aerosol sampling systems and originally ran from a computer hard disk. LiveCPD can be run from an internal or external drive such as a CD-ROM or USB flash drive, instead of the hard drive, providing for completely portable and very robust system operation suitable for both short-term sample collection (i.e., field intensives and aircraft sampling) and long-term monitoring programs. LiveCPD is freely available from NOAA and uses only open-source software. It is based on the Debian Linux distribution, customized by AERO to meet the special needs of automated data acquisition. Because LiveCPD normally boots from a CD-ROM, no computer-specific configuration information can be stored in the normal manner on the boot device. Since the LiveCPD CD is not a writable device, all custom configuration information is stored on a separate writable device (e.g., RAM or USB flash drive), which "overrides" the default configuration files on the CD during system boot up. If a file exists on both the override and boot devices, then the version on the override device is used. More information on how to use this override system is given below. If no override device is found, then all configuration defaults as they are programmed on the LiveCPD CD are used. This document describes version 2 of LiveCPD, which is also referred to as LiveCPD2. ========================== INSTALLATION AND OPERATION ========================== LiveCPD can be obtained from AERO on a CD or downloaded from ftp://ftp.cmdl.noaa.gov/aerosol/etc/cpd/LiveCPD.iso. The downloaded CD image is burned to a CD-R or CD-RW disk. Do not just copy the file to the CD-R(W) disk; special software may be required on operating systems that do not include software for burning CD images (e.g., older versions of Windows). Reboot the target computer with this CD in the drive. If it does not boot into the Live CD, you may have to adjust BIOS settings (ensure that CD-ROM is listed before any hard disks in boot order). Operation can be as simple as booting the computer with the CD if you choose to operate the system using all the defaults programmed on the CD. In this case, data will be written to RAM in a directory called /aer/nil/log. Normally, though, you will connect some form of external USB storage device to the system for data and configuration storage (e.g., a USB flash drive). This device must have a capacity of at least 32 megabytes. If LiveCPD is booted with an uninitialized flash drive plugged into the computer, the Normally, four tasks must be done when booting a LiveCPD system for the first time: 1. Define the station identification code in the 'cpd.ini' or 'cpd.conf' configuration file; 2. Edit 'cpd.ini' or 'cpd.conf' to define the instruments in operation and the serial ports that they are connected to; 3. Configure the networking parameters. 4. Reboot the system. Procedures for these tasks are described below. The full LiveCPD system can also be installed to, and booted from, a USB flash drive. Hardware and BIOS support for booting from USB flash drives is not standardized, however, and may not be supported on all hardware. Executable files (Linux and Windows) for installing LiveCPD to a USB flash drive are available at: ftp://ftp.cmdl.noaa.gov/aerosol/etc/etc/cpd/usb/livecpd2-installer.sh ftp://ftp.cmdl.noaa.gov/aerosol/etc/etc/cpd/usb/livecpd2-installer- win32.exe =========== BASIC USAGE =========== Graphical User Interface ------------------------ All of the icons across the bottom of the screen perform their associated action when double-clicked. Below this row of icons is a status bar with a desktop pager, icons of running programs, a help button, and clock. The 4- box grid at the left of the bar is the desktop pager; clicking on one of the squares changes the current workspace to that instance of the desktop. Applications launched on one desktop are only visible from their parent desktop. The row of icons across the bottom of the screen allows quick access to the most frequently used programs. These include: "Start CPD" Opens a new terminal window and starts the CPD client program. If the CPD server program is not already running, it will be started as well. Runs "cpd.start" in a new terminal window. "CPX" Starts CPX, the graphical display client for CPD. "Send Data" Archives current data and sends all archived data (including the most recent data) to AERO for processing. Runs "cpd.archive send=1". "CPD Config" Allows modification of the "cpd.ini" or "cpd.conf" configuration file using the NEdit text editor. "Firefox" Start the Firefox web-browser. "Terminal" Creates a new terminal. "Stop CPD" Stops any instance of CPD that is currently running. Runs "cpd.kill" after a confirmation dialog box. "Shutdown" Reboots or shuts down the computer after a confirmation dialog box. Additional icons on the Desktop include: "Storage" Brings up a file browser to display the contents of a secondary removable storage device. If a secondary device is not present, the file browser will display the contents of the /mnt directory. "Data" Brings up a file browser in the CPD Data directory. "Open Tunnel" Establishes an encrypted connection to an AERO server, which subsequently can be used to login to the LiveCPD system from remote clients. This allows remote access for system maintenance and troubleshooting in cases where firewalls block direct remote access to the LiveCPD system. The file browser works similar to graphical file browsers in other operating systems. Files can be dragged and dropped from one place to another. After dropping a file, a context menu will be displayed with a choice of actions that can be taken. Double clicking on a file performs the associated action for that file type (e.g., open a text editor). To change this action, right click on the file and select: File 'FILENAME' -> Set Run Action Similarly, to edit any file with the vi text editor: File 'FILENAME' -> Open as text Clicking on the "Help" icon in the lower-right corner of the screen brings up a list of sources of additional information. These include: "INI Documentation" A description of all possible entries in the cpd.ini or cpd.conf file, which is used to control the operation of 'cpd'. "Station Flow Diagram" A diagram showing the position of all sensors and instruments in the flow system. Variable names are constructed based on their position in the flow system. The second page of the file is a cross-reference list of old and new variable names. "Operating Manual" AERO's operating manual for aerosol sampling systems. "LiveCPD2 Documentation" This file "System Status" A summary of system operating status "Contacts" A list of contact people at AERO Accessing secondary removable storage devices --------------------------------------------- LiveCPD normally operates with a single USB flash drive for data storage. Additional removable storage devices will be automatically detected when they are plugged in and their partitions will be automatically mounted under the /mnt/hotplug directory. The first partition on the first of these additional devices to be plugged in will be automatically mounted as /mnt/hotstorage, and can be accessed by double-clicking on the "Storage" icon on the desktop ============= GENERAL NOTES ============= Readme File ----------- This readme file can be accessed at any time, even if the graphical interface is not running, by pressing CTRL-ALT-F12. To return to the graphical interface from the readme file press CTRL-ALT-F7. Similarly, a root shell (full access) that can be accessed by pressing CTRL-ALT-F11; again, CTRL-ALT-F7 returns from the root shell to the graphical interface. Pressing CTRL-ALT-BACKSPACE followed by CTRL-ALT-DELETE will reboot the system. Command-line operations ----------------------- To control and make changes to the system you will be working in a UNIX terminal window on the Linux desktop. Double-clicking the "Terminal" icon at the bottom of the screen will open a window to work in. When you first open a window your location will be /home/cpd. You can perform most of the required tasks for configuration of the system from this location. Text Editing ------------ While custom configuring the system, you may need to edit configuration text files. The 'vi' text editor is used from the command line. If you have never used 'vi' before, the F1 key will launch an on-line help window. A quick guide for 'vi' usage is as follows: Press "i" to enter insert mode in order to type text (in this mode, 'vi' acts like most other editors). Press ":w" to save changes (you'll have to re-enter insert mode to add text again). Press ":q" to quit out of the editor and return to the command line prompt. ==================== SYSTEM CONFIGURATION ==================== Editing Configuration Files --------------------------- LiveCPD works by having most of the system on a read-only filesystem image (usually a CD-ROM). Station-specific configuration is done by means of an override device and image. The override image file normally resides on a USB flash drive that must be connected to the computer during boot up. This file contains a compressed archive of all files that are overridden. Normally any directory that does not contain overridden files resides on the read-only baseline image; any directories that contain an overriden file are moved into the read/write temporary memory (RAM). All overridden files are tracked for changes and when they change the override image file is updated to reflect that change. Data are normally stored as conventional files on the same physical device as the override image file. Configuration files are stored on the USB drive within a special file called /mnt/override/override.img and data are be stored on the USB drive in /mnt/override/aer/nil/log. To change files they must usually be overridden, using the command override /path/to/file This also performs any necessary directory moves to get the directory containing the file into read/write memory. Upon bootup, the system has already configured "cpd.ini" and "CPX" to be editable without the need of any special programs. They can be edited as normal. To edit any other file, you must first use the 'override' command. The LiveCPD system is supplied with several text editors, including 'vi' and 'nano' (text mode), as well as 'nedit' (graphical mode). Changes to the files take effect immediately though the programs they affect, if any, may need to be restarted so that the file can be re-read by the system. NOTE: If two viable override devices are plugged into the system, the device that is used to boot the system is arbitrary. But once the system has booted from one device, that device is excluded from being re- initialized and the above still holds. Editing the CPD configuration file for CPD1 (default) ----------------------------------------------------- After an override device is initialized, a default "cpd.ini" file is set up. This ".ini" file contains a definition for all instruments supported by CPD. The default version of "cpd.ini" that comes on LiveCPD has only a Nephelometer, CNC, and PSAP enabled. These instruments are configured for USB serial ports 1, 2, and 3 respectively. A copy of this full ".ini" file is also available in $CPD/etc/cpd.ini.example. Click on the "CPD Config" icon to edit the cpd.ini file. There are two lines in the [Disk] section that must be edited to change the station identification code. The default code is "NIL", and must be changed to the correct code for the actual station in the "LogDir=" and "StationID" entries. Make sure to use lower case for "LogDir" and upper case for "StationID": [Disk] LogDir=/aer/nil/log/ StationID=NIL The directory "/aer/stn/log" will be created on the storage device, where "stn" is the three digit station code (in lower-case), and data files will be placed there. Editing the CPD configuration file for CPD2 ------------------------------------------- First it is necessary to switch the system to CPD2 mode. This is done by making the file /etc/cpd1 empty: override /etc/cpd1 sudo rm -f /etc/cpd1 sudo touch /etc/cpd1 Followed by rebooting. Once the system has booted up in CPD2 mode a default "cpd.conf" file is set up. A copy of this default is available in /aer/prg/cpd2/etc/cpd.nil.conf. Click on the "CPD Config" icon to edit the cpd.conf file. There are two lines near the top that must be set to change the station identification code. The default code is "NIL", and must be changed to the correct code for the actual station, in the "Disk;!StationID,NIL" and "Disk;LogDir,/aer/nil/log" lines. The directory "/aer/stn/log" will be created on the storage device, where "stn" is the three digit station code (in lower-case), and data files will be placed there. See the cpd.conf documentation (in the help menu on the lower right) for more information about the structure of the configuration file. Configuring the network ----------------------- To change the network configuration, use: sudo livecpd2.setupnetworking This updates network settings, making the changes persistent, and restarts the network with the updated settings. Firewall configuration ---------------------- For remote access incoming port 22 (SSH) must be allowed to the computer's IP. For data processing, outgoing port 21 (FTP) must be allowed to ftp.cmdl.noaa.gov. To use NTP time synchronization UDP port 123 must be allowed. To restrict incoming SSH to be only from vortex uncomment the specified lines in /etc/init.d/applyfirewall (NOTE: you must override it, see above). Applying Configuration Changes: ------------------------------- Once all of the desired steps are completed, a final reboot is required for the system to properly use all the updated settings. This is only required if you have changed core system settings (i.e., it is not needed if you all that you have changed is the CPD configuration, in this case just stop and restart CPD). ======== ADVANCED ======== Override Device Management -------------------------- Create a new default override device (non-destructive): sudo livecpd2.createoverride Erase anything on an override device and initializing it: sudo livecpd2.createoverride --format Copy current override device to a new one: sudo livecpd2.createoverride (Answer yes to copy prompt.) Queue a backup of the override device to be sent to NOAA: sudo livecpd2.sendbackup This actually saves an image of the backup in /aer/$STN/log/send/override_$STN_$DATETIME.img which will be transferred when data is next sent. Download and restore a station backup (requires an internet connection and immediately reboots the system): sudo livecpd2.restoreoverride Download and restore the latest CPD(2) configuration from NOAA. Rebooting after is recommended (and may be required if the station switched): livecpd2.restorecpd Override a file: override /path/to/file Immediately save changes to a file: save /path/to/file Force override device write (not normally needed): sudo livecpd2.updateoverride --force To change the running crontab file ---------------------------------- override /var/spool/cron/crontabs/cpd crontab -l > /tmp/crontab.tmp vi /tmp/crontab.tmp crontab /tmp/crontab.tmp To use the hard disk as the override device ------------------------------------------- This example assumes that the hard disk is /dev/hda, but some computers assign the hard disk to /dev/sda: sudo fdisk /dev/hda sudo mkdosfs /dev/hda1 sudo mkdir /mnt/hda1 sudo mount /dev/hda1 /mnt/hda1 sudo cp /mnt/override/override.img /mnt/hda1 \ || touch /mnt/hda1/override.img sudo umount /dev/hda1 sudo reboot This will cause data to be logged on the hard disk as well. To set up symbolic links to modifying special system control files ------------------------------------------------------------------ Some files exist only on the CD and can not be changed or edited (see the attached directory structure map). In order to edit files, they must be placed on a writable medium such as the USB drive or in RAM. A simple way to accomplish this is to set up a "symlink" which points the files on the CD to the writable device. NOTE: Files in /usr, /bin, /sbin, /lib, and /boot cannot be edited. To set up a symlink (for instance in the rcX.d directories), first create the directory in /mnt/override/ then create a symlink from there, as if it were /. For example, to set up a symbolic link to enable samba: override /etc/rc5.d/S50samba cd /etc/rc5.d ln -s ../init.d/samba S50samba To access old archived data --------------------------- Old data can be accessed by using livecpd2.mountstorageimage to mount the old storage image at /mnt/storageview: livecpd2.mountstorageimage /mnt/override/storage/storage_20110101T010101_20110201T010101Z.img The desktop "Archive" icon in the upper left will also open a file browswer at that location, then double clicking on an archive file will open it. To intialize a new bootable USB device -------------------------------------- sudo livecpd2.installusb LiveCPD can create a 'clone' of itself on a USB storage device, which can then be used as the boot device. This can also initialize it as an override device or copy the existing one. To create an ISO (CD) image that can be burned to make a bootable CD -------------------------------------------------------------------- livecpd2.makeiso mnt/hotstorage/livecpd2.iso Note the location of the ISO must have enough free space. To burn an ISO use: sudo wodim -tao -v mnt/hotstorage/livecpd2.iso See the man page at: man wodim To update the root image on a bootable USB device ------------------------------------------------- From within LiveCPD: sudo livecpd2.updateroot This requires double the existing boot file space (approximately 250 MB) and an operational Internet connection to retrieve files from ftp.cmdl.noaa.gov. Alternatively from another computer the update files can be downloaded manually. They are located at ftp://ftp.cmdl.noaa.gov/aerosol/etc/cpd/usb/update/. All files in that directory ("root.img", "vmlinuz", and "initrd.gz") should be downloaded and replace the existing files in the root directory of the USB device. To save a screenshot -------------------- Press Shift-PrintScreen. The screenshot will be in /aer/$STN/log/send/screenshot_$STN_$DATETIME.bz2. You can also specify a filename by manually by doing: livecpd2.screenshot /tmp/outputscreenshot.bz2 To select a specific window to capture an image of: xwd | bzip2 -9 - > /tmp/windowscreenshot.bz2 This will cause the cursor to change to a crosshair and will capture an image of the first window clicked on. Note that this is in native X11 screen capture format. To convert it you can use "convert" on a system with ImageMagik (www.imagemagick.org) installed (LiveCPD2 does not have it installed) by doing: convert screenshot.bz2 screenshot.png Alternatively you can use it to view the screenshot: display screenshot.png If the graphical interface fails to start ----------------------------------------- The graphical interface is a Linux desktop with a large NOAA icon in the center, a task bar at the bottom, and several icons on the desktop. LiveCPD automatically probes the hardware on bootup and configures the graphical interface for the video adapter in use. In the automatic probing doesn't work, try a different computer. Switching to CPD2 data acquisition software ------------------------------------------- To switch the system to use CPD2 for data acquisition do the following as root: override /etc/cpd1 rm /etc/cpd1 touch /etc/cpd1 The system must be rebooted to apply the changes. The file /etc/cpd1 will cause the system to start in CPD1 mode if it is of non-zero size. Since the override device does not allow deletion of files from the root image (which currently ships with that file), it must be overriden to zero size ===================== HARDWARE REQUIREMENTS ===================== CPU: Intel Pentium III or higher RAM: 256 MB minimum Boot device: CD-ROM drive or USB flash drive Data storage: USB flash drive (128 MB or larger) Hard disk: not required or used. Network: Ethernet adapter highly recommended, but not required Video: SVGA 1024x768 minimum. Slightly lower resolution, for example, the 1024x600 resolution found in many netbook computers, Serial ports: multi-port USB-to-RS232 adapter recommended