Showing posts with label Embedded System. Show all posts
Showing posts with label Embedded System. Show all posts

Tuesday, January 1, 2013

Raspberry Pi Multi-User Graphical Terminal Server

Would you like to turn your Raspberry Pi into a Multi-User Terminal Server so that multiple users may use it simultaneously? I wanted to experiment the multi-processor features of the language Erlang, which is a functional language used in soft-real-time-systems such as telephone switches. To that end I setup two Raspberry Pi's (Hereafter RPi) as Terminal Servers so that I could log into them as different users at the same time, to do some multi-user Erlang experiments. With the limited RAM of the RPi you are not going to be doing a lot with many users, however it meet my needs of my Erlang experiments. I explain here how to turn the RPi in to a mutli-user X Org Graphical Terminal Server.


Contents:

  1. Why Arch Linux ARM?
  2. Install Arch Linux ARM on a SD Card
  3. Connecting to the RPi via Secure Shell
  4. Setting up the file system
  5. System Configuration - Change root password, locale, timezone, hostname
  6. Full system update using the package manager, pacman
  7. Install the X Window System and Display Manager
  8. Install the Desktop Environment Fluxbox
  9. Install the Virtual Network Computing (VNC) remote framebuffer
  10. Making it look good
  11. Desktop Icons
  12. Desktop Background with Feh
  13. Configure Fluxbox for Background and Icons
  14. Give the RPi a static IP address
  15. Install sudo or sux and add your first user account
  16. Install Development Environment
  17. Making use of the Arch User Repository (AUR)
  18. WiringPi library and GPIO access, for GPIO with C or command line
  19. The Bleeding Edge
  20. GPU/ARM Memory Split. How to get more RAM
  21. Troubleshooting
  22. References

Why Arch Linux ARM?:

After playing with different RPi distributions I settled on using Arch Linux. Arch is a 'live' distribution that releases each new/updated package as it is available rather than waiting for month or years long release cycles. I have been using Gentoo and Funtoo, as my home development system for years, and I like the 'live' update philosophy. Building Gentoo packages natively would take far to long, and I did not care to set up a cross-build environment right now.

Arch recently switched from using the common 'rc' bash configuration scripts to using systemd to bring up the system and configure it, so virtually all things you find on how to set up a system right now are out of date. This one is based on using systemd from the start.

Install Arch Linux ARM on a SD Card:

To get started download the current Arch RPi distribution, and follow the official Arch RPi install guide. After you have the Arch image written to your SD card, if it is larger than 2G you need to resize, and add partitions, to it to use all of the space. Also take a look at SDCard Tuning to speed up the SDCard.

For creating/modifying partitions the simplest way I have found to do this is to use the program GParted, the graphical repartitioning utility. If you do not have a native GParted on the system with the card reader, then boot up the System Rescue CD on a USB stick to bring up GParted. In theory you can repartition the RPi while running from the SD card that is being repartitioned, which is risky to do in that it could potentially lose data, I do not recommend doing so. While using GParted to create the partitions, also use it to format the new swap and fat32 partitions, do not format the /boot and root (/) partitions!

I set up four partitions 1) /boot. 2) The root system/user space which needs resized via GParted, to use most of the card space. 3) a 1G swap partition, twice the RAM size of the 512M models, mine is a 256M model. Some will debate the wisdom of swap on a SDCard, I'm accepting the risk in this application. 4) 1G partition as a FAT32 (Windows) partition for future experiments. Parted, the command line version of GParted, shows the layout below.


Remount the SD card and restart the RPi. Now is a good time to point out a ugly problem with the RPi. If it is not properly shutdown it frequently damages the file system on the SD card so, always use the 'poweroff' command to shut the RPi down before removing power from the board! It is also a good idea to make a back up image of the card from time to time. Doing so between install steps of this guide is a good idea, to be able to return to a known good spot, rather than having to start completely over if a step is missed.

Connecting to the RPi via Secure Shell:

By default the RPi uses DHCP to get an IP address from the router, in my case 172.16.17.4. Connect to the RPi using the Secure Shell SSH. In this guide I use the following notation: '#' indicates commands I entered on the RPi via the SSH connection under the root account (it is also used as a comment delimiter in some configuration files, not my idea), '*' indicates commands I entered on my host GNU/Linux machine, and ';' indicates a comment that should not be entered on the RPi:

  
* ssh root@172.16.17.4
The authenticity of host '172.16.17.4 (172.16.17.4)' can't be established.
ECDSA key fingerprint is c8:52:bf:53:9c:c3:db:b4:11:c3:57:36:78:3c:05:c4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.17.4' (ECDSA) to the list of known hosts.
root@172.16.17.4's password: [Which is 'root' by default, we will change shortly.]
  

Setting up the file system:

Arch uses a package manager called 'pacman', to install packages such as a native version of Parted to confirm our partition layout. Parted also shows us what the native mount points are called, '/dev/mmcblk':

  
# pacman -S parted
# parted
GNU Parted 3.1
Using /dev/mmcblk0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: SD SU16G (sd/mmc)
Disk /dev/mmcblk0: 15.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system     Flags
 1      1049kB  99.6MB  98.6MB  primary  fat16           boot, lba
 2      99.6MB  13.7GB  13.6GB  primary  ext4
 3      13.7GB  14.8GB  1074MB  primary  linux-swap(v1)
 4      14.8GB  15.9GB  1125MB  primary  fat32


Running the 'p'rint command of 'fdisk /dev/mmcblk' shows the information that we need to update the mount point control file /etc/fstab, to match our new partition layout:

  
# fdisk /dev/mmcblk
Disk /dev/mmcblk0: 15.9 GB, 15931539456 bytes, 31116288 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c21e5

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1   *        2048      194559       96256    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          194560    26818559    13312000   83  Linux
/dev/mmcblk0p3        26818560    28915711     1048576   82  Linux swap / Solaris
/dev/mmcblk0p4        28915712    31113215     1098752    b  W95 FAT32


The file /etc/fstab, which controls disk mount points, needs to be edited to match our partition layout, and the mount point for our Windows partition needs to have a mount point created. The base Arch system comes with the editors nano and vi installed, but I prefer using EMACS. Using Ctrl-X, CTRL-E will bring up EMACS from any Bash command prompt, once EMACS is installed with pacman:

  
; -Ss is used to search for packages.
# pacman -Ss emacs
   extra/emacs 24.2-2
    The extensible, customizable, self-documenting real-time display editor

; -S is used to install packages.
# pacman -S emacs bash-completion

   resolving dependencies...
   looking for inter-conflicts...

   Targets (88): alsa-lib-1.0.26-1  at-spi2-atk-2.6.2-1  at-spi2-core-2.6.3-1...

   Optional dependencies for imagemagick
    ghostscript: for Ghostscript support
    openexr: for OpenEXR support
    libwmf: for WMF support
    librsvg: for SVG support
    libxml2: for XML support
    jasper: for JPEG-2000 support
    libpng: for PNG support
   (88/88) installing emacs

# pacman -S ghostscript libwmf librsvg libxml2 jasper libpng
   warning: librsvg-2.36.4-1 is up to date -- reinstalling
   warning: libxml2-2.8.0-1 is up to date -- reinstalling
   warning: libpng-1.5.13-1 is up to date -- reinstalling
   resolving dependencies...
   looking for inter-conflicts...


That shows one of the annoying aspects of pacman, it recommends packages that are already current. Make the needed edits to /etc/fstab using the editor of your choice or copy the section below and use scp to copy the information below to a to the /etc/fstab on the RPi. Create the mount point for Windows to use, and then edit /etc/fstab to match the following:

  
# mkdir /mnt/win
# emacs /etc/fstab

# ==== Cut Here ====
#
# /etc/fstab: static file system information
#
# [file system]        [dir]         [type]    [options]          [dump] [pass]

# 'Virtual/Pseudo' terminals (like created by ssh or screen):
devpts                 /dev/pts      devpts    defaults            0      0

# RAM Disk for /tmp et.al:
shm                    /dev/shm      tmpfs     nodev,nosuid        0      0

# Boot partition:
/dev/mmcblk0p1         /boot         vfat      defaults            0      0

# Root file system:
/dev/mmcblk0p2         /             ext4      defaults,noatime    0      0

# Enable swap partition on boot:
/dev/mmcblk0p3         none          swap      defaults            0      0

# Windows compatible partition:
/dev/mmcblk0p4         /mnt/win      vfat      defaults            0      0
# ==== Cut Here ====

# reboot

; Alternatively:
# mkdir /mnt/win

; On the host machine:
* scp ~/RPi/etc/fstab root@172.16.17.4:/etc/fstab

# reboot

System Configuration - Change root password, locale, timezone, hostname:

Before we get to far along we need to set our hostname, the RPi's timezone, and get the clock to set itself with NTP. We give each of our RPi's their own unique host name with 'hostnamectl' (sometimes picking good host names is the hardest part of bring up systems):

  
# hostnamectl
   Static hostname: alarmpi
   Pretty hostname: n/a
         Icon name: n/a
        Machine ID: 5c49cb5bf08724338eb09a6900092bb0
           Boot ID: 14284b2e022844dd8762c23ecd1e2409
  Operating System: Arch Linux ARM
            Kernel: Linux 3.2.27-18-ARCH+
      Architecture: armv6l

# hostnamectl set-hostname Erlang1

# hostnamectl
   Static hostname: erlang1
   Pretty hostname: Erlang1
         Icon name: n/a
        Machine ID: 5c49cb5bf08724338eb09a6900092bb0
           Boot ID: 14284b2e022844dd8762c23ecd1e2409
  Operating System: Arch Linux ARM
            Kernel: Linux 3.2.27-18-ARCH+
      Architecture: armv6l


 Check the current timezone:

  
# timedatectl status
      Local time: Sun, 2012-12-23 01:15:31 GMT
  Universal time: Sun, 2012-12-23 01:15:31 UTC
        Timezone: Europe/London
      UTC offset: +0000
     NTP enabled: yes
  NTP synchronized: no
   RTC in local TZ: no
        DST active: no
   Last DST change: BST -> GMT, DST became inactive
                    Sun, 2012-10-28 01:59:59 BST
                    Sun, 2012-10-28 01:00:00 GMT
   Next DST change: GMT -> BST, DST will become active
                    the clock will jump one hour forward
                    Sun, 2013-03-31 00:59:59 GMT
                    Sun, 2013-03-31 02:00:00 BST


I'm between Pittsburgh and Eire PA so we need to find the East Coast timezone with timedatectl list-timezones:

  
# timedatectl list-timezones
...
America/Mexico_City
America/Miquelon
America/Moncton
America/Monterrey
America/Montevideo
America/Montreal
America/Montserrat
America/Nassau
America/New_York
America/Nipigon
America/Nome
America/Noronha
America/North_Dakota/Beulah
America/North_Dakota/Center
America/North_Dakota/New_Salem
...


Then to set the zone, and to confirm the result:

  
# timedatectl set-timezone America/New_York

# timedatectl
      Local time: Sat, 2012-12-22 20:22:09 EST
  Universal time: Sun, 2012-12-23 01:22:09 UTC
        Timezone: America/New_York
      UTC offset: -0500
     NTP enabled: yes
  NTP synchronized: no
    RTC in local TZ: no
       DST active: no
  Last DST change: EDT -> EST, DST became inactive
                   Sun, 2012-11-04 01:59:59 EDT
                   Sun, 2012-11-04 01:00:00 EST
  Next DST change: EST -> EDT, DST will become active
                   the clock will jump one hour forward
                   Sun, 2013-03-10 01:59:59 EST
                   Sun, 2013-03-10 03:00:00 EDT


As mentioned earlier the default configuration for Arch starts DHCP to get a dynamic address from the local router. Once it has the address, it starts the OpenNTP daemon to set the system software clock to the current time. While Arch for the RPi ships using OpenNTP. Arch recommends using NTPd instead. OpenNTP has not given me any problems so far, so I left it as it was for now. Note the date change in the log:

  
Dec 31 19:00:11 erlang1 systemd[1]: Started dhcpcd on eth0.
Dec 31 19:00:11 erlang1 systemd[1]: Starting Network.
Dec 31 19:00:11 erlang1 systemd[1]: Reached target Network.
Dec 31 19:00:11 erlang1 systemd[1]: Starting OpenNTP Daemon...
Dec 23 08:15:27 erlang1 ntpd[133]:  set local clock to Sun Dec 23 08:15:27 EST 20
Dec 23 08:15:27 erlang1 systemd[1]: Started OpenNTP Daemon.


Check our locale:

  
# localectl
   System Locale: LANG=en_US.UTF-8
                  LC_COLLATE=C
       VC Keymap: us
      X11 Layout: n/a


While LANG is set reasonably there, things such as GParted, tend to break with the default Arch image. To get things to work you first need to enable locales available to the system by uncommenting them in /etc/locale.gen and then executing locale-gen as root. The locale set via localectl must be one of the uncommented locales in /etc/locale.gen. The default system locale is configured in /etc/locale.conf. Set the default locale:

  
; Enable your locale in /etc/locale.gen then:
# locale-gen
; Show all available POSIX locales:
# locale -a
# localectl list-locales
# localectl set-locale LANG=en_US.utf8 LC_COLLATE=C LC_MESSAGES=en_US.utf8

  
# localectl status
   System Locale: LANG=en_US.utf8
                  LC_COLLATE=C
   VC Keymap: us
   X11 Layout: n/a


Our initial configuration will be complete once we change the default password of 'root' to something secure:

  
# passwd
   Enter new UNIX password:  ****************
   Retype new UNIX password: ****************
   passwd: password updated successfully
# reboot


Once the root password, hostname and timezone, is set it is best to reboot the RPi using the 'reboot' command.

Full system update using the package manager, pacman:

After the system is back up use pacman, to bring all packages on the system up to date:

  
# pacman -Syyu
:: Synchronizing package databases...
 core                      36.2 KiB   127K/s 00:00 [######################] 100%
 extra                    417.6 KiB   346K/s 00:01 [######################] 100%
 community                415.9 KiB   383K/s 00:01 [######################] 100%
 alarm                      5.1 KiB  2.80M/s 00:00 [######################] 100%
 aur                       12.2 KiB   602K/s 00:00 [######################] 100%
:: Starting full system upgrade...
:: Replace dbus-core with core/dbus? [Y/n] Y

resolving dependencies...
looking for inter-conflicts...

Targets (66): bash-4.2.039-1  binutils-2.23-1  ca-certificates-20121105-1
              raspberrypi-firmware-2021221-1 ...

Total Download Size:    95.73 MiB
Total Installed Size:   366.16 MiB
Net Upgrade Size:       -1.75 MiB

Proceed with installation? [Y/n] Y


The RPi is no speed demon and it took 20 minutes to upgrade those 66 package. In the upgrade spew there are messages such as:

  
(18/65) installing dbus
Optional dependencies for dbus
    libx11: dbus-launch support

(45/65) upgrading logrotate
New optional dependencies for logrotate
    cron: scheduled log rotation


that give hits at other packages that may need to be installed later. After a major upgrade I always reboot the system to be sure that any new kernel and board firmware gets loaded. I discuss the new GPU/ARM RAM sharing options of the firmware, and how to load a newer kernel, at the end of this article. Once the reboot is complete our base system is up to date.

Install the X Window System and Display Manager:

Now install the sound and video drivers for the X Window System that our Display Manager and Desktop Enviroments are built upon. The Display Manager lets us chose at login time what Desktop Environment we want to use for that session:

  
; Install Alsa, Advanced Linux Sound Architecture, required for sound:
# pacman -S alsa-firmware alsa-utils

; Install Xorg, pre-requisite for GUI apps:
# pacman -S xorg-server xorg-xinit xorg-utils xorg-server-utils xorg-xmessage dbus-launch

; Install Xorg video driver:
# pacman -S xf86-video-fbdev

; Basic utilities:
# pacman -S aterm xterm sux

; Display Manager that lets us select amoung different Desktop
; Environments:
# pacman -S lightdm lightdm-gtk-greeter

; Enable the Display Manager to start on the next reboot:
# systemctl enable lightdm.service

Install the Desktop Environment Fluxbox:

With the limited resources of the RPi, Desktop Enviroments (DE) like GNOME and KDE are out of the question. There are several DE's that are more resource conscious: Awesome, Enlightenment, Fluxbox on Arch, Openbox to name a few. I'm going to concentrate on using Fluxbox for the remainder of this article. There is no reason that you can not install them all and find the one that fits your requirements the best. Install Fluxbox:

  
# pacman -S fluxbox idesk menumaker

; Create a menu system for Fluxbox based on the packages that are
; installed on the system.  Alas it is not perfect, the package manager
; 'pacman' shows up in the Games menu, sigh.
# mmaker -t Aterm -t Xterm Fluxbox -f





At this point the system could be rebooted and begin using the graphical Desktop Enviroment.

Install the Virtual Network Computing (VNC) remote framebuffer:

Lets finally get to the meat of this article, the Terminal Server. Install the Virtual Network Computing remote framebuffer that allows one computer remotely control another computer, and the eXtended Internet daemon (xinetd) to start VNC based on the incoming port number:

  
# pacman -S tigervnc xinetd


The file /etc/xinetd.d/xvncserver needs to be created and numerous lines need to be added to /etc/services (copy/edit or use scp as before):

  
# /etc/xinetd.d/xvncserver:

# TigerVNC :
#  -fp (font path) may be required for the server to be able to start.
#  Tip: read /usr/bin/vncserver to find out undocumented switches

#  http://forums.gentoo.org/viewtopic-t-72893-start-400.html :
#  server_args = :1 -inetd -query localhost -once -geometry 1620x1200 -depth 16  -SecurityTypes None

#  The lower two digits of the port number are used to select the color depth and screen resolution.
#  For example using '50' will give a color depth of 8 with a resolution of 640x480.
#  In general numbers in the '7x' range are a good place to start for modern displays.

# Port 5950:
 service vnc-640x480x8
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 640x480 -depth 8 -once -SecurityTypes None
 }
# Port 5951:
 service vnc-800x600x8
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 800x600 -depth 8 -once -SecurityTypes None
 }
# Port 5952:
 service vnc-1024x768x8
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1024x768 -depth 8 -once -SecurityTypes None
 }
# Port 5953:
 service vnc-1280x1024x8
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1280x1024 -depth 8 -once -SecurityTypes None
 }
# Port 5954:
 service vnc-1600x1200x8
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1600x1200 -depth 8 -once -SecurityTypes None
 }
# Port 5960:
 service vnc-640x480x16
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 640x480 -depth 16 -once -SecurityTypes None
 }
# Port 5961:
 service vnc-800x600x16
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 800x600 -depth 16 -once -SecurityTypes None
 }
# Port 5962:
 service vnc-1024x768x16
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1024x768 -depth 16 -once -SecurityTypes None
 }
# Port 5963:
 service vnc-1280x1024x16
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1280x1024 -depth 16 -once -SecurityTypes None
 }
# Port 5964:
 service vnc-1600x1200x16
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1600x1200 -depth 16 -once -SecurityTypes None
 }
# Port 5970:
 service vnc-640x480x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = :1 -inetd -query localhost -geometry 640x480 -depth 24 -once -SecurityTypes None
 }
# Port 5971:
 service vnc-800x600x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 800x600 -depth 24 -once -SecurityTypes None
 }
# Port 5972:
 service vnc-1024x768x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1024×768 -depth 24 -once -SecurityTypes None
 }
# Port 5973:
 service vnc-1280x1024x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1280x1024 -depth 24 -once -SecurityTypes None
 }
# Port 5974:
 service vnc-1600x1200x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1600x1200 -depth 24 -once -SecurityTypes None
 }
# Port 5975:
 service vnc-1920x1080x24
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1920x1080 -depth 24 -once -SecurityTypes None
 }
# Port 5980:
 service vnc-640x480x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 640x480 -depth 32 -once -SecurityTypes None
 }
# Port 5981:
 service vnc-800x600x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 800x600 -depth 32 -once -SecurityTypes None
 }
# Port 5982:
 service vnc-1024x768x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1024x768 -depth 32 -once -SecurityTypes None
 }
# Port 5983:
 service vnc-1280x1024x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1280x1024 -depth 32 -once -SecurityTypes None

 }
# Port 5984:
 service vnc-1600x1200x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1600x1200 -depth 32 -once -SecurityTypes None
}
# Port 5985:
 service vnc-1920x1080x32
 {
          protocol = tcp
          socket_type = stream
          wait = no
          user = nobody
          server = /usr/bin/Xvnc
          server_args = -inetd -query localhost -geometry 1920x1080 -depth 32 -once -SecurityTypes None
}


The following lines need to be append to /etc/services:

  
# Local services

#
# VNC Servers:
#
vnc-640x480x8 5950/tcp
vnc-800x600x8 5951/tcp
vnc-1024x768x8 5952/tcp
vnc-1280x1024x8 5953/tcp
vnc-1600x1200x8 5954/tcp

vnc-640x480x16 5960/tcp
vnc-800x600x16 5961/tcp
vnc-1024x768x16 5962/tcp
vnc-1280x1024x16 5963/tcp
vnc-1600x1200x16 5964/tcp

vnc-640x480x24 5970/tcp
vnc-800x600x24 5971/tcp
vnc-1024x768x24 5972/tcp
vnc-1280x1024x24 5973/tcp
vnc-1600x1200x24 5974/tcp
vnc-1920x1080x24 5975/tcp

vnc-640x480x32 5980/tcp
vnc-800x600x32 5981/tcp
vnc-1024x768x32 5982/tcp
vnc-1280x1024x32 5983/tcp
vnc-1600x1200x32 5984/tcp
vnc-1920x1080x32 5985/tcp


The file /etc/lightdm/lightdm.conf must be edited to enable the X Display Manager Control Protocol on UDP port 177, so that the XDMCPServer section reads as follows. If this step is missed you get only a blank/black screen when trying to connect:

  
[XDMCPServer]
enabled=true
port=177
#key=




At this point we can reboot and connect to the RPi via a VNC viewer. Be aware that VNC is not secure. Nothing is encrypted, so usually VNC connections are done via the Secure Shell (SSH). On GNU/Linux I use TigerVNC and if I am using Windows I use UltraVNC (Be forewarned that there are ads on the UltraVNC download section that look like the download link that are annoying ads, choose carefully).

Securing this VNC setup is worthy of an guide of its own, however it is ground that has been traveled by others so a search will uncover several. IPTables and SSH / SSH Keys / PuTTY are good places to start.

To test the system, you *must reboot* the RPi to complete the configuration, then run a vncviewer giving the address of the RPi and a port number that represents the graphical resolution. The resolutions are found in the VNC Server section that was added to /etc/services. For example 'vncviewer 172.16.17.4:50' will bring up a resultion of 640x480 with a color depth of 8. It looks bad but it is fast. 'vncviewer 172.16.17.4:85' will bring up 1920x1080 with a color depth of 32, if your monitor supports it (there is no magic here, it has to fit), that looks good but is slow. I find that numbers of 72 to 74 work the best with my host system.

At this point we have a working terminal server. However we are still running under the root account which is not good, the network needs to be on a fixed IP address for the Erlang experiments, and most of all things are ugly. Ascetics are an important marketing point to show off the RPi, however not really required due to the RPi's frugal resources. Lets make it look pretty anyway.

Making it look good:

I find that broken icon, which is the very first thing someone sees, on the login screen particularly annoying, so install some of the stock Arch icons in the correct place, to get the Arch Logo Icon:

  
# pacman -S feh conkey archlinux-artwork
# find /usr/share/archlinux/icons -name "*64*" -exec cp {} /usr/share/icons/hicolor/64x64/devices \;


Edit the file /etc/lightdm/lightdm-gtk-greeter.conf and change the logo line to read:

  
logo=/usr/share/icons/hicolor/64x64/devices/archlinux-icon-crystal-64.svg



 Installing the system monitor Conky on the Desktop gives us real time status on the amount of memory being used and many other items. Copy and Paste, or use scp, the following Conky configuration file to /root/.conkyrc:

  
# /root/.conkyrc :
# Conky, a system monitor, based on torsmo

background yes
use_xft yes
xftfont Sans:size=8
xftalpha 1
update_interval 2.5
total_run_times 0
own_window yes
own_window_transparent yes
own_window_type desktop
own_window_hints undecorated,below,sticky,skip_taskbar,skip_pager
double_buffer yes
maximum_width 500
minimum_size 200 200
draw_shades no
draw_outline no
draw_borders no
draw_graph_borders yes
default_color white
default_shade_color blue
default_outline_color white
alignment top_right
gap_x 12
gap_y 120
no_buffers yes
uppercase no
cpu_avg_samples 2
override_utf8_locale no

TEXT
${font sans-serif:bold:size=10}RASPBERRY PI${hr 2}
${font sans-serif:normal:size=8}$sysname $kernel $alignr $machine
Host:$alignr$nodename
Uptime:$alignr$uptime

${font sans-serif:bold:size=10}ARM11 @ $freq MHz${hr 2}
${font sans-serif:normal:size=8}CPU Usage: $alignr${cpu cpu1} %
${cpubar cpu1}

${font sans-serif:bold:size=10}MEMORY${hr 2}
${font sans-serif:normal:size=8}RAM $alignr $mem / $memmax $alignr $memperc %
$membar
${font sans-serif:normal:size=8}SWAP $alignr $swap / $swapmax $alignr $swapperc %
$swapbar

${font sans-serif:bold:size=10}SD${hr 2}
${font sans-serif:normal:size=8}/$alignc ${fs_used /} / ${fs_size /} $alignr ${fs_used_perc /} %
${fs_bar /}

${font sans-serif:bold:size=10}TOP PROCESSES [Total: $processes]${hr 2}
${font sans-serif:normal:size=8}${top name 1}${alignr}${top cpu 1} % ${top mem 1} %
$font${top name 2}${alignr}${top cpu 2} % ${top mem 2} %
$font${top name 3}${alignr}${top cpu 3} % ${top mem 3} %
$font${top name 4}${alignr}${top cpu 4} % ${top mem 4} %

${font sans-serif:bold:size=10}NETWORK${hr 2}
${font sans-serif:normal:size=8}IP address: $alignr ${addr eth0}
DwnLS:  ${downspeed eth0} $alignr total: ${totaldown eth0}
UpLS:    ${upspeed eth0} $alignr total: ${totalup eth0}

${font sans-serif:bold:size=10}USERS${hr 2}
$user_names

Desktop Icons:

A Desktop is no such thing without having Icons on it. The program iDesk is used by several Desktop Environments to manage icons. Use 'default.lnk' as a template to create other desktop icons:

  
# pacman -S idesk xdialog
# mkdir ~/.idesktop
# cp /usr/share/idesk/dot.ideskrc ~/.ideskrc
# cp /usr/share/idesk/default.lnk ~/.idesktop/

 


Desktop Background with Feh:

  
# mkdir ~/Wallpaper
# cd ~/Wallpaper

; Find a wallpaper on the net then download with wget, such as this
; one from the Doom on the RPi tutorial by J. McConnell (See the
; references below):
# wget http://dlnmh9ip6v2uc.cloudfront.net/tutorialimages/RaspberryPi/pi_wallpaper.jpg
# feh --bg-scale /root/Wallpaper/pi_wallpaper.jpg


Configure Fluxbox Wallpaper and Icons:

For the desktop icons and background to actually appear they need to be enabled in the Fluxbox start up script, /root/.fluxbox/startup. Add the following lines right before 'exec fluxbox':

  
# /root/.fluxbox/startup :
# ...
# System monitor:
conky&

# Desktop Icons:
idesk&

# Wallpaper:
sh ~/.fehbg&
# ...
  

Give the RPi a static IP address:

When running any kind of server on the RPi it needs to have a static IP address, so that the router knows where to route the inbound traffic. The files /etc/conf.d/network and /etc/systemd/system/network.service need to be created then enabled. The file /etc/resolv.conf needs to be modified to point to the local router. My local network is on the IP section 172.16.xxx.xxx. This has prevented conflicts with the common 192.168.xxx.xxx and the vast 10.xxx.xxx.xxx spaces when traveling, when connecting back home. 172.16.17.1 is my router, substitute yours as required:

   
# /etc/resolv.conf :
nameserver 172.16.17.1


   
# /etc/conf.d/network :

interface=eth0

# The static IP address we want to use:
address=172.16.17.130

netmask=24
broadcast=172.16.17.255

# The local router:
gateway=172.16.17.1


   
# /etc/systemd/system/network.service :

[Unit]
Description=Network Connectivity
Wants=network.target
Before=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=/etc/conf.d/network
ExecStart=/sbin/ip link set dev ${interface} up
ExecStart=/sbin/ip addr add ${address}/${netmask} broadcast ${broadcast} dev ${interface}
ExecStart=/sbin/ip route add default via ${gateway}
ExecStop=/sbin/ip addr flush dev ${interface}
ExecStop=/sbin/ip link set dev ${interface} down

[Install]
WantedBy=multi-user.target


   
; Enable the new static IP network service:
# systemctl enable network.service

; Disable the old DHCP network service:
# systemctl disable dhcpcd@eth0.service

# reboot
; When the system comes back up, if all goes well,
; it will be on the new static IP address.

; If it goes bad you will have to connect monitor
; and keyboard to the RPi to fix the problem.


Install sudo or sux and add your first user account:

For the ease of system installation up to this point we have been running under the Super User 'root' account. It is foolish to run under this account for day-to-day operations due to the possibility of some 'oops' happening that could destroy the system. Set up a normal user account for use during day-to-day operations. Setup as many accounts as you require. Here I set up an account called 'test' using the 'adduser' command:

   
[root@erlang1 ~]# adduser test

  Login name for new user: test

  User ID ('UID') [ defaults to next available ]:

  Initial group [ users ]:

  Additional groups (comma separated) []: audio,games,locate,lp,optical,power,scanner,storage,video,usb,wheel

  Home directory [ /home/test ]

  Shell [ /bin/bash ]

  Expiry date (YYYY-MM-DD) []:

  New account will be created as follows:

  ---------------------------------------
  Login name.......:  test
  UID..............:  [ Next available ]
  Initial group....:  users
  Additional groups:  [ None ]
  Home directory...:  /home/test
  Shell............:  /bin/bash
  Expiry date......:  [ Never ]

  This is it... if you want to bail out, hit Control-C.  Otherwise, press
  ENTER to go ahead and make the account.

  Creating new account...

  Changing finger information for test.
  Name []: Test
  Office []:
  Office Phone []:
  Home Phone []:

  Finger information changed.
  Enter new UNIX password:
  Retype new UNIX password:
  passwd: password updated successfully

  Account setup complete.
[root@erlang1 ~]#
   

There are two common different philosophies on how to approach the problem of running potentially dangerous system commands. One is to enable the Super User account on a trusted normal user account by a special command 'su' or 'sux'. The other is to use the 'sudo' prefix to any potentially dangerous system command issued from a normal user account, that has had sudo permission enabled for that user. Using sux is easiest but the most dangrous.

Historically 'su' stood for Super User (Also known as 'Big Wheel', hence the 'wheel' group used in the account setup), however in our world of Political Correctness it has evolved to mean 'Substitute User'. We would not want to hurt anyone's feelings that might be doing system administration. Sigh.

sux is a wrapper around su which will transfer your X credentials, allowing X and non-X programs to be run. By default, for security reasons, root will be unable to connect to a non-root user's X server without the logged-in user credentials. Running sux as root has no effect at root permissions are already active. Running sux as a normal user will request the root users password:
   
[test@erlang1 ~] $ sux
password: [Enter the root user's password.]
[root@erlang1 test]# [Note that the default prompt has changed from '$' to '#' as long as you are running as root.  Use 'exit' to return to normal user status.]
# exit
[test@erlang1 ~]$
   

The alternative to using sux is to use sudo. Some sudo users will go as far as disabling/removing the root account so that it is impossible to login to it at all. It is up to you to decide which mentioned is best for your situation.

   
; Allows users with permission to perform root functions with sudo:
# pacman -S sudo

; Change sudo permissions for non-vi users:
# EDITOR=emacs visudo

; or for vi users:
# visudo

; Uncomment the line that says:  %wheel ALL=(ALL) ALL

; or if you don't want to type in your password every time you use sudo
; uncomment the line that says:

# reboot
; login as your with the user/pass you just created.

; Use 'sudo !!' to repeat the last command if you forgot to use sudo for that command.


The sections on Desktop Icons, Desktop Background and Configure Fluxbox need to be repeated for each user installed, changed appropriately for the new users home directory (~).

Install Development Tools:

Now that installation of the Terminal Server is complete, you may want to install the development tools to further your explorations of the Raspberry Pi:

   
# pacman -S base-devel bzr cvs ed git mecurial python2 subversion

   :: There are 12 members in group base-devel:
   :: Repository core
     1) autoconf  2) automake  3) binutils  4) bison  5) fakeroot  6) flex
     7) gcc  8) libtool  9) m4  10) make  11) patch  12) pkg-config

  Enter a selection (default=all):

Making use of the Arch User Repository (AUR):

The Arch User Repository (AUR) is a community-driven repository for Arch users. The program 'Packer', among other alternatives, searches and installs AUR packages from the AUR database. Once installed packer may be used in place of pacman to search for packages.

  
# mkdir ~/builds
# cd ~/builds

# wget http://aur.archlinux.org/packages/pa/packer/packer.tar.gz
# tar -zxvf packer.tar.gz

; Install packer:
; Add --asroot only if still running under the root account.
# cd packer
# makepkg -s --asroot
# makepkg -i --asroot
# EDITOR=emacs packer -S customizepkg

; Now that Packer is installed use it to update the local AUR
; database.

# packer -Syyu

; From now on use 'packer -Ss package_name' to search for
; packages in-place of using pacman.


Packer will ask about editing various files as they are being installed. If you do not want to be adding 'EDITOR=emacs' in front of each packer command invocation then add the following line to the file ~/.profile. It will take effect the next time you login or issue 'source ~/.profile':

  
# export EDITOR=emacs
# source ~/.profile

WiringPi library and GPIO access, for GPIO with C or command line:

If you got your RPi to play with to learn about computer hardware or to get a start into the world of Embedded Systems then install the WiringPi package to get the ability to access the GPIO hardware. Visit the Wiring Pi site for details. The Wiring Pi package attempts to make the code similar to standard Arduino code to easy portability. Installing WiringPi.

Wiring Pi C program examples are found in the directory /usr/share/wiringpi/examples/source_code. Copy that directory to a sub-directory of your normal user account and build them there:

   
; Make sure the package is the latest by using the 'y' sub-option to
; pacman.  It is unneeded if the local package database was recently
; updated via -Syyu:

# pacman -Sy wiringpi

; Test GPIO with Command line:

; Set the pin to output mode, and set it high/low:

# gpio mode 0 out
# gpio write 0 1
# gpio write 0 0

The Bleeding Edge:

At any given time newer kernels and/or firmware for the RPi may be available than what is shipping in the current Arch image file. If you like to take risks, and you think it might fix a bug you are having give it a try, after making a backup image of your SD card. Is the bug that you know better than the one that you don't?

    
; Find out our current kerenal version:
# uname -a
Linux erlang1 3.2.27-18-ARCH+ #1 PREEMPT Fri Dec 21 14:18:42 UTC 2012 armv6l GNU/Linux

; Install the up and coming kernel and RPi firmware.
; This may fix bugs, this may have new bugs.
; If there is none you get 'not found'.
# pacman -Sy linux-raspberrypi-next linux-headers-raspberrypi-next raspberrypi-firmware-next

; Find out our new current kernel version, if any:
# reboot
# uname -a

GPU/ARM Memory Split. How to get more RAM:

Recent RPi kernels and firmware have an experimental feature to dynamically spit the limited amount of RAM between the ARM CPU and the video GPU. To enable it edit the configuration files /boot/config.txt and /boot/cmdline.txt. A well commented version of config.txt, if a bit outdated, is on GitHub.

    
; Show the RAM we have now:
# free -h -t

; coherent_pool: Allocates from a pool of uncached memory that was reserved at boot time.
; http://www.kernel.org/doc/Documentation/DMA-API.txt

; Add the following to /boot/cmdline.txt :
coherent_pool=6M smsc95xx.turbo_mode=N


    
# Add the following to /boot/config.txt :

# gpu_mem: GPU memory in megabyte.
#          Sets the memory split between the
#          ARM and GPU.
#          ARM gets the remaining memory.
#          Min 16. Default 64

# gpu_mem_256: GPU memory in megabyte for the 256MB Raspberry Pi.
#              Ignored by the 512MB RP. Overrides
#              gpu_mem. Max 192. Default not set
#gpu_mem_256=160

# I'm not doing anything graphic intensive so I want more ARM RAM:
gpu_mem_256=64

# gpu_mem_512: GPU memory in megabyte for the 512MB Raspberry Pi.
#              Ignored by the 256MB RP.
#              Overrides gpu_mem.
#              Max 448. Default not set.
gpu_mem_512=316

# CMA - Dynamic Memory Split
# CMA is experimental it could go away or change in the future

# The Contiguous Memory Allocator (CMA) makes it possible for device
# drivers to allocate big contiguous chunks of memory after the system
# has booted. - http://lwn.net/Articles/461849

# cma_lwm: When GPU has less than cma_lwm (low water mark) memory
#          available it will request some from ARM.
cma_lwm=16

# cma_hwm: When GPU has more than cma_hwm (high water mark) memory
#          available it will release some to ARM.
cma_hwm=32

# The "online" shared memory is owned by the GPU. The "offline" shared
# memory is owned by ARM.  That means the first 16M is dedicated to GPU,
# to allow for initial allocations of framebuffer etc. which occur
# before the CMA driver has started.
# - http://www.raspberrypi.org/phpBB3/viewtopic.php?f=29&t=19334&start=125
cma_offline_start=16

; Show the RAM we have after the change:
# reboot
# free -h -t
 

Troubleshooting:

If something is not working correctly there are four places that may be interrogated for information. The first is the system journal via the 'journalctl' command. Without any parameters journalctl will show a lengthy spew of system events. 'journalctl -n' will show the most recent ten events. 'journalctl -f' will show events as they happen. See 'man jounralctl' on how to use the journal filter to see only events of interest.

Second 'systemctl' may be used to interrogate the state of the system, and enable/disable services. 'systemctl --failed' will show items with problems. 'systemctl list-units' and 'systemctl list-jobs' show the state of the system. 'man systemctl' gives more in-depth options.

Third, the standard GNU/Linux command 'dmesg' shows the state of the unit as booted.

Lastly, the /proc file system shows the state of the system and devices.

  
# journalctl
;  Not shown here

# journalctl -n
   -- Logs begin at Wed, 1969-12-31 19:00:02 EST, end at Mon, 2012-12-31 20:01:01 E
   Dec 31 17:01:01 erlang1 CROND[994]: pam_unix(crond:session): session closed for...
   Dec 31 18:01:02 erlang1 crond[1002]: pam_unix(crond:session): session opened for...
   Dec 31 18:01:02 erlang1 CROND[1003]: (root) CMD (run-parts /etc/cron.hourly)
   Dec 31 18:01:02 erlang1 CROND[1002]: pam_unix(crond:session): session closed for...
   Dec 31 19:01:01 erlang1 crond[1010]: pam_unix(crond:session): session opened for...
   Dec 31 19:01:01 erlang1 CROND[1011]: (root) CMD (run-parts /etc/cron.hourly)
   Dec 31 19:01:01 erlang1 CROND[1010]: pam_unix(crond:session): session closed for...
   Dec 31 20:01:01 erlang1 crond[1025]: pam_unix(crond:session): session opened for...
   Dec 31 20:01:01 erlang1 CROND[1026]: (root) CMD (run-parts /etc/cron.hourly)
   Dec 31 20:01:01 erlang1 CROND[1025]: pam_unix(crond:session): session closed for...

# journalctl -f
;  Not shown here

;  Not shown here
# systemctl
# systemctl -a
# systemctl --failed
# systemctl --list-units
# systemctl --list-units-files
# systemctl --list-jobs

# dmesg|grep -i vc
# dmesg|grep -i cma

# ls /proc
; to long to list here.

# cat /proc/vc-mem
   Videocore memory:
      Physical address: 0x00000000
      Length (bytes):   268435456

# cat /proc/version
   Linux version 3.2.27-18-ARCH+ (nobody@nitrogen) (gcc version 4.7.2 (GCC) ) #1 PREEMPT Fri Dec 21 14:18:42 UTC 2012

References:

The Raspberry Pi Foundation is a UK registered charity (Registration Number 1129409). See http://www.raspberrypi.org/trademark-rules on usage of the RPi trademarks and logos.

Wednesday, August 3, 2011

The current Temporal War on planet Earth. The August 2011 edition of Metrologia.

Going on here on Earth right now is a real Temporal War. This Temporal War is not Science-Fiction, it is about the future of Time itself, as in how it is measured. Right on time, the August 2011 issue of Metrologia has appeared. This issue, Volume 48, Number 4, is all about Modern Applications of Timescales:
  • Time, the SI and the Metre Convention
  • Role of the ITU-R in time scale definition and dissemination
  • Evolution of timescales from astronomy to physical metrology
  • Timescales at the BIPM
  • Local representations of UTC in national laboratories
  • Earth rotation monitoring, UT1 determination and prediction
  • Relativistic time transfer in the vicinity of the Earth and in the solar system
  • Solar time, legal time, time in use
  • Time scales, their users, and leap seconds
  • Time references in US and UK astronomical and navigational almanacs
  • Software for timescale applications
  • Timing in telecommunications networks
  • BeiDou Navigation Satellite System and its time scales
  • GNSS times and UTC
The article are free until the end of August 2011, after which time they go behind a paywall.

In this War there are several factions, such as those that believe that Time should be tied to the Earth rotation. The problem is that the Earth rotation is not constant. To keep our clocks on the wall synchronized the concept of Leap Seconds has to be introduced were a second is added or subtracted every six months from Universal Coordinated Time. From the Embedded System perspective this means an hour can legitimately have 61 seconds in it. Have you accounted for that in your system? An hour could also only have 59 seconds. Maintaining time spans across years becomes a problem, necessitating the use of tables of Leap Seconds.

Leap Seconds can currently be tracked at the Time Service Dept., U.S. Naval Observatory.

In one of the other major factions they want to do away completely with the concept of Leap Seconds. The clock-on-the-wall-time would slowly drift away from the look-out-the-window-at-the-Sun-Dial-time. This faction believes that it will simplify our lives as programmers.

There are also a few lessor factions at play. Based on input from the International Astronomical Union, the Bureau International des Poids et Mesures, the Consultative Committee for Time and Frequency (CCTF), the International Telecommunication Union (ITU), is close (in 2012) to the moment of making a decision on whether or not to decouple UTC from its tight link to the rotation of the Earth.

Who is going to win this Temporal War? I do not know, it might come down to the group with the biggest Ray-Gun. One thing is for sure, that in a year or two keeping accurate Time in our Embedded Systems might just get a lot harder. Follow the daily battle and decide which camp you want to put your future with, on the Leap Second List.

Sunday, February 27, 2011

The Anatomy of a Race Condition: Toyota vs AVR XMega

NASA has released their report on Toyota's sudden acceleration problem. The report indicates that there was no problems found with the electronics, hardware or software.
They blame the issue on user error and bad floor mats. As no problem was found, we can be 100% certain that no problems at all exist in the hundreds of thousands of lines of software code, in the vehicles electronics, right?
"Because proof that the ETCS-i caused the reported UAs [Unintended Accelerations] was not found does not mean it could not occur." - pg 17.
"Today's vehicles are sufficiently complex that no reasonable amount of analysis or testing can prove electronics and software have no errors. Therefore, absence of proof that the ETCS-i has caused a UA does not vindicate the system." - pg. 20.
Something that I find most annoying is that the areas where the embedded system hardware is discussed the most, is the area of the most redaction (blacked out sections). Why?

If a problem was to still be lurking, unfound, it could be what is known as a Software Race Condition. What does a software race condition actually look like?

We can find an easy example to pick apart in the AVR-LibC bug tracker, bug#29774: "Prologue/epilogue stack pointer manipulation not interrupt safe in [AVR] XMega".

To understand the problem here, you need a bit of historical background. In AVR's prior to the XMega, when an Enable Interrupt instruction was executed, the instruction following the Enable was guaranteed to execute with interrupts still turned off. In the mists of time someone thought it was a cool hack to save an instruction cycle by restoring half of the stack pointer, enabling interrupts,then restoring the other half of the stack pointer. The problem with such novel hacks is they invariably come back to bite you in the future.
Like a bad Soap-Opera story you can probably already see where thisis going? In the XMega when interrupts are enabled the following instruction is not guaranteed to execute before an interrupt occurs. Now the stage is set for the race condition.

The current generation of XMega parts can run code in a singlec ycle at up to 32 MHz. That means we have at minimum one 1/32 MHz, or31.25 nano-second window for the software race to happen. In a complex system there are probably more than one interrupt enable happening. To add more pain, the XMega can nest interrupts three levels deep.

You see that if an interrupt occurs exactly at the point where interrupts are enabled, only half of the stack pointer has been restored. So the new interrupt saves its registers someplace,odds are high it is not the right place! The new interrupt eventually returns, tries to restore its registers,from someplace that might have been read-only-memory, and bang we are off to the races with a crashed system doing who knows what.Maybe a full open throttle? No message shows up in any logs because there was no event logged through a call to the event logging system,as this was never an anticipated event; "systematic software malfunction in the main central processor unit (CPU) that is not detected by the monitor system".

Due to the short length of the 31.25 ns race window possibility, a crash may never happen, may happen every 18 hours and 22 minutes, or as often as I win the lottery [Give Wheeling Systems a try]. It could take some certain combination of options and users actions to cause the conditions of enabling interrupts while returning from an interrupt, while getting an interrupt. Turn the radio dial, press the brake peddle while the over automated headlights turn themselves on perhaps?

While the this bug was actually reported in the AVR-LibC bug tracker, the problem is actually within the AVR port of GCC. Specifically the file gcc-version/gcc/config/avr/libgcc.S.

I fixed my copy of WinAVR-GCC with a hex editor, so my projects would not suffer for this bug. Realistically how many other people will have done that? Not many I would guess. It is impossible to tell from the hideous Atmel website (all glitz, no useful information) what the state of the bug truly might be today.

For those that want to fix the problem the solution is to simply write the lower half of the stack pointer first: "To prevent corruption when updating the Stack Pointer from software, a write to SPL will automatically disable interrupts for up to 4 instructions or until the next I/O memory write". As GCC does not yet nativity support the XMega, the XMega features are maintained as a set of patches. Those patches have been updated to fix the problem.

It would be easy for some to say that one should not use Open Source compilers for real production projects, as I've seen a few prominent people state. I have a copy of IAR's AVR compiler, at no small price tag, that I've seen produce complete crap for output. So just because you paid, perhaps a lot of money, for it doesn't mean it is error free.

Some standards require that the code generated by the tool be inspected. At what level of detail is the question? I once actually ran into an assembler that produced correct listings, however the generated .HEX file was wrong. That problem took days to find. Disassembling, with an independent tool from a different provider, the generated .HEX file is one option, however it is not always easy to figure out what optimized compiler code is doing in a reasonable amount of time.

What kind of tool problems have you ran into?

Now returning back to Toyota. Section 6.7.1.2 tells us that a Renesas,formally NEC, V850E1, and GreenHills ISO/ANSI Compiler are used for the control software of interest to us. Alas the section that might shed light on Race Conditions is completely redacted.

Starting on page 112 Tin Whiskers become a prominent failure mechanism. Keep in mind that according to page 19 of the report only six vehicles were analyzed. The whisker problem discussed from a seventh vehicle accelerator assembly only.

Tin (and Other Metal Whisker) Whisker are such a problem that NASA has given them their own Homepage. Tin Whiskering on PCBA Capacitors in Storage by Terry Munson gives a different, but still depressing, view of the Tin Whisker problem.

The comments about the whiskers over at Circuit Assembly Magazine are also educational, that I recommend that you read, for example:

Brett Emison pokes several holes in the NASA report, What NASA's Report Said About Toyota Sudden Acceleration for example:
The NHTSA/NASA report did little to address issues documented by drivers who actually experienced an unexplained sudden unintended acceleration event.
NASA's findings do not solve the question of what caused Kevin Haggerty's well documented sudden acceleration event. Haggerty owned a 2007 Toyota Avalon that experienced at least 5 different sudden acceleration events. Haggerty did not have accessory floor mats and his OEM mats were secured in place. Sticky pedals couldn't have caused the problem because he didn't have his foot on the pedal. On Haggerty's final incident, he was actually able to drive the vehicle while the engine was racing out of control into his local Toyota dealership.
He got to the parking lot, shifted to neutral and stopped the car with its brake smoking and engine racing out of control. He got out of the car and the engine was still racing (no pedal misapplication) Service technicians were able to look at he car and confirm the unintended acceleration was not caused by floor mats, sticking pedals or driver error. They also confirmed no computer error codes (meaning the computer was not detecting whatever was causing the problem).
Bob Landman
LDF Coatings, LLC
http://www.ldfcoatings.com
I do wonder if NASA applied their Software Safety Guidebook to Toyota's source code? I assume they did not, as it is not listed among the techniques they did apply.

...Tune in tomorrow for an other episode of As The Stack Churns...

Sunday, November 7, 2010

Microsoft Open Sources F#. Will Functional Languages save us?

Michael Barr recently [September, 2010] wrote a column on The Sad State of Embedded Software Process, which tells us that the first embedded system was done about forty years ago, and we still are doing a collectively poor job at building them.

The VDC Research Report What languages do you use to develop software?, tells us that after those forty years 81.9% of our Embedded Systems are still using the language 'C'.

Perhaps there is a connection to using a forty year old language,with not having any great improvements in reliability after forty years?

Why can't any modern language gain significant traction in the Embedded Space? Whom has the clout to develop and promote a new, safe, language for limited resource hard real time (dead line can not be missed without catastrophic consequences) Embedded Systems that will actually gain traction?

I do not have those answers. I'm sure I could come up with a language, as I'm sure you could too, but how to get it used is the question?

Some of the 'safest' languages don't even make VDC's list, unless they are perhaps grouped into the 'Other' category of 11.5%, languages like Esterel and Erlang for example.

To me Esterel seems to be the most obscure language that is used in the most safety critical applications such as flight controls and railroad safety. Esterel has the company Esterel Technologies behind it, as well as an Open Source implementation of an Esterel Compiler. To get a feel of what it is like at the top end of the Safety Critical scale check out some of the Webinars such as Software and Hardware Foundations for Safety Critical.

Next in my obscure list of safe languages would be Erlang. Erlang runs a large percentage of the worlds telephone network, and is considered a 'soft' real time system (dead line can be missed without catastrophic consequences). There has been no larger and longer running networked system than the telephone company. I'm always amused to see such things as 'new' networking technologies rediscovering and reinventing problems and solutions that Erlang solved over a decade ago.


Because Erlang almost has what I'm looking for such as being small (in the sense of running on small micros), and meeting hard real time deadlines, Erlang is the one Functional Language that I play with the most, but does not meet those two criteria.

Functional Languages, in contrast to Procedural Languages like C, take a different approach and mind set to creating safe code.

WikiPedia details the concepts behind Functional programming, the two most important concepts being:

  • Functions have no side effects; a function can not change a global variable for exmple.
  • Data is immutable. Variables are single assignment; Variables don't change value after being set the first time.

Eliminating side effects can make it much easier to understand and predict the behavior of a program, because you are never asking yourself questions like "What is the value of X right now?", X can only be assigned once, so you always know its value.

Professor Stephen A. Edwards whom is a tenured associate professor in the Computer Science Department of Columbia University, once told me that freshman students to his class that had no programming experience picked up Functional Languages, and languages like Esterel quickly, while people that had experience with programming languages such as 'C', struggled to make the transition. Variables that never vary their value can be tough to get your mind around at first, if you programmed in 'C'.

Professor Edwards has put fourth his paper High-level Synthesis from Functional Languages proposing that Functional Languages can be directly compiled to FPGA hardware.

Functional Languages where destined to live in obscurity until this week [Nov. 5th, 2010] when Don Syme of Microsoft announced that the F# Compiler plus Library Source Code is now available as a free download.

While F#,which has its roots in Objective CAML, has little hope of running on a microprocessor based Embedded System, Functional Languages just got a big push in to being more main stream with potential integration to things like Visual Studio. It is my hope that soon there will be a micro friendly Functional Language, that gains widespread adoption.

Will Functional Languages save us? We can only hope.

In closing it is worth mentioning that UML comes in at 5.8% on the VDC report, so I thought I'd mention Miro Samek's new graphical modeling tool QM, that I'm just starting to learn myself. The QM tool is available now for a free download and free, unrestricted use.

Sunday, July 25, 2010

"Killed by Code: Software Transparency in Implantable Medical Devices"

The Software Freedom Law Center (SFLC) has recently, July 21, 2010, published an interesting paper: Killed by Code: Software Transparency in Implantable Medical Devices.


SFLC makes a case for why Safety Critical systems should all have their source code available for public review.  They are promoting Free and Open Source Software (FOSS) be mandated for medical devices, and all devices in general; myself I want to see Toyota's source code to look for the race condition [unfortunate terminology in this case] of unintended acceleration that I believe exists.


I covered the issues related to the FDA's promotion of FOSS in medical devices previously: FDA says commercial software kills, but Open Source won't? and 200,000 Infusion Pumps ordered destroyed by FDA, due to software defects and other problems.


Philosophically I agree with SFLC's position, and in Utopia it would work out well that all devices have Free Open Source Code.  However in the real world of cooperate greed where only the bottom line of next quarter is all that maters, I do not see medical device manufactures, or automotive manufactures, giving up what they would consider 'Trade Secrets' so that their competition could use them.  A more workable solution would be an independent auditing agency, which still must be held to the highest standards to prevent such things as cooperate kickbacks.  Nor would we want the agency to be funded by the people that are having their code audited.


Also just because a project is FOSS does not mean it is secure.  One only has to take a look at the recent events of the Unreal IRC Server Project.



"We found out that the Unreal3.2.8.1.tar.gz file on our mirrors has been replaced quite a while ago with a version with a backdoor (Trojan) in it.".


This incident demonstrates that the entire chain of custody from the Source Code to the code on the device must be traceable, so that the code that is really being run, is the code that came from the legitimate sources of the project.


Unreal had this to say about addressing the issue, to give some ideas of what needs to be done:


Posted by Syzop on June 14, 2010, 3:36 pm EDT

After receiving many questions of what we are doing with regards to the hack incident, here's my reply:


First, we now PGP/GPG sign releases. Our GPG key is releases@unrealircd.com (0x9FF03937). When downloading UnrealIRCd you will be given instructions on how to verify the integrity of the file.


Second, we're now isolating/shielding the main site from the rest, and making parts unmodifiable, to prevent catastrophes in case of a break-in.


Third, we added several methods of detection when files and other data is modified.


Fourth, we'll only serve the files from the main site for now. While the mirror admins did not have any blame in this, it does mean we only have to protect our own site(s).


And finally we did some other things which I won't mention here.


In short: we've really tightened security since the break-in to make sure this will never ever happen again. As you may understand, we really can't afford a repeat of this incident.


On an unrelated side note, I find the claims in various media that this security incident indicates that Linux and Open Source cannot be trusted and that Microsoft and closed-software is better really silly. It lacks any foundation. A hacker, once in, could just as easily have inserted the backdoor in Windows software. In fact, it is *THANKS* to it being Open Source that this backdoor got noticed, though - I fully agree - much too late.


Hash Deep is helpful to find development files that have changed unintentionally, due to either simple disk corruption or malicious intent.  On Embedded Devices themselves I run CRC checks of the code.  SRecord is a big help in this area.



Returning back to SLFC's paper.  SFLC points to the case of Riegel v. Medtroni in February 2008, stating:




"Since the FDA is a federal agency, its authority supersedes state law. Based on the concept of preemption, the Supreme Court held that damages actions permitted under state tort law could not be filed against device manufacturers deemed to be in compliance with the FDA, even in the event of gross negligence."...


"It is clear that medical device manufacturers have responsibilities that extend far beyond FDA approval and that many companies have failed to meet their obligations," William H. Maisel said in recent congressional testimony on the Medical Device Reform bill.50 "Yet, the U.S. Supreme Court ruled in their February 2008 decision, Riegel v. Medtronic, that manufacturers could not be sued under state law by patients harmed by product defects from FDA-approved medical devices ... . [C]onsumers are unable to seek compensation from manufacturers for their injuries, lost wages, or health expenses. Most importantly, the Riegel decision eliminates an important consumer safeguard - the threat of manufacturer liability - and will lead to less safe medical devices and an increased number of patient injuries."



Here is part of that actual US Supreme Court Decision, that I took from Cornell:



"The Riegels contend that the duties underlying negligence, strict-liability, and implied-warranty claims are not pre-empted even if they impose " 'requirements,' " because general common-law duties are not requirements maintained " 'with respect to devices.' " Brief for Petitioner 34-36. Again, a majority of this Court suggested otherwise in Lohr. See 518 U. S., at 504-505 (opinion of Breyer, J.); id., at 514 (opinion of O'Connor, J., joined by Rehnquist, C. J., and Scalia and Thomas, JJ.).6 And with good reason. The language of the statute does not bear the Riegels' reading. The MDA provides that no State "may establish or continue in effect with respect to a device ... any requirement" relating to safety or effectiveness that is different from, or in addition to, federal requirements. §360k(a) (emphasis added). The Riegels' suit depends upon New York's "continu[ing] in effect" general tort duties "with respect to " Medtronic's catheter. Nothing in the statutory text suggests that the pre-empted state requirement must apply only to the relevant device, or only to medical devices and not to all products and all actions in general." --- Justice Scalia, Opinion of the Court; RIEGEL v. MEDTRONIC, INC. (No. 06-179)
451 F. 3d 104, affirmed.



Not being a lawyer, it seems to me that the Court just gave medical device manufactures a free pass to make defective products because there is no longer a threat of liability?  We have all seen the End User Licenses Agreements (EULA) for software that all say, using lots of Lawyer Weasel Words, "No mater what this software does to you and yours.  It is not our fault.  To bad for you."  Is this where hardware is now?