Showing posts with label Jack Ganssle. Show all posts
Showing posts with label Jack Ganssle. Show all posts

Saturday, February 4, 2012

Even "Design Patterns" still have bugs. A common Queue bug.

Recently I picked up the book Design Patterns for Embedded Systems in C: An Embedded Software Engineering Toolkit by Bruce Powel Douglass. I thought I might pick up some new techniques by reading this book. Always looking for that non-existent magic bullet for creating bug free code all of the time. Alas I was disappointed at the very first Design Pattern, the Queue. The queue design pattern as implemented in the book has a significant bug. The second queue design pattern, will show the bug even faster.

The queue design pattern is the classic first-in/first-out single writer, single reader queue. This 'Design Pattern' has been around for decades. I first used it on a Motorola/Hitachi 6301 in assembly language. Today I use Dave Hylands 'cbuf.h' to implement my queues.

Typically such queses are used where the writer is inside an interrupt routine receiving data from an external process, via something like a UART, Ethernet etc. and the reader is outside of the interrupt. Such a queue allows for the data to be received when presented from an asynchronous source, so it is not lost, rather than processing the data in the interrupt, which would length its run time (always a bad thing). The reader of the queue processes the data when it has the time and resources to do so, outside of the interrupt.

So what is this nasty queue bug? It is the asynchronous usage of the 'size' counter both inside, and outside of the interrupt routine without any type of protection:

#define QUE_SIZE (512U)
uint16_t size_u16;
void que_write( uint8_t const data_u8 )
{
...
++size_u16;
...
}

uint8_t que_read( void )
{
...
--size_u16;
...
}

On parts where 'size' is incremented atomically, say as a 16 or 32 bit indivisible operation there is no problem at all. Our nasty race condition bug will show itself at apparently completely random times on 8-bit machines.

Jack Ganssle went into the gory details of this class of race condition in his article on Asynchronicity (Also take a look at Herb Sutter's multi-part article on Lock-Free Code: A False Sense of Security), so I'll just summarize the problem here. Which is, if the que_write() interrupt happens just as que_read() runs, while the queue size count is 0x00FF, the size will end up wrong. que_read() reads the value of 0x00FF which is interrupted by que_write() that increments the value to 0x0100. que_read() not knowing this happened decrements the 'size' count and saves it as 0x1FE or 0x00FE depending on the implementation. A byte has been lost, or 256 have been gained, in the 'size' count. Things could deteriorate farther when the head/tail full/empty flags no longer end up matching the 'size' count.

Also, properly sizing a queue can be tricky. If it is to small, the queue will overflow and data will be lost. To large and memory space is wasted.

In the case of 'cbuf.h' the size must always be a power of two. As I've covered before I like to use compile time assertions to keep from shooting myself in the foot. I use the following to make sure a non-power-of-two will fail to compile:

#define UART0_RX_BUFFER_SIZE (128U) /* Size of receive buffer [Defined in my project file] */

/* Make sure buffer is sized to be a power of two [In my actual code, in different file]: */
#define UART0_RX_BUFFER_MASK ( UART0_RX_BUFFER_SIZE - 1U )
#if ( UART0_RX_BUFFER_SIZE & UART0_RX_BUFFER_MASK )
#error RX buffer size is not a power of 2
#endif

Something else that really bugs me about the book is that I ordered it in eBook format rather paper format. Elsevier eBook format is some creation of theirs that can not be read outside of their special reader for Windows or the Mac. So if you want to read on Linux, xBSD or a tablet, your just out of luck. Also as an anti-piracy measure if you print out anything, it comes out fuzzy. To make matters even worse I found they charged my credit card $1.19 for a fright charge! A freight charge for an eBook?? You'll be far better off to buy the real paper book, take a band-saw to the binding and scan it yourself that use this eBook carp. Maybe you'll want to join the Elsevier Boycott as well...


Sunday, May 8, 2011

NASA Goddard Space Flight Center (GSFC) Software Assurance

Back in the blog I wrote, The Anatomy of a Race Condition: Toyota vs AVR XMega I mentioned the NASA Software Safety Guidebook. Seems that link went viral, as it popped up lots of other places the next day as people spread it around.

I was a bit surprised, but glad, to find that the Software Safety site and this Software Safety blog are listed number one and number two by SEMRush (a 'Competitor Research Service'), beating out NASA themselves at number four, for the term "Software Safety".

Will today's link to NASA's Goddard Space Flight Center's Software Assurance page go viral as well?
"The NASA Goddard Space Flight Center (GSFC) Software Assurance Website provides tools, procedures and training materials for software and safety assurance personnel, software engineers, as well as program and project managers."
Of the most practical day to day value are the numerous Checklists, for example one on Code Inspections, and the Forms and Templates. There are also examples of Procedures, Guidelines, Work Instructions, links to tools etc. The Automated Requirement Measurement (ARM) Tool has been developed as aid to "writing the requirements right," not "writing the right requirements". As of this moment Humans still create the initial requirements for any device or Embedded System, and Humans are prone to errors. Unless an other Human catches the error(s) in a requirement document early in the development life cycle, no downstream tool will clean up the mess or mitigate the cost overruns.

NASA Site for On-line Learning and Resources (SOLAR), has some on line training as well. The Defense Acquisition Guidebook also has a section on Software Safety online training; MIL-STD-882D, "DoD Standard Practice for System Safety".

While in the Software Safety area of space flight check out Software Safety and Rocket Science by Gerard J. Holzmann in ERCIM News. Issue #75 covered Safety-Critical Software.

In my last blog entry I mentioned Circuit Cellar Magazine. In the April 2011 issue George Novacek took on the DO-178B software assurance standard. George details the standard, as best as you can in the allowed couple of pages, then seemed to imply at the end, that doing all of this paper work doesn't make the system much safer. Over the years I've seen both sides. Not having good written requirements leads to nothing but never ending project changes (changes are normal, but if you don't know what your setting out to make, you never know when you are done), cost overruns, and missed deadlines. On the other side having so much paper work that you can never actually ship a product out the door puts any company selling Embedded Systems out of business.

Sunday, February 1, 2009

Embedded Systems A Volatile Business

At the end of Embedded systems - a volatile business Jack Ganssle says: Bob Paddock sent me the link [Embedded System Compilers generate dangerous code], and thereby wrecked my day.

Always happy to wreck a day, by pointing out Software Safety issues. :-)

I believe that Jack and I see eye-to-eye on most embedded system issues, but I have to disagree in one area. In his column Skip bugging to speed delivery Jack stated: We only inspect new code because there just isn't time to pour over a million lines of stuff inherited in an acquisition.

Development times are always shorter that we want, so not wanting to look at inherited code may seem like a good shortcut to save time. Myself I do not see it that way. Just because a bug is old, does not mean that it is something to be ignored. The Zune problem, that we have already covered here is a perfect example. The Board Support Package was 'inherited' so it seems no one bothered to inspect the code.

Atmel's new TouchLib product shows us an other example of code that can't be inspected at all. TouchLib is only available as a binary library file. The source code is not available, even under NDA (I asked), to allow for inspection. In Atmel's view this is their way of protecting their Intellectual Property.

If you take the trouble to actually read the three different TouchLib licenses, from registering, installing and in the TouchLib archive, all three more or less say: "If this product screws up it is not our problem".

Why would I want to use code that I can not verify as safe and correct? My company is the one that would have to deal with potential warranty issues, calls from angry customers etc. I should just tell them "Sorry, we used software that we got off the Internet for free, but have no idea how it works"? What would your reaction to that be as a customer? I don't think you'd be very happy. Unhappy customers don't come back as paying customers.