Building product variations without duplication of code and configuration. Makefile Tip #1

Mitch Frazier published a How To, Turn Make Options into Tool Flags, in The Linux Journal a couple of years ago, that I have found very useful in my Embedded development projects.

Using Mitch's technique I can have several different top level Makefiles to define options, source locations, and target output directories, that then includes a common Makefile called Makefile.mak. This allows me to build several variations of a product from the same set of sources, while keeping their options distinct and outputs separate.

Mitch's original technique is not MISRA friendly, as it relies on using 'ifdef', that is a missing option is considered a disabled option. MISRA does not allow this because the missing option may have been an oversight. Here is my modified MISRA friendly version:

Create a file called MakefileOptions.inc with the following code:

#---------------------- Cut Line ----------------------------------#
# Option names that start with a minus sign are disabled by default,
# option names without a minus sign are enabled by default.

#CONFIG_OPTIONS=\
#    OPTION_A \
#    -OPTION_B \
#    -OPTION_C

SET_CONFIG_OPTIONS=$(filter-out -%,$(CONFIG_OPTIONS))
UNSET_CONFIG_OPTIONS=$(patsubst -%,%,$(filter -%,$(CONFIG_OPTIONS)))
ALL_CONFIG_OPTIONS=$(SET_CONFIG_OPTIONS) $(UNSET_CONFIG_OPTIONS)
#$(info Set: $(SET_CONFIG_OPTIONS))
#$(info Unset: $(UNSET_CONFIG_OPTIONS))

# Turn config options into make variables.
$(foreach cfg,$(SET_CONFIG_OPTIONS),$(eval $(cfg)=1))
$(foreach cfg,$(UNSET_CONFIG_OPTIONS),$(eval $(cfg)=))

# Make sure none of the options are set to anything except 1 or blank.
# Using "make OPTION=0" doesn't work, since "0" is set, you need "make OPTION=".
$(foreach cfg,$(ALL_CONFIG_OPTIONS), \
    $(if $(patsubst %1,%,$(value $(cfg))), \
        $(error Use "$(cfg)=1" OR "$(cfg)=" not "$(cfg)=$(value $(cfg))")))

# Turn them into tool flags (-D).
#                                                   if   cfg > 0       True:      False:
TOOL_DEFINES+=$(foreach cfg,$(ALL_CONFIG_OPTIONS),$(if $(value $(cfg)),-D$(cfg)=1,-D$(cfg)=0))
#$(info $(TOOL_DEFINES))
#---------------------- Cut Line ----------------------------------#

A the top of your normal Makefile 'include' the above file like so:

include MakefileOptions.inc

The for a typical project I'll have a project uniquely named makefile, such as 'widget1.mak', that goes along these lines:

#---------------------- Cut Line ----------------------------------#
# Hey Emacs, this is a -*- makefile -*-
#----------------------------------------------------------------------------
#
TARGET=Widget1
F_CPU=4000000
MCU=atxmega128a1

# Option names that start with a minus sign are disabled by default,
# option names without a minus sign are enabled by default.
# Values on the command line override these:
CONFIG_OPTIONS=\
-ENABLE_CHIRP \
ENABLE_LEDS \
HAVE_RADIO \
USE_CRC_FLASH_CHECK \
-USE_DEBUGGING \
-USE_MOTION_SETTING_SCREEN

# List C source files here. (C dependencies are automatically generated.)
SRC = Accel/accel.c \
MENU_WIDGET1/menu.c \
....

ASRC = HARDWARE_XMega/sp_ReadFuseByte.S HARDWARE_XMega/sp_commoncmd.S ...

EXTRAINCDIRS = MENU_WIDGET1 HARDWARE Accel Alarm Backlight Battery CRC ...

include Makefile.mak

#---------------------- Cut Line ----------------------------------#

Each project variation gets a similar file that is invoked with 'make -f widget1.mak'.

An example C usage:

/*---------------------- Cut Line ----------------------------------*/
#if( USE_CRC_FLASH_CHECK > 0 )
extern uint16_t __data_load_end[1]; /* Defined by the linker script.  Set to address of last byte of .text+.data section */
#include 
static __inline__ uint16_t crc_flash_check_has_error( void );
static __inline__ uint16_t crc_flash_check_has_error( void )
{
...
}
#endif
/*---------------------- Cut Line ----------------------------------*/

Automatic Serial Number for Test Driven Development. Makefile Tip #0

With the pending release of James Grenning's new book Test Driven Development for Embedded C there has been a lot interest in TDD in the embedded space.

Jack Ganssle did a two part interview with Mr. Grenning this summer [2010], on the subject of TDD in Embedded Development:

• An interview with James Grenning, Part 1
• An interview with James Grenning, Part 2

Something that I have found very useful is an automatic build or serial number generated with each invocation of Make. The number can be accessed from C for use in version messages, or in the output from the invocation of Make directly.

This technique came out of a discussion by Matt VanDeWerk and Dave Hylands on the AVR-GCC list in 2005.

Create a file called 'buildnumber' that says this in the same directory as your Makefile:

#-------------------Cut Line -----------------------------------#
# This is the current build number of the latest build
# DO NOT EDIT THIS FILE - it is updated automagically
# by the build process.

BUILDNUMBER=0
#-------------------Cut Line -----------------------------------#

In the Makefile add these lines:

#-------------------Cut Line -----------------------------------#
# Add before any 'all:', or other target:
# Auto-incrementing number, which may be used in version numbers:
include buildnumber
NEW_BUILDNUMBER = $(shell echo $$(( $(BUILDNUMBER) + 1 )))

CDEFS += -DNEW_BUILDNUMBER=$(NEW_BUILDNUMBER)
CDEFS += -DNEW_BUILDNUMBER_TEXT=\"$(NEW_BUILDNUMBER)\"

%.elf: $(OBJ)
 @echo Test run: $(NEW_BUILDNUMBER)
# Do normal stuff here, append these two lines, which rewrite the
# buildnumber file for the next Make invocation:
 @echo "# Automatically generated file - do not edit" > buildnumber
 @echo "BUILDNUMBER=$(NEW_BUILDNUMBER)" >> buildnumber
#-------------------Cut Line -----------------------------------#

/*-------------------Cut Line -----------------------------------*/
/* C Example usage:
static uint8_t const BuildNumber_str[] = NEW_BUILDNUMBER_TEXT; /* Build number */
/*-------------------Cut Line -----------------------------------*/

Make does not get enough credit for being the powerful tool that it is.

Meta State Machine Polygons of the Boost Book

I have mentioned the book Introduction to The Boost C++ Libraries by Boris Schäling here in the past.  The book is now available in three languages Chinese from the work of Albert Lai and his team of translators, in English and the original in German.


While on the subject of the Boost C++ Libraries, I want to mention that the new release, version 1.44, has added two new libraries:

 
• Meta State Machine: High-performance expressive UML2 finite state machines, from Christophe Henry.
 
• Polygon: Booleans/clipping, resizing/offsetting and more for planar polygons with integral coordinates, from Lucanus Simonson.

The MSM/UML library looks interest for doing some embedded code, while the Polygon library might be the start of a new circuit board layout program.

Requirements Creep

In my last blog entry I mentioned how stressful things can get when we are not given good, or any, requirements for a project, beyond the ship date.  Right after I posted that, I happened to see a message on the IEEE P730 Working Group on Software Quality Assurance Standards email list, by Robin Goldsmith of Go Pro Management, that I thought was worth posting here.  With Robin's permission here is what he wrote:

       "A major cause of creep, quality problems, and overruns comes from the common failure to recognize that what most people call "requirements" (i.e., specifications) actually are a form of high-level design.  The word "specification" is  a synonym for design.  Requirements specifications are human-defined attributes of a product, system, or software which is a presumed way how to satisfy the REAL, business requirements deliverable whats that provide value when met.  The product and its specifications provide value if and only if they satisfy the REAL, business requirements.


       Creep mainly occurs when the product/specifications fail to satisfy the REAL, business requirements, usually because the REAL business requirements have not been defined adequately, which in turn is mainly due to people thinking the product specifications are the requirements.


       The difference often is easier to see in an acquisition situation.  Most formal and informal Requests for Proposals (RFPs) specify the requirements of a product, system, or software the buyer thinks they need.  When the seller sells them such a product, the buyer--not the seller--is on the hook if the product fails to provide expected value.  In contrast, appropriate acquisition involves the seller proposing a product, system, or software to satisfy the buyer's described REAL business requirements.  Then the legal burden is on the seller if the proposed product fails to provide expected value.


       In addition to putting the burden of legal responsibility on the seller, there's a software engineering advantage of requesting the seller to propose a product that meets the buyer's REAL business requirements.  This gives the buyer the opportunity to take advantage of the seller's usually far greater knowledge of the subject area their product addresses, rather than having the less-informed buyer specify the product requirements.


       Well-formed formal and informal contracts and acquirer-supplier agreements  more reliably lead to success because they identify both high-level and detailed REAL business requirements, which usually include various constraints on how those requirements may be met.


       In effective product acquisitions, the seller does include a requirements specification in their proposal; and when the proposal is accepted by the buyer, the seller's entire proposal including any requirements specifications becomes part of the contract the seller is legally bound to deliver--by providing a product that conforms to the contracted requirements specifications.


       Of course, often the seller is engaged to develop a requirements specification as part of the contracted work.  Effective acquisitions then involve a separate subsequent proposal for providing a product that conforms to the requirements specification.  Especially for government acquisitions, the seller producing the requirements specification may be precluded from also proposing a product to meet the specifications they've defined."

Robin is the author of the book, Discovering REAL Business Requirements for Software Project Success; ISBN: 978-1580537704, which I've just added to my reading list.





It requires diligence from all of us to keep the Creeping Feature Creature at bay...

Stressed Out? Lets put up nets to catch the jumpers!

Are you stressed at the end of each month?  Is the owner of the company in the QC department moving things around saying "work on this, this and this, screw what you told the customer about delivery" because those items with higher monetary values will make the Been Counters happy?  Have you had the misfortune of working for a boss that told a new colleague "We don't write anything down here, like product requirements or specifications, because that reduces flexibility".  Talk about stress, nothing you ever supply the boss can possible be correct, because only he knows what the requirement was, so everything is always late, and had to be redone, sometimes several times a day.  No, that boss was not worried about Federal Rule of Civil Procedure Rule 26; Duty to Disclose; General Provisions Governing Discovery.


Maybe your wife is reading an online copy of Mr. Mean: Saving Your Relationship from the Irritable Male Syndrome by Jed Diamond Ph.D (ISBN: 978-0984260010) right now?  Due to being so stressed out from working daily with the boss that does not understand the need for requirements.




There are two items brought all of this up, to make we want to write this post.  First is this news story I just read: Foxconn Technology Group raised workers' wages and installed safety nets on buildings to catch would-be jumpers.  One thing we can say for sure is that Foxconn management is following the American Corporation way of fixing the symptom instead of fixing the root cause.


The second reason is that now even Intel is studying stress:

"Intel Scoop will feature social scientists such as anthropologists and psychologists from Intel on a thoughtful discussion about the relationship between technology and stress – how technology can elevate but also alleviate stress at the same time. Follow and join the discussion here."

Not just their tongue-in-cheek Do You Suffer From Hourglass Syndrome?, but a sincere study of psychological stress by psychologist and anthropologist.



Something that many in the world of Embedded Systems, and companies in general, often lose sight of is that our products are, in one way or the other, meant to be used by people, either directly like a Cell Phone or indirectly like a sewage treatment control system.  Losing sight of people leads to stress, or to make a case from the bottom line, stress caused by not having customers to buy our Embedded widgets (no customers eventual means no job, which is about as stressful as it can get).


Sometimes it can be good to read something seemingly completely outside our obscure Embedded area like Pizza Today Magazine or more relevantly publications from The Society for Consumer Psychology.  The SCP publishes the Journal of Consumer Psychology (JCP).  JCP publishes articles that contribute both theoretically and empirically to an understanding of consumer judgment and behavior and the processes that underlie them.  If you want to influence people to buy your products, then it is important to understand what forces control their judgment. For example:





The effect of deal exclusivity on consumer response to targeted price promotions: A social identification perspective; Journal of Consumer Psychology Volume 20, Issue 1, January 2010, Pages 78-89:

Abstract:

Discounts offered selectively to consumers are commonplace in the market and reflect the assumption that individuals will respond positively to targeted discounts. We consider whether exclusive deals evoke more positive responses than inclusive offers, an outcome referred to as a deal exclusivity effect. Contrary to the intuition that targeted promotions will always be evaluated more favorably than inclusive offers, we show that deal exclusivity effects (1) can be attenuated based upon factors influencing the extent to which recipients identify with other deal recipients and (2) are mediated by the offer's ability to enable the recipient to engage in self-enhancement.

Michael J. Barone, Tirthankar Roy, The effect of deal exclusivity on consumer response to targeted price promotions: A social identification perspective, Journal of Consumer Psychology, Volume 20, Issue 1, January 2010, Pages 78-89, ISSN 1057-7408, DOI: 10.1016/j.jcps.2009.10.002.  (http://www.sciencedirect.com/science/article/B8JGB-4XMTJWX-1/2/93e73522f597037ff3cbef9995299f8b)

What is even more interesting is the timing of a customers interest:



The effect of consumers' diurnal preferences on temporal behavior;Journal of Consumer Psychology Volume 20, Issue 1, January 2010, Pages 53-65:

Abstract:

"The purpose of this study is to determine whether customers' diurnal preferences, tested at different times of the day, affect their responses and behavior. Three studies explore whether synchrony between the peak circadian arousal period and the time of participant testing influenced participants' temporal perception and behavior. Overall, the results imply a robust synchrony and time-of-day effects on the dependent variables. The authors discuss the theoretical significance of their findings and the managerial implications for consumer research and practice."

Jacob Hornik, Chezy Ofir, Rinat Shaanan-satchi, The effect of consumers' diurnal preferences on temporal behavior, Journal of Consumer Psychology, Volume 20, Issue 1, January 2010, Pages 53-65, ISSN 1057-7408, DOI: 10.1016/j.jcps.2009.08.002. (http://www.sciencedirect.com/science/article/B8JGB-4X7FRVW-1/2/f7ba3a4b584e51c635eaa75bb635b372)

The good, the bad, and the ugly: Influence of aesthetics on product feature judgments by Joseph W. Albab and Darren W. Dahla, will probably be an interesting read as well when I can my hands on a copy.


In relation to stress, can't help but to mention the effects of Full Moons on people and electronics.  See the last chapter of the book "Full Moons" by Paul Katzeff; ISBN 0-8065-0843-9.





While the boss is changing his mind about what a product should do, yet again, take deep breaths, and learn Breathing: Three Exercises.  You may exhale now...

Buggy Toyota Software. Don't they have hills in Japan?

After years of experience with "American" cars, for reasons of reliability and hidden rust (Did they design places for the rust to hide on purpose?), and not wanting to buy from a company that stopped honoring its warranties, my wife and I went for a used low mileage Toyota Van.


After the years of hype about Toyota Reliability I keep running into software bugs.  The annoying kind I could fix if they supplied source code with their vans.


I'm not talking about their well know sudden acceleration issues, but more every day issues, that are clearly caused by software.


First of all if you put the windows down on the sliding doors, then the doors will not latch into an open position.  The manual says that this is a safety feature.  How is having to race a door to keep it from smashing your hand anytime you unload groceries or load up on spring water at the local spring (Neither being on a level surface) a safety feature?


To make maters worse, the amount that windows must be down before the doors do not latch, is different between the two sides of the van, and the grade that the van is parked upon impacts the latch point as well, so the only real choice is to always remember to put up the windows on beastly hot August days.


Then we have the headlights.  There is a very rigorous sequence of events that must be followed to get the headlights to turn themselves off automatically.  Deviate from that sequence in any way, and you end up with a dead battery.  No chime, or anything else, that your lights are on when you open the door (there is a chime, as it goes off for reasons yet unknown while driving down the road at times, usually related to something about the passenger air bag).


I simply do not comprehend why simple software issues like this even have to exist in our vehicles.


Slash Dot has a story on the New Jaguar XJ Suffers Blue Screen of Death as well.


Am I the only person left that wants my vehicle to be a tool to transport me and mine from point A to point B, and not be an Infotainment Center?


Not to leave sudden acceleration issues completely out of a Toyota related post, I noticed this in the 2008 Owner's Manual:

Installation of a mobile two-way radio system

As the installation of a mobile two-way radio system in your vehicle could affect electronic systems such as multiport fuel injection system/sequential multiport fuel injection system, electronic throttle control system, cruse control system, dynamic laser cruise control system, anti-lock brake system, traction control system, vehicle stability control system, SRS airbag system and seat belt pretensioner system, be sure to check with your Toyota dealer for precautionary measures or special instructions regarding installation.

So what do I do about the guy driving next to me with the two-way system, her cell phone in the passing car, the cell phone tower I drive by, or the transmitter from the traffic control system at the intersection?



If Toyota can not get simple things like headlights and door correct, what should we think about their ability to handle complex real time code?

"Nine ways to break your systems code using volatile"

Having just done a review of some code for a client, full of 'volatile' directives with no technical justification, I relized that I have been remiss in posting a link to John Regehr's Nine ways to break your systems code using volatile.  My apologies John.


I have covered John's seminal work in uncovering problems with volatile in the past here and here.  Volatiles Are Miscompiled, and What to Do about It by Eric Eide and John Regehr *must* be on your reading list if you are writing Embedded System code.


Bill Cox made a comment to my post that I completely agree with, and coding philosophy that I follow:

It requires a lot of discipline to minimize or eliminate global or non-static variables from your code, and it's worth it.

For an other tip related to 'volatile' take a read of Evaluating embedded code by Nigel Jones.

Hacker's Delight at Bit Twiddling Hacks

Michael Barr's Firmware Update newsletter for August 13th [2010] is just out.


Mike made a comment about his favorites bit twiddling methods, taken from Bit Twiddling Hacks by Sean Eron Anderson.  Worthy of any programmers time to read.


I wanted to add that there is a book written about the subject as well, Hacker's Delight by Henry S. Warren Jr; Addison-Wesley, 2003. ISBN: ISBN-13: 978-0201914658  Along with the books adjunct site.




The book covers the fast way to get many of the common firmware problems solved.  It is one of the books within easy reach of my development system most of the time.


It would be interesting to see if using the modern 'Typeof' found in newer compilers would let us do any of these bit twiddles even better.


Now for the down side.  If I was doing a Code Inspection and came across a magic number like 0x077CB531UL, with no explanation that this is a de Bruijn sequence, in the comments without lots of references, I would be appalled.


Bit Twiddling hacks are great in the few times that you need them, where you have to minimize branches, or maximize performance in a limited area, but always keep in mind the best programmers write simple code...

Parts Shortage, the dreaded "A" word: Allocation

There is an old story of when HP was little more than a garage, that Motorola bought one of their signal generators.  When the new generator arrived at Motorola, it did not work.  Being the best of geeks, the Motorola people took the unit apart, rather than getting on the phone as would happen most places today.  What the Motorola engineers found in some transistor sockets was a sheet of paper that read: "We shipped, you didn't".


DigiTimes is reporting that Passive component manufacturing equipment makers [are] extend[ing] delivery lead times, due to a shortage of parts.  One could presume the passive components the machines are indented to produce, are the parts that can not be gotten.  Que artist M. C. Escher classic Drawing Hands...


On the other hand we might also presume that the passive component manufactures can't get parts, because of the industry wide fab capacity shortage.  Not enough places in the world to build the ICs that we depend on for our products.


That some of the major fabricators set on large known earthquake faults doesn't give me the warm fuzzies either for a long term outlook.  Many of my designs are typically for industrial infrastructure, where life cycles are measured in decades, not the eighteen month cycle of Cell Phones.  I want parts that are going to around for a while, and a company that I can depend on them to deliver.


Some of the companies I work with are small, one, was until recently, only two people, others less than 50.  Many in heavily regulated industries.  To change anything in a product requires submitting paperwork to the Government, at costs into the thousands of dollars.  When a MegaCorp decides that your usage of only tens-of-thousands of their parts per year, is to little to be bothered with, is not something I can adequately put into words here on a family oriented blog.  How do I get MegaCorp to paying for these Paper Pusher type changes that part shortages bring about?


What brings my rant to light here today, is that on Friday a purchasing agent told me that the large component distributor Arrow Electronics Inc., reported that Atmel gave all of Arrow's allocation (Millions?) of DataFlash memories to a "special large customer".  Arrow's people are livid, and you probably are now too, if you were waiting for such parts.


I'll speculate that this has to do with Atmel Enters into Definitive Agreement with INSIDE Contactless for Sale of SMS Business and Atmel Completes Sale of Wafer Manufacturing Operation in Rousset, France to LFoundry GmbH.  'You buy our memory card business, and we'll see that you get the memories that you require'; at everyone else's expense.


Going forward before I design in a part I'm going to find out if the company has their own fabs.  Doesn't mean that some Bean Counter won't sell it in the future, but you can only do what you can do...


Have you been impacted by the current allocation crises, yet?