Judge Finds Ford Fraudulently Concealed Electronic Causes of Unintended Acceleration

This week, July 27 2011, Safety Research & Strategies, Inc published two documents taken from the court of Senior Judge William T. Swigert of the Fifth Judicial Circuit in Sumter County, Florida.

This 51-page decision, by Judge Swigert excoriated Ford for systematically concealing a long history, stretching back to the 1970s, of studying the problem of electromagnetic interference and unintended acceleration.

SRS has an on going page devoted specifically to new Toyota Sudden Unintended Acceleration studies. It is interesting to note that in 2004 Toyota licensed some of its technology to Ford.

Last week July 21,2011, SRS published How NHTSA and NASA Gamed the Toyota Data, covering issues of Tin Whiskers (see my past blogs Soldering Defect Database: How many ways can a solder joint fail? More than you might think and The Anatomy of a Race Condition: Toyota vs AVR XMega), 'losing' the data the NASA report was based on, and other dubious practices. What NHTSA's Data Can Tell Us about Unintended Acceleration and Electronic Throttle Control Systems. by R. A. Whitfield of Quality Control Systems Corporation gives a brief analysis of the raw National Highway Traffic Safety Administration (NHTSA) data.

In my past blog entry The Anatomy of a Race Condition: Toyota vs AVR XMega on NASA's on Toyota's sudden acceleration problem, I went into some of the details as to what I believe is happening. I explain why the Sudden Acceleration issue would not issue a diagnostic code if the problem is based in the firmware of the engine control unit, which is something I never seen covered in any of these reports. Guess they are all done by people with no experience in designing the systems they are reporting on?

NASA to outsources Validation and Verification. Do you want the job?

Sometimes you run into things that just want you to shake your head and wonder, what is going on here? Case in point the National Highway Traffic Safety Administration outsourced the study of Toyota's Sudden Acceleration problem to NASA because they where perceived to be the best around to study this problem. Now this week we find that NASA wants to outsource their Validation and Verification work. Good enough for cars but not space shots??

Before getting to the NASA solicitation I want to define what Validation and Verification mean. Like many people confuse Weather Watches and Warnings many people are unclear on the difference between Validation and Verification.

• Weather watch is used when the risk of an event has increased, but its occurrence and timing is still uncertain.
• Weather warning is used for conditions posing a threat to life or propert.

Definition of Terms:

The FDA's Glossary of Computerized System and Software Development Terminology, defines many of the terms used on in this blog and our Software Safety site.

Defect:  The difference between the expectation and the actual results.

Validation and Verification:

Validation and Verification are a set of terms you find when working with Software Safety.  Many people do not understand how they differ from each other.

These are my working definitions for Validation and Verification (V&V):

• Validation: Have we built the correct device?  Do we meet the customer's requirements?
• Verification: Have we built the device correctly? Did we find and remove all of the 'bugs'?

Requirements and Specifications:

Clarifying the distinction between the terms "requirement" and "specification" is important.

My working definitions for Requirements and Specifications:

Requirements are a statement of what the customer wants and needs.  Requirements are used for validation.  Specifications are the documentation of how the customer requirements are met by the system design.  Specifications are used for verification.

A requirement can be any need or expectation for a system or for its software. Requirements reflect the stated or implied needs of the customer, and may be market-based, contractual, or statutory, as well as an organization's internal requirements. There can be many different kinds of requirements (e.g., design, functional, implementation, interface, performance, or physical requirements). Software requirements are typically derived from the system requirements for those aspects of system functionality that have been allocated to software. Software requirements are typically stated in functional terms and are defined, refined, and updated as a development project progresses. Success in accurately and completely documenting software requirements is a crucial factor in successful validation of the resulting software, and the project as a whole.

A specification  "means any requirement with which a product, process, service, or other activity must conform." (See 21 CFR§820.3(y).) It may refer to or include drawings, patterns, or other relevant documents and usually indicates the means and the criteria whereby conformity with the requirement can be checked. There are many different kinds of written specifications, e.g., system requirements specification, software requirements specification, software design specification, software test specification, software integration specification, etc. All of these documents establish "specified requirements" and are design outputs for which various forms of verification are necessary.

Device failure (21 CFR§821.3(d)). A device failure is the failure of a device to perform or function as intended, including any deviations from the device’s performance specifications or intended use.

Now with that background under our belt we can get a better grasp of what NASA is seeking in their, Proposals For Software Verification And Validation. This contract will provide resources for NASA-directed software verification and validation services; software safety assurance support for agency missions; and potential software development work for other government agencies.

NASA INDEPENDENT VERIFICATION AND VALIDATION SERVICES
Solicitation Number: NNG11310421R
Agency: National Aeronautics and Space Administration
Office: Headquarters
Location: Office of Procurement (HQ)

Synopsis:

Added: Sep 21, 2010 3:02 pm Modified: Mar 02, 2011 5:35 pm

NASA/Goddard Space Flight Center announces the release of the final RFP for NASA Independent Verification and Validation Services. Proposals submitted in response to this RFP shall be submitted by April 5, 2011, at 3:00pm Eastern Standard Time.The due date for questions or comments is March 24, 2011 to ensure our timely response. All questions and comments must be submitted in writing via email to the following email addresses: Laura.E.Freeman[-a-t-despaming]nasa.gov. Telephone questions will not be accepted. Technical documents related to this procurement can be obtained from the Procurement Proposal Library at: http://www.nasa.gov/centers/ivv/recompete/index.html. Documents related to this procurement will be available over the Internet. These documents will reside on a World Wide Web (WWW) server, which may be accessed using a WWW browser application. The Internet site, or URL, for the NASA/HQ Business Opportunities home page is http://prod.nais.nasa.gov/cgi-bin/eps/bizops.cgi?gr=D&pin=04 Offerors are responsible for monitoring this site for the release of the solicitation and any amendments. Potential offerors are responsible for downloading their own copy of the solicitation and amendments (if any).

Here is the link to NASA's V&V Facility, and I'll repeat the link I gave last week to NASA's Software Safety Guidebook. NASA has other standards and programs that are worth studying such as their Standards and Technical Assistance Resource Tool, for example the Software Formal Inspections Standards and Langley's Formal Methods.

So do you think you and I should team up and add ourselves to the Interested Vendors List for this V&V opportunity, or maybe one of the other 23,000+ opportunities?

Safe and Secure Systems, Software Symposium - S5; Software survivability: where safety and security converge

This week (June 15th) wrapped up the 2010 Safe & Secure Systems & Software Symposium - S5 sponsored by the Air Force Research Laboratory Control Systems Development and Applications Branch (AFRL/RBCC), located at Wright-Patterson Air Force Base, purported home of Hangar 18.

The Air Force Research Laboratory Control performs basic research, exploratory development, and advanced development of flight control system technologies for highly reliable, fault tolerant vehicle stabilization, and flight path control. Develops concepts, components, criteria, analytical methods, and design tools for flight vehicle applications. Emphasizes development of fault tolerant control system architectures control automations, flight critical hardware and software, such as actuators, sensors and computational elements; and on-board and networked vehicle management systems. Integrates flight control with airframe, avionics, propulsion, utilities, mission, and vehicle flight path management. Conducts full spectrum of development and application activities ranging from initial concept definition through system mechanization, laboratory evaluation, and flight validation.

The AFRL/S5 event brought together industry, academia and government to collaborate on the common goal of improving the airworthiness and assurance certification process of future aerospace flight control systems with both incremental and revolutionary technological innovations in safety and security verification and validation (V&V) techniques that support maintaining cost and risk at acceptable levels.

The executive summary list for the S5 event includes:

• Improving V&V for flight critical/safety and mission/information security
• Designing for Airworthiness Certification:
• Software for Complex Systems
• Fundamentals for the Future

A good paper to start with, to get the flavor of the event, and to see the relevance to Embedded Systems, is: Software survivability: where safety and security converge by Karen Mercedes Goertzel and Booz Allen Hamilton.

The S5 agenda contains links to all of the presented papers (those are links at that site, but it is not obvious in all browsers).

The Goertzel/Hamilton paper puts particular emphasizes that few of us give much consideration, which is something that we all need to change, that is Threats and Attacks, on our systems. Threats and Attacks "Require human intention and intelligence in planning and execution", which stand in contrast to the normal Hazards that we do consider; the stuff of life that just happens, like lightning strikes, failed components, lead free tin-wiskers.

Today's Adversaries are:

• Knowledgeable: They know more about our software than we do, including its vulnerabilities.
• Skill and sophisticated: Not just "script kiddies". Attackers know how to exploit vulnerabilities, how to augment/assist direct attacks with social engineering and surreptitious malware (worms, Trojans, bots, spyware).
• Quick: "Zero day" is the rule, with new attacks appearing before vulnerabilities are discovered by developers, let alone patched.
• Motivated and well-resourced: Not just recreational hackers, but organized criminals, nation-state Info Warriors, Cyber Terrorists.

Our adversaries are not motivated by the Next Quarters Bottom Line, unlike our shorted sited corporate leaders. They are motivated by greed (hmm...that makes it sound like our corporate leaders are our adversaries doesn't it?) or the desire to do us harm. The Bad Guys usually have better resources and a single task to accomplish, unlike most Embedded System development groups, and deadlines that are not motivated by trade show schedules. The Bad Guys take advantage of the mantra of "There is never time to do it right now, but there will always be time to do it over someday in the future", that far to many organizations use as their design standard.

Goertzel/Hamilton mention (see Naval Sea Systems Command (NAVSEA)/Naval Ordnance Safety and Support Activity (NOSSA)) one of my pet-peeves "unnecessary functionality" as one of the "Hazards and risks that arise in software". The Creeping Feature Creature can be a powerful taskmaster, that leads to increased time to market and development costs that impact the bottom line, for features that will never be used by a customer, but seem "really cool" to management and developers.

Goertzel/Hamilton do give some recommendations on how to improve Software Safety and Security. Alas not of them seem practical in the corporate world due to time, budget, and size constraints.

Spend some time reading all of the other papers, to see where Safety Critical System development is headed.

Are Embedded System Engineers more adulterous than other Engineers? Webinar March 26 Soft Skills

You are undoubtedly asking why I would even ask such a question as "Are Embedded System Engineers more Adulterous than other Engineers?" right?
On the drive into the office the Talk Radio DJ brought up a survey done by the controversial dating website AshleyMadison.com. Apparently this dating site is intended for married people. That is just wrong on so many different levels.
What caught my attention was that "Engineer" was listed in the survey of people most likely to have an affair. Out in the vast wasteland of Government Pork there probably really is someone willing to fund a study to answer the question we posed above.
Normally when I hear such things on radio or Internet I do my best to find the original source of the information. In this case we know that the source is Ashley Madison. However to actually see the survey you must register with the site. "Honey, I only signed up to that site to do research for my software safety blog". No I don't think I'll go do that road, and instead point you to a secondary source: Who Cheats? Docs and Stay at Home Moms!; Ever wonder what professions top the list as the most guilty of infidelity? We were shocked by the answer!.
Some of the comments to the article are interesting in themselves:

"Yes, an engineer will pursue an affair so that the wife and the mistress will think he is with the other woman so he could sneak to the office to catch up on work."

"The president and founder of AshleyMadison.com Noel Biderman notes these top professions are often high stress and require many to work long hours."

Alas in this industry we do seem to work hours that are far to long, in high stress environments (If our products screw up people may die, how higher stress job can you get?), where Dilbert seems to be a documentary rather than a cartoon.
If we are not carefully we can lose the focus that our products are ultimately designed to be used, either directly or indirectly, by people. So being able to interact with people is important skill to have. Having a well developed set of Soft Skills helps to understand things like having to hold a gear shift lever in a position for three seconds makes sense in the cold sea of design cubicles, but makes no sense in the real world.
Just as a race car driver does not get into the race car to drive, he gets into the race car because he is driven. People in our field tend to be driven to the field, usually at an young age. We certainly did not get into it for the fame or the money.
In my personal experience the best designers that I have worked with over the years have shared one or more of these traits:

• Apophenia; The experience of seeing patterns or connections in random or meaningless data.
• Dsylexia; Interchange letters and numbers, leading to problems with spelling and mathematics.
• Pareidolia; Finding of images or sounds in random stimuli.
• Tend to be "Hands On" Tactile Learners.

To the last point our schooling systems tend to develop Auditory and Visual Learners well, while failing miserably with Tactile Learners. Try taking the Learning Style Inventory yourself, and have your children take it.
Also, at least in our early years, as a group we tend to fit the stereotypical image of Nerd: "A person who passionately pursues intellectual activities, esoteric knowledge, or other obscure interests that are age-inappropriate rather than engaging in more social or popular activities. Therefore, a nerd is often excluded from physical activity and considered a loner by peers, or will tend to associate with like-minded people".
Speaking strictly for myself the movie Revenge of the Nerds was a documentary of my public school experience, and made be become an advocate for Home Schooling, and self-teaching.
We may be driven to this field by the seeming cold hard logic of it, and our fears of social interactions. Alas this is counter productive to designing products for use by people.
In the Soft Skill development area are couple books my wife has added to my pile of reading material:

• People Skills by Robert Bolton.
• Why Can't You Read My Mind? by Jeffrey Bernstein and Susan Magee. [Susan can help you get that book you've always dreamed of writing done.]

While I've added these to her pile:

• ESP Induction Through forms of Self-Hypnosis [How to improve your ESP] by Richard Alan Miller.
• Controlling the Human Mind : The Technologies of Political Control or Tools for Peak Performance by Dr. Nick Begich.[Full Disclosure: Dr. Begich sent me a gratis review copy.].

both published by Earth Pulse Press.
Do not be so quick to dismiss those titles last two titles in our discussion of Soft Skills. Having a well developed intuition is a good skill to have while debugging, and when trying to divine missing requirements, from incomplete information we are usually given to design products. The traits of Apophenia, Dsylexia, Pareidolia usually seen as a problem in general society can also be a significant strength in troubleshooting.
Also as a bit is aside, the area of "Anomalous Human Potential" is one of those new areas of Embedded System design that is virtual unknown outside of deep military circles. If you want to see how deep the rabbit hole goes, start here: The Mind Has No Firewall by Timothy L. Thomas. From Parameters, Spring 1998, pp. 84-92, US Army War College.
What do you recomended for developing Soft Skills? Perhaps a good Webinar (Ironic isn't it)?
The Pittsburgh Section of the American Society for Quality has started a series of Lunchtime Webinars.
Fitting in with our theme of Soft Skills is the one coming up March 26, 2010, 12pm-1pm (Eastern Standard Time), Getting Things Done as a Quality Professional presented By: Brien Palmer, Principal InterLINK Management Consulting.

Do you find yourself frustrated as a Quality professional? Do you find it difficult to get management hear your ideas? Then plan on spending lunch hour of March 26th in an immensely valuable webinar.
Most of us are very skilled in the analytical areas of logic, use of data, formal problem-solving processes, statistics, and so forth. We are trained in these areas, we use them constantly, and we are inclined by temperament to think of them as the only way of thinking. However, this way of thinking -this temperament- represents only a small part of the general population, and other people can consider us a bit odd.
In fact, we analytical types often lack what can be called "complimentary" skills, such as people skills, communication skills, assertion, presentation skills, political acumen, business knowledge, ability to managing change, etc. Because Quality professionals often lack these skills, we can get marginalized in organizations. Ideas get lost, management goes un-convinced, Quality takes a back seat, companies get in trouble, and so forth.
This cycle can be frustrating, but it doesn't have to be inevitable. In fact, analytical people are fast learners when they set their minds to it. When analytical people focus on non-analytic skills, they experience vast improvements in personal and organizational effectiveness.
This program will introduce Quality professionals to some immediately-applicable complimentary skills.
Cost: Participant pays $25 for voice-over-the-internet (VOIP) webinar presentation. Invite your company for Pizza Friday at no additional charge! Need speakers on computer to hear Presenter and speakerphone to ask a question to the Presenter, otherwise the chat function can be used.
To Register: Call 412-261-4300, upon payment participant will be registered for the webinar. If you have any problems with registering, please contact Robin Dudash.
Re-Certification Units (RU): Gain 0.1 RU under the Professional Development category.
Speaker Bio:

Brien Palmer is a management is a management consultant specializing in leadership development and quality management systems. In twenty-five years as a consultant, he has served a broad spectrum of clients, from large, internationally known companies such as Commonwealth Edison and Ontario Hydro to smaller, family-owned businesses. He has produced significant business results in a wide range of business settings, including high technology, manufacturing, public utilities, financial services, construction, and non-profit organizations.
Brien is a senior partner in InterLINK Management Consulting, a small, Pittsburgh-based consultancy (www.InterLINKbusiness.com).
Brien serves on the Board of Directors of the American Society of Quality (ASQ). He is an ASQ Certified Quality Auditor, and has written many articles for Quality Progress magazine. He is the author of one of a Quality Press' best-selling books: Making Change Work-Practical Tools for Overcoming Human Resistance to Change. He also created ASQ's first internet-based "virtual seminar", which is still in production: The Case for Quality: Taking it to Management.