Probabilistic Programming

The Defense Advanced Research Projects Agency's (DARPA) budget proposals are always an interesting read. They lead to things like this, potentially Open Source Project: Probabilistic Programming for Advancing Machine Learning (PPAML); Solicitation Number: DARPA-BAA-13-31

Machine learning is at the heart of modern approaches to artificial intelligence. The field posits that teaching computers how to learn can be significantly more effective than programming them explicitly. This idea has revolutionized what computers can do in a wide range of domains, including Intelligence, Surveillance, and Reconnaissance (ISR), Natural Language Processing (NLP), Predictive Analytics, Cyber, and various scientific disciplines. Example applications include self-driving cars, image search and activity detection, object tracking, topic models, spam filters, recommender systems, predictive databases, and gene sequencing. Unfortunately, building effective machine learning applications currently requires Herculean efforts on the part of highly trained experts in machine learning. Probabilistic Programming is a new programming paradigm for managing uncertain information. The goal of the Probabilistic Programming for Advancing Machine Learning (PPAML) program is to facilitate the construction of machine learning applications by using probabilistic programming to: (1) dramatically increase the number of people who can successfully build machine learning applications; (2) make machine learning experts radically more effective; and (3) enable new applications that are inconceivable today.

Machine learning applications work by building a model of a phenomenon of interest and then training or conditioning that model with observed data, similar to the way we may 'teach' a Neural Network today. What Probabilistic Programming is not is the classic Inference Engine, rather it is a front end to classic IEs or yet unthought of IEs. BUGS would be an example of a classic IE. The Probabilistic Programming approach separates the model from the solver, making it possible for one set of users to develop the model without having to implement or know the details of the solver. With Probabilistic Programming languages, once they exist, developers will be able to focus on developing their models while solver experts will be able to embed their expertise in reusable new style inference engines.

Where I see Probabilistic Programming being of the most use in Embedded Space is in the area of network routing for the Internet of Things (leaving aside the issues of lack of conventional radio spectrum and the raising Internet noise floor slowing everything down for the moment).

There are some Software Safety issues to address, such as how is a Probabilistic Programming verified and validated? No human may actually know how it works, to explain how a particular conclusion was reached. MISRA well known for their C and C++ safety Guidelines does have an obscure section on Autocode generation. However that is still based on conventional technology of today. What happens when true Artificial Intelligence systems do start creating programs that we rely on?

To keep up to date follow the Probabilistic-Programming.org Mailing List.

MISRA tutorial and seminar announcement. Standards or Guidelines?

If you are a world traveler, The Motor Industry Software Reliability Association (MISRA) just announced a tutorial and seminar:

"A tutorial and seminar in collaboration with the Safety-Critical Systems Club (SCSC) will be held on 24 and 25 November 2010 in London, UK. On the first day there will be two half-day tutorials, on the MISRA C and MISRA C++ guidelines for the use of these languages in critical systems. The second day will be a seminar that will consider how the MISRA guidelines support safety-related systems development in various industry sectors, and we are particularly keen to feature presentations of case studies. If you would like to speak at this seminar, please discuss your idea with me in the first instance.

Further details of the programme will be available in due course.

There will also be a small exhibition on the second day. Details of the exhibition are available from Joan Atkinson at the SCSC

____________________
Dr David Ward
MISRA Project Manager
Article:

http://www.misra.org.uk/forum/viewtopic.php?f=210&t=1058&p=2032#p2032"

For those not familiar with MISRA, MISRA publishes a set of 'Guidelines' on 'Best Practices' for the languages C and C++. For example what languages constructs are bug prone and should be avoided in embedded systems used in vehicles. Many companies outside of the automotive industry have adopted the use of these guidelines, to help keep bugs out of their products.

Netrino also has a similar set of guidelines, with some links to some real world tips: Embedded C Coding Standard.

A technical semantic nit I want to pick is that these are guidelines. Standards are set by recognized bodies like IEEE and IEC - International Electrotechnical Commission. I frequently see things like "MISRA Standard". The distinction is a subtle one, but could be important to a product liability suite. Which is not to say that a company can not set 'internal standards' for coding practices. English is such a wonderful language... :-(

IEEE P730 Standards Software Quality Assurance, IEEE/EIA 12207.0 Standard for Information Technology - Software Life Cycle Processes, and IEC 61508 Functional safety of electrical/electronic/programmable electronic safety-related systems, are all example of standards used in Embedded Systems. There are more examples at my Software Safety site.

I'm Scared

This week I spent a couple of days at regional seminars. At the Texas Instruments Technology Days 2010 in Cleveland, the thing I found most interesting is TI's new Analog Mirror. A single, large, pixel of their already popular DLP technology. Must be something cool we can do with this mirror? Dynamic Signs maybe?

The other seminar this week was sponsored by Atmel, to drum up design wins for their AVR32 family of parts, here in the Pittsburgh market region.

Most of the seminar was showing how to use AVR32 Studio that is based on Eclipse. Alas I probably won't use these parts because the seven year old machine supplied by the IT department would never handle such a large application. Just a fresh reboot of the machine already has 230M+ of the 512Meg of memory used with cooperate bloatware mandated by IT (Makes their jobs easier).

What is relevant to us here today is a comment that the instructor made, paraphrased:

We had to switch to Eclipse because all of the people coming to the Embedded World from the Microsoft World did not know how to write their programs without such a tool.

I find that scary. We'll end up with large, and potentially unsafe, expensive products, because Management assumes any programmer that can write a business application or web app. can design a safe embedded system if they only have the right tool.

Something else I find scary is the article Think it - Draw it - Build it by Mark Saunders, at Embedded.com.

Mark introduces us to the new Cypress Semiconductor PSoC Creator embedded design tool, for Cypress's new PSoC 3 and PSoC 5 programmable system-on-chip architectures. The PSoC 5 parts are something I want to investigate for use in a board test system I'm designing. You don't know what the board under test may need in a test system so a configurable system is the way to go.

However I find the marketing of this product family to be scary:

...You will not need to know the CPU architecture we're using, or how the analog comparator or digital timer components are implemented...

...[PSoC Creator] abstracts away the hardware so you do not need to be an expert on the device you are using or the inner workings of peripherals you program it with...

For the comparator is any overdrive required, how much? Is there any hysteresis to prevent oscillation? What happens when the counter rolls over, does it do The Right Thing? Hopefully some quality time spent with a quality data sheet (sadly far to many data sheets lack any quality) would answers these questions. Would the people using Eclipse from the Microsoft world even know to ask such questions?

Now are you starting to see why I'm scared? I fear that we are heading down the path of Think it - Draw it - Build it - Ship it - So we get paid for it - Doesn't mater if it works right.

How scared are you about the quality of our future Embedded Systems?