Probabilistic Programming

The Defense Advanced Research Projects Agency's (DARPA) budget proposals are always an interesting read. They lead to things like this, potentially Open Source Project: Probabilistic Programming for Advancing Machine Learning (PPAML); Solicitation Number: DARPA-BAA-13-31

Machine learning is at the heart of modern approaches to artificial intelligence. The field posits that teaching computers how to learn can be significantly more effective than programming them explicitly. This idea has revolutionized what computers can do in a wide range of domains, including Intelligence, Surveillance, and Reconnaissance (ISR), Natural Language Processing (NLP), Predictive Analytics, Cyber, and various scientific disciplines. Example applications include self-driving cars, image search and activity detection, object tracking, topic models, spam filters, recommender systems, predictive databases, and gene sequencing. Unfortunately, building effective machine learning applications currently requires Herculean efforts on the part of highly trained experts in machine learning. Probabilistic Programming is a new programming paradigm for managing uncertain information. The goal of the Probabilistic Programming for Advancing Machine Learning (PPAML) program is to facilitate the construction of machine learning applications by using probabilistic programming to: (1) dramatically increase the number of people who can successfully build machine learning applications; (2) make machine learning experts radically more effective; and (3) enable new applications that are inconceivable today.

Machine learning applications work by building a model of a phenomenon of interest and then training or conditioning that model with observed data, similar to the way we may 'teach' a Neural Network today. What Probabilistic Programming is not is the classic Inference Engine, rather it is a front end to classic IEs or yet unthought of IEs. BUGS would be an example of a classic IE. The Probabilistic Programming approach separates the model from the solver, making it possible for one set of users to develop the model without having to implement or know the details of the solver. With Probabilistic Programming languages, once they exist, developers will be able to focus on developing their models while solver experts will be able to embed their expertise in reusable new style inference engines.

Where I see Probabilistic Programming being of the most use in Embedded Space is in the area of network routing for the Internet of Things (leaving aside the issues of lack of conventional radio spectrum and the raising Internet noise floor slowing everything down for the moment).

There are some Software Safety issues to address, such as how is a Probabilistic Programming verified and validated? No human may actually know how it works, to explain how a particular conclusion was reached. MISRA well known for their C and C++ safety Guidelines does have an obscure section on Autocode generation. However that is still based on conventional technology of today. What happens when true Artificial Intelligence systems do start creating programs that we rely on?

To keep up to date follow the Probabilistic-Programming.org Mailing List.