Learning From Less Data

The old adage that practice makes perfect applies to machines as well, as many of today’s artificially intelligent devices rely on repetition to learn. Deep-learning algorithms are designed to allow AI devices to glean knowledge from datasets and then apply what they’ve learned to concrete situations. For example, an AI system is fed data about how the sky is usually blue, which allows it to later recognize the sky in a series of images.

Complex work can be accomplished using this method, but it certainly leaves something to be desired. For instance, could the same results be obtained by exposing deep-learning AI to fewer examples? Boston-based startup Gamalon developed a new technology to try to answer just that, and this week, it released two products that utilize its new approach.

Gamalon calls the technique it employed Bayesian program synthesis. It is based on a mathematical framework named after 18^th century mathematician Thomas Bayes. The Bayesian probability is used to refine predictions about the world using experience. This form of probabilistic programming — a code that uses probabilities instead of specific variables — requires fewer examples to make a determination, such as, for example, that the sky is blue with patches of white clouds. The program also refines its knowledge as further examples are provided, and its code can be rewritten to tweak the probabilities.

Probabilistic Programming

While this new approach to programming still has difficult challenges to overcome, it has significant potential to automate the development of machine-learning algorithms. “Probabilistic programming will make machine learning much easier for researchers and practitioners,” explained Brendan Lake, an NYU research fellow who worked on a probabilistic programming technique in 2015. “It has the potential to take care of the difficult [programming] parts automatically.”

Gamalon CEO and cofounder Ben Vigoda showed MIT Technology Review a demo drawing app that uses their new method. The app is similar to one released by Google last year in that it predicts what a person is trying to sketch. However, unlike Google’s version, which relied on sketches it had previously seen to make predictions, Gamalon’s app relies on probabilistic programming to identify an object’s key features. Therefore, even if you draw a figure that’s different from what the app has previously seen, as long as it recognizes certain features — like how a square with a triangle on top is probably a house — it will make a correct prediction.

The two products Gamalon released show that this technique could have near-term commercial use. One product, the Gamalon Structure, using Bayesian program synthesis to recognize concepts from raw text, and it does so more efficiently than what’s normally possible. For example, after only receiving a manufacturer’s description of a television, it can determine its brand, product name, screen resolution, size, and other features. Another app, called Gamalon Match, categorizes products and prices in a store’s inventory. In both cases, the system can be trained quickly to recognize variations in acronyms or abbreviations.

Vigoda believes there are other possible applications, as well. For example, if equipped with a Beysian model of machine learning, smartphones or laptops wouldn’t need to share personal data with large companies to determine user interests; the calculations could be done effectively within the device. Autonomous cars could also learn to adapt to their environment much faster using this method of learning. The potential impact of smarter machines really can’t be overstated.

Source