THANKS TO A form of AI called deep learning, computers are now really good at telling the difference between a dog and a cat. But Facebook’s Artificial Intelligence Research (FAIR) lab wants to make machine vision far more useful, going well beyond digital parlor tricks.
FAIR research scientist Piotr Dollar says the first step lies in helping machines not just recognize that a particular thing appears in a photo—say, a cat or a chair or a gun—but spot each individual detail in a photo and understand where it sites in relation to everything else. His team has built a set of tools that does just that.
These tools could provide building blocks that Facebook needs to fashion more sophisticated machine vision systems. For example, the company eventually build software that could fully describe photos to blind users (it’s already part of the way there). Plus, it could use these tools in building augmented reality systems that display images in the real world with fine-grained precision.
But this isn’t just about Facebook. The company has these tools to the public—open sourced them—so that any developer can play with the code and use it in their own projects.