Matterport, the spatial data company, has reportedly teamed up with Silicon Valley giant, Facebook, to provide thousands of virtual, interactive digital duplicates of real-world places to researchers to help them train their young AIs.
To enable this, Facebook, on its side has made two advancements; the new Facebook Habitat 2.0 training platform as well as the dataset the company has designed to support it. Facebook had developed Habitat a few years ago in its pursuit of something it calls, ‘embodied AI’, essentially AI models that interact with the actual world. Facebook had assembled numerous passably photorealistic virtual spaces for the AI to navigate.
Numerous robots and AIs have learned mobility and object identification in idealized, artificial environments that are more akin to video games than reality. A real-life living room is not the same as one that has been rebuilt. An AI's expertise will more easily translate to real-world applications such as home robotics if it learns to move around in something that looks like reality.
However, these settings were just polygon-deep, with no interactivity and no true physical modeling. For example, if a robot runs into a table, it does not topple over and drop goods all over the place. The AI-based robot could go into the kitchen, but could not open the refrigerator or fetch something from the sink. With enhanced interaction and 3D objects instead of just interpreted 3D surfaces, Habitat 2.0 and the latest ReplicaCAD dataset change these training environments for the better.
Meanwhile, in partnership with Facebook, Matterport is also making several other major steps. After developing an expansive platform through the last few years, the firm has amassed a massive collection of 3D-scanned structures. While it has previously collaborated with other researchers, the firm decided that it was time to make a bigger portion of its collection available to the overall community.
RJ Pittman, CEO of Matterport, stated that every form of physical structure in existence, or near to it, has been matterported by the firm, ranging from high-rises, houses, offices, and hospitals, to airplanes, cruise ships, McDonald’s, and Taco Bells. All of the scanned data, included in its own digital twin, is crucial for study.
Under the collaboration, both the datasets would be made open and available to researchers everywhere for their work.
Source credit: https://techcrunch.com/2021/06/30/facebook-and-matterport-collaborate-on-realistic-virtual-training-environments-for-ai/