Connect with us

Tech

Teaching artificial intelligence to connect senses like vision and touch

In Canadian author Margaret Atwood’s book “Blind Assassins,” she says that “touch comes before sight, before speech. It’s the first language and the last, and it always tells the truth.”

Published

on

Teaching artificial intelligence to connect senses like vision and touch


In Canadian author Margaret Atwood’s book “Blind Assassins,” she says that “touch comes before sight, before speech. It’s the first language and the last, and it always tells the truth.”

While our sense of touch gives us a channel to feel the physical world, our eyes help us immediately understand the full picture of these tactile signals.

Robots that have been programmed to see or feel can’t use these signals quite as interchangeably. To better bridge this sensory gap, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have come up with a predictive artificial intelligence (AI) that can learn to see by touching, and learn to feel by seeing.

The team’s system can create realistic tactile signals from visual inputs, and predict which object and what part is being touched directly from those tactile inputs. They used a KUKA robot arm with a special tactile sensor called GelSight, designed by another group at MIT.

Using a simple web camera, the team recorded nearly 200 objects, such as tools, household products, fabrics, and more, being touched more than 12,000 times. Breaking those 12,000 video clips down into static frames, the team compiled “VisGel,” a dataset of more than 3 million visual/tactile-paired images.

“By looking at the scene, our model can imagine the feeling of touching a flat surface or a sharp edge”, says Yunzhu Li, CSAIL PhD student and lead author on a new paper about the system.

“By blindly touching around, our model can predict the interaction with the environment purely from tactile feelings. Bringing these two senses together could empower the robot and reduce the data we might need for tasks involving manipulating and grasping objects.”

Recent work to equip robots with more human-like physical senses, such as MIT’s 2016 project using deep learning to visually indicate sounds, or a model that predicts objects’ responses to physical forces, both use large datasets that aren’t available for understanding interactions between vision and touch.

The team’s technique gets around this by using the VisGel dataset, and something called generative adversarial networks (GANs).

GANs use visual or tactile images to generate images in the other modality. They work by using a “generator” and a “discriminator” that compete with each other, where the generator aims to create real-looking images to fool the discriminator. Every time the discriminator “catches” the generator, it has to expose the internal reasoning for the decision, which allows the generator to repeatedly improve itself.

Vision to touch

Humans can infer how an object feels just by seeing it. To better give machines this power, the system first had to locate the position of the touch, and then deduce information about the shape and feel of the region.

The reference images — without any robot-object interaction — helped the system encode details about the objects and the environment. Then, when the robot arm was operating, the model could simply compare the current frame with its reference image, and easily identify the location and scale of the touch.

This might look something like feeding the system an image of a computer mouse, and then “seeing” the area where the model predicts the object should be touched for pickup — which could vastly help machines plan safer and more efficient actions.

Touch to vision

For touch to vision, the aim was for the model to produce a visual image based on tactile data. The model analyzed a tactile image, and then figured out the shape and material of the contact position. It then looked back to the reference image to “hallucinate” the interaction.

For example, if during testing the model was fed tactile data on a shoe, it could produce an image of where that shoe was most likely to be touched.

This type of ability could be helpful for accomplishing tasks in cases where there’s no visual data, like when a light is off, or if a person is blindly reaching into a box or unknown area.

Click to comment

Leave a Reply

Business

This is what global tax reforms could mean for Asia’s tech giants

A new set of agreed global tax reforms will change where tech giants and other global giants pay taxes, explain experts from the IMF. Investment hubs such as Singapore and Hong Kong SAR could lose up to 0.15% of GDP as a result.

Published

on

7f7ede46 singapore skyline unsplash crop

Asia’s advanced and emerging market economies have several locally headquartered tech giants and host foreign companies.

(more…)
Continue Reading

Ecommerce

Pakorn Peetathawatchai, President, The Stock Exchange of Thailand (SET)

Published

on

Pakorn Peetathawatchai, President, The Stock Exchange of Thailand (SET)

What measures has SET taken to support listed companies’ compliance with ESG standards?
PAKORN PEETATHAWATCHAI:

PAKORN: When we first began promoting ESG-compliant investments, we were met with little interest. We attributed this to a lack of clear data to showcase the economic benefits of ESG investment, and perhaps limited clarity as to what constitutes a sustainable or ESG-compliant investment. The launch of the THSI list and, subsequently, the SETTHSI Index, was designed to address this. Our most recent data, comparing returns for the SETTHSI Index with the broader SET and SET100 indices from April 2020 to April 2021, underscores the economic benefits of these investments: the group compliant with ESG standards outperformed the other two indices on every data point. 

As of May 2021 Thailand was home to CG and ESG assets under management totalling BT54.8bn ($1.7bn) across 50 funds – up from 23 funds in 2019. Meanwhile, of the BT187.1bn ($5.9bn) raised in green, social and sustainability bonds since 2018, BT136.4bn ($4.3bn) was raised in 2020 – 83% from the government and the remainder from development banks and private players. This rising demand, in a move to manage risk and generate returns, has been complemented by growing supply and promotion: supply from ESG-compliant businesses aiming for resiliency and sustainable growth, as well as promotion from regulators highlighting investment opportunities with good CG and SD practices. Indeed, the pandemic has been a catalyst in shifting the view of ESG compliance from a luxury to a requirement in the new normal.

In what ways can enhanced standard-setting and regulatory mechanisms overcome the remaining barriers to improved ESG performance?

PAKORN: A multi-stakeholder approach is crucial for enhanced ESG performance – not only in Thailand, but around much of the globe. This can also help to address the standout incumbent challenge: access to reliable, wide-ranging ESG data. For example, the 2020 update to the 56-1 One Report established clear ESG standards and triggered online and offline capacity-building programmes to support listed firms’ compliance. SET is developing an ESG data platform with a structured template to promote the availability of comparable data, maximise value added from corporate sustainability disclosures, and foster collaboration between the business value chain and stakeholders. This is expected to support Thai companies along their ESG journey in an economically sustainable way, result in a greater number of sustainability-focused products and services, drive sustainable investing in the Thai investment community and ultimately “make the capital market work for everyone”, as outlined in the SET’s vision.
 

 

Read More

Continue Reading

Recent

Most Read

Join 14,216 other subscribers