Research Brief

Browser extension helps the visually impaired interpret online images

Anonymous — Tue, 30 Jan 2018 21:01:29 +0000

Browser extension helps the visually impaired interpret online images Anonymous (not verified) Tue, 01/30/2018 - 14:01

Imagine internet browsing without the ability to make sense of images. It’s a problem that visually impaired computer users face every day. While screen reading technology gives users audible access to written content, it needs written descriptions to interpret images, and often there isn’t any.

Some website developers include descriptions of images in the code (called “alt text”) because it improves their websites’ search engine rankings. However, there’s no mechanism for determining whether these descriptions are accurate or informative. As a result, developers often enter one-word descriptions such as “image” or “photo,” leaving the visually impaired with no useful information about the image.

To help address this problem, an ATLAS Institute researcher developed a system that collects captions and alt text associated with other instances of the same photo elsewhere online, associating human-authored descriptions with every website where it appears. Called Caption Crawler, the image captioning system compiles descriptions in a database: if a photo has never been queried, it will offer alt text in about 20 seconds; if the photo has previously been processed, alt text is available almost immediately.

The technology was developed by Darren Guinness, a PhD student in the ATLAS Interactive Robotics and Novel Technologies (IRON) Lab and the Superhuman Computing Lab, working in conjunction with Microsoft Research’s Edward Cutrell and Meredith Ringel Morris. The research, which merges the benefits of a fully automated system with the quality of human-authored content, will be presented at the Association for Computing Machinery’s (ACM) 2018 Conference on Human Factors in Computing Systems (CHI) in Montreal in April.

Users who want Caption Crawler to replace poor-quality alt text, press a keyboard shortcut to request a replacement. The screen reader automatically speaks the new caption, which is the longest caption found for a particular photo. Users can also use a different shortcut to access any additional found captions.

Caption Crawler only works with images used on multiple websites, but the approach is effective because about half of website administrators provide informative photo descriptions, Guinness says.

“Although this approach cannot caption unique images that only appear in a single place online, it can increase the accessibility of many online images,” he says. “Caption Crawler is a low latency, incredibly low-cost solution to a big problem. It produces human-quality captioning without incurring additional costs in human labeling time.”

Caption Crawler combines a Google Chrome Browser Extension with a Node.js cloud server. The browser extension searches the Document Object Model (DOM) of the active webpage for image tags and background images, which are then sent to the server for caption retrieval. When Caption Crawler finds a caption for an image, the caption is streamed back to the browser extension, which then associates the caption to the image.

Research shows humans produce higher quality captions than automated computer and machine-learning based approaches, Guinness says. Caption Crawler uses a hybrid system that captures both, prioritizing human captioning over machine learning and computer vision-based approaches. If no human-authored captions can be found, computer-generated captions from Microsoft’s CaptionBot are used to describe the image. When the text from CaptionBot is read aloud, the screen reader first speaks the words “CaptionBot,” so that the user is aware that the caption is not human-authored.

“Hybrid systems that meld both human-quality text and machine learning approaches hold a lot of promise for improving access to online media,” Guinness says.

Full CHI 2018 paper

[video:https://vimeo.com/249025146]

ATLAS researcher Darren Guinness developed technology that conveys online photo content to the visually impaired

Off

Traditional

White

Novel Program Teaches Computer Science Alongside Art, Biology

Anonymous — Thu, 07 Dec 2017 20:46:22 +0000

Novel Program Teaches Computer Science Alongside Art, Biology Anonymous (not verified) Thu, 12/07/2017 - 13:46

Lila Finch, an ATLAS PhD student, has designed a set of learning experiences that teach computer science alongside art and biology.

Off

Traditional

White

When celebrities die, social media gets toxic

Anonymous — Wed, 06 Dec 2017 23:29:37 +0000

When celebrities die, social media gets toxic Anonymous (not verified) Wed, 12/06/2017 - 16:29

Katie Gach's research about how people mourn finds that people are mean to each other online, even in times of tragedy.

Off

Traditional

White

Who needs safe water?

Anonymous — Mon, 27 Nov 2017 21:36:06 +0000

Who needs safe water? Anonymous (not verified) Mon, 11/27/2017 - 14:36

After growing up drinking water from community water filtration stations in Pakistan, Mustafa Naseem discovered in his research just how often the water from these sources can be compromised. Each year, around 41,000 children in Pakistan die from diarrhea caused by contaminated water supplies. Now, as Information and Communication Technology for Development (ICTD) Expert in Residence at CU ��«��Ƶ’s ATLAS Institute, Naseem has set his sights on changing that.

In September, Naseem and a team from the Information Technology University (ITU) in Lahore, Pakistan, were jointly awarded $374,000 from the U.S. Agency for International Development (USAID) and Pakistan’s Higher Education Commission (HEC) to design a low-cost, modular water quality testing and metering system to be installed and tested at 20 Punjab filtration plants over a three-year period.

“I’m really excited to have the opportunity to contribute to a water-related issue in Pakistan,” said Naseem. “Many people there have limited access to clean drinking water. It’s an issue close to my heart.”

The project, which also provides support for a graduate student to oversee implementation, was selected from more than 200 eligible applications submitted to the Pakistan-United States Science & Technology Cooperation program in 2017. Under the program, which was established in 2005, the governments of Pakistan and the United States cooperate in science, technology, engineering and education for mutual benefit. The program is implemented by the U.S. National Academy of Sciences in the U.S. and the HEC in Pakistan.

Safety

To improve water quality monitoring, Naseem’s team will design a low-cost, automated, modular water testing unit equipped with sensors that monitor free-chlorine, turbidity, Ph and total dissolved solids. The system will be connected to the Internet via cellular data networks, allowing results to be uploaded regularly to the municipal water authority’s servers, where automated alerts can warn technicians when levels exceed parameters.

Naseem, who investigated water technologies in Lahore over a three-year period, says improvements in the frequency and method of water quality testing can have a big impact on public health. They are also necessary to meet guidelines established by the World Health Organization.

Presently, water monitoring is performed just once a month, and errors can occur during sampling and testing. In addition, systems rely on filters that should be changed at intervals based on how much water has passed through the filter; however, there’s often no mechanism for measuring water volume.

“Between the monthly tests, no one knows the quality of the water,” he said. “Even if samples are collected and stored properly, the laboratory performs well and the plant workers act on laboratory recommendations, there’s a lot of uncertainty.”

Distribution

In addition to tackling water safety, their project aims to solve another major challenge: distributing water more equitably. Naseem explains that water dispensing stations in Lahore and other cities have limited supplies and sometimes run dry. However, no limits are placed on how much each family can take. As a result, wealthier households that fetch large amounts of water by truck place a greater burden on the system than poorer households that collect their water by hand.

The team at ITU, led by Tauseef Tauqeer, head of the Electrical Engineering Department and an associate professor, will focus on this problem by continuing work on an automated dispensing unit originally developed by students in a design lab that Naseem mentored in 2014. Named the Water ATM, the system would use government-issued RFID cards to identify users and limit how much water is dispensed based on household size.

During the third year, the team will look towards commercialization so that a local industry partner can scale-up the technology they’ve developed.

A fringe benefit of this technology is that it could enable water dispensing stations to remain open 24-hours a day, cutting down on the long lines that currently form during daylight hours.

Beyond technology

Naseem explains that the work of creating remote water sensing units is complex, but it’s not their biggest challenge. Success depends on designing a system that communities want to embrace, he says. “Sensitivity to social and cultural concerns is critical,” says Naseem. “And giving the community a voice is also key.” If the rationing system upsets enough powerful people accustomed to unlimited water supplies, they could have the whole program shut down. Similarly, if the technology is going to be adopted, it must be reliable and easy to use.

Another concern is what to do when water supplies exceed safety parameters. It’s not a straightforward matter, explains Naseem. While the water company will have the ability to shut down plants from afar, that would create another set of problems: “Even if the water is not at the required safety levels, it may be the only source of drinking water for a community,” says Naseem. “It’s a community-run filtration plant, and the residents need to have some amount of say as to what happens.”

One option is to install digital signs that display the current safety level of water coming out of the faucet. When it’s unsafe, residents could continue to draw water, but the signs would let them know they need to boil or further treat it before drinking.

“Information and Communication Technology for Development is not just about technology; it’s about these ethical considerations. You have to make difficult decisions,” says Naseem. “When I enter these communities, there’s so much desperation and there’s so much love. We are sending students to vulnerable places, and there is a danger of their having a savior complex. I want to make sure students realize that there is beauty and knowledge we can rely on in these communities.”

If successful, the team could have an immediate impact on the region. Their key partner, the government-run Punjab Saaf Pani Company, plans to install 1,500 water filtration plants across Punjab in the next five years. If their plans to subsequently commercialize the technology are successful, their reach could be much greater—an inexpensive, early-warning water safety system could improve the public health of countless communities around the world.

Mustafa Naseem and a team from the Information Technology University (ITU) in Lahore, Pakistan, design a low-cost, modular water quality testing and metering system to be installed and tested at 20 Punjab filtration plants over a three-year period.

Off

Traditional

White

Computer music as gateway to learning contemporary computer science

Anonymous — Wed, 25 Oct 2017 15:19:46 +0000

Computer music as gateway to learning contemporary computer science Anonymous (not verified) Wed, 10/25/2017 - 09:19

Designing and programming networked technologies might seem like an advanced topic for computer science education, but researchers from ��«��Ƶ’s ATLAS Institute and Goldsmiths, University of London have demonstrated that even middle school students with limited prior programming experience can develop interactive, creative, networked technologies using specialized technology.

Principal investigator Ben Shapiro, an assistant professor of computer science at the ATLAS Institute, ��«��Ƶ, worked with his students, and his Goldsmiths colleague Rebecca Fiebrink to construct BlockyTalky WeJam, an interactive computer music toolkit for youth to create computer music systems for collaborative performances. BlockyTalky allows young people to create networks of sensing devices and musical synthesizers, and to program the musical and interactive behaviors of these devices. It also enables users to build a variety of other kinds of physical, sensor-rich interfaces, which students can combine in distributed systems of their own design.

The study found that exposure to BlockyTalky supported learning about computer science concepts, and changed students' perceptions of computing. “Many American students and adults have negative stereotypes of science and scientists, believing that they are socially distant, dangerous, workaholic, peculiar, irreligious and missing fun in their lives,” said Shapiro. “Creative computing, like in computer music, provides an opportunity to combat pervasive and insidious misperceptions of computer science and computer scientists.”

The work was reported in the article, “Tangible Distributed Computer Music for Youth,” published in the Summer 2017 issue of Computer Music Journal. The work was supported by the US National Science Foundation (CNS-1418463), the National Center for Women and Information Technology, and LEGO Education.

Middle school students with limited prior programming experience develop interactive, creative, networked technologies.

Off

Traditional

White

Paper robots teach kids basic engineering principles

Anonymous — Sat, 23 Sep 2017 17:59:09 +0000

Paper robots teach kids basic engineering principles Anonymous (not verified) Sat, 09/23/2017 - 11:59

NSF grant supports research aimed at cultivating a passion for STEM fields among young children.

Off

Traditional

White

Software framework turns small robots into controllers

Anonymous — Fri, 22 Sep 2017 23:12:51 +0000

Software framework turns small robots into controllers Anonymous (not verified) Fri, 09/22/2017 - 17:12

ATLAS researchers have developed a software framework that enables developers who lack experience programming robots to repurpose off-the-shelf toy and educational devices to serve as input-output devices for desktop applications.

Dan Szafir, assistant professor of computer science in the ATLAS Institute, Shaun Kane, assistant professor of computer science and ATLAS faculty fellow, and doctoral student Darren Guinness, presented the GUI (graphical user interfaces) Robots framework at the Designing Interactive Systems (DIS 2017) conference in Edinburgh this summer.

The GUI Robots toolkit enables developers to transform low cost robots (often less than $100 USD) into wireless controllers for desktop applications. The toolkit supports connecting such devices with a wide range of software applications, including web browsers, 3D modeling tools and video games, extending the application user interfaces into the physical world in interesting new ways.

With most software applications confined to screen-based graphical user interfaces, tactile input and haptic feedback are an important frontier for interaction design.

Szafir says, “developers can use our framework to quickly prototype tangible interactions and attach them to existing applications. As consumer-oriented, wirelessly-connected robots become ubiquitous, our work can enable new user experiences in which an ecosystem of helpful robots extends traditional graphical user interface applications.”

Similar to a Nintendo Wii console where the controller detects movements in three dimensions, providing realistic on-screen action games, users can repurpose devices to apply the same type of movement in any existing application. For instance, as part of the project a Sphero Ollie was configured as a controller for Rovio Entertainment’s popular Angry Birds game: rolling the robot back and then moving it forward launched a bird.

Robots can also be programmed to provide physical feedback, such as vibrating when a bird is launched. Or, a user could manipulate a 3D object on the screen by moving and rotating the robot controller in the air.

To test the GUI Robots toolkit, researchers asked twelve developers to build controllers for two applications: Angry Birds and Windows Movie Maker. All of the developers were able to build working prototypes of the Angry Birds controller within a half hour; the Movie Maker controller took them a little longer.

As an extension to their work, the team is exploring the use of GUI Robots to provide haptic displays for visually impaired users. Movement or tactile feedback from a robot would be deployed to communicate the kind of information that is typically displayed in charts, diagrams, movies and even interactive simulations. Other promising uses include educational software, controllers for musical instruments and 3D modeling.

Read the research

Software turns digital toys into digital tools for desktop applications.

Off

Traditional

White

Hear-Here—Radio by anyone and everyone

Anonymous — Wed, 06 Sep 2017 15:27:56 +0000

Hear-Here—Radio by anyone and everyone Anonymous (not verified) Wed, 09/06/2017 - 09:27

What do people say when given an opportunity to speak on the radio for two-seconds?

They count. They list the names of cheeses. And they play instruments and sing.

We know this because of Hear-Here, a crowdsourced, crowd-directed electronic space where anything goes, as long as you can say it, sing it or play it in two seconds. During broadcasts/recording sessions, anyone who wishes to participate simply has to load a web page, wait for their turn and then start making sound.

August Black, technologist-in-residence at the ��«��Ƶ’s ATLAS Institute, developed the application that allows multiple users from remote locations to input live sound from their browsers to an FM station and a live streaming audio feed. He presented his research during the Audio Mostly 2017 conference in London, August 23–26.

“People are used to just listening to the radio,” says Black, who has a PhD in Media Arts & Technology from University of California, Santa Barbara. “With Hear-Here, anyone can contribute. They are not just being consumers.”

Hear-Here’s novelty lies in its simplicity and its direct overlapping of radio and Internet methodologies, says Black, who developed the initial prototype in 2015 during a 10-day Wave Farm residency, where artists experiment with broadcast media and the airwaves.

The system connects users in a browser-based, peer-to-peer network using WebRTC, a free, open source project that provides browsers and mobile applications with Real-Time Communications capabilities. A master server collects the input from participants and broadcasts it live on FM radio and online.

Anyone who wants to participate simply accesses a webpage that shows the number of current participants and their own position in the line-up. When it’s their turn, they have two seconds of air time.

Black, who began his academic career in sculpture and painting at Syracuse University, became interested in the intersection of radio and the Internet in 1996 while interning with the National Radio of Austria’s art-radio program. At the time the Internet was in its infancy, but the Ars Acustica Experts group of the European Broadcast Union built radio infrastructure to broadcast radio events simultaneously in different parts of Europe.

In 1998, Black wrote streaming software that allowed him to broadcast his live radio program simultaneously on the Internet and radio from different locations, including from an underwater location in the historic city center of Linz, Austria. At the time, most people did not have access to the Internet and data speeds were too slow for listeners to interact, but it was novel to overlap the Internet with radio, he said.

In 2002, Black developed software for a virtual mixing console so users could upload and mix audio together simultaneously online while being on air. Recent Internet advances in web sockets, which allow users to send requests and respond back and forth, and WebRTC, which allows browsers to send audio, video and data without a server, made Hear-Here possible, he says.

“The two seconds creates an environment in the same way a speed bump defines speed on the road, or a fence delineates space in a city,” he says.

Black plans to do further research with Hear-Here, including reducing the time between turns to nanoseconds so voices overlap, and increasing speaking time so participants have the opportunity to say entire phrases.

“The interesting part about making art on the radio is that you don’t see your audience and are speaking to everyday folks—they could be truck drivers or people cooking in the kitchen,” Black says. “The end result can be very surprising.”

A crowdsourced, crowd-directed electronic audio space where anything goes—for two seconds.

Off

Traditional

White