research & development

R&D is knowledge and activity used by companies to create new and improved products, processes, and services to fill market needs

17 Ways to Say Yes: Toward Nuanced Tone of Voice in AAC and Speech Technology

Title17 Ways to Say Yes: Toward Nuanced Tone of Voice in AAC and Speech Technology
Publication TypeJournal Article
AbstractPeople with complex communication needs who use speech-generating devices have very little expressive control over their tone of voice. Despite its importance in human interaction, the issue of tone of voice remains all but absent from AAC research and development however. In this paper, we describe three interdisciplinary projects, past, present and future: The critical design collection Six Speaking Chairs has provoked deeper discussion and inspired a social model of tone of voice; the speculative concept Speech Hedge illustrates challenges and opportunities in designing more expressive user interfaces; the pilot project Tonetable could enable participatory research and seed a research network around tone of voice. We speculate that more radical interactions might expand frontiers of AAC and disrupt speech technology as a whole.
AuthorsPullin, G., and Hennig S.
Year of Publication2015
PublicationAugmentative & Alternative Communication
Volume31
Issue2
Pages170-180
ISSN0743-4618 (print), 1477-3848 (online)
Publisher DOIhttp://informahealthcare.com/doi/abs/10.3109/07434618.2015.1037930
Keywords (MeSH)communication, communication aids for disabled, communication disorders, community-based participatory research, equipment design, pilot projects, speech, voice

Adjusting dysarthric speech signals to be more intelligible

TitleAdjusting dysarthric speech signals to be more intelligible
Publication TypeJournal Article
AbstractThis paper presents a system that transforms the speech signals of speakers with physical speech disabilities into a more intelligible form that can be more easily understood by listeners. These transformations are based on the correction of pronunciation errors by the removal of repeated sounds, the insertion of deleted sounds, the devoicing of unvoiced phonemes, the adjustment of the tempo of speech by phase vocoding, and the adjustment of the frequency characteristics of speech by anchor-based morphing of the spectrum. These transformations are based on observations of disabled articulation including improper glottal voicing, lessened tongue movement, and lessened energy produced by the lungs. This system is a substantial step towards full automation in speech transformation without the need for expert or clinical intervention. Among human listeners, recognition rates increased up to 191% (from 21.6% to 41.2%) relative to the original speech by using the module that corrects pronunciation errors. Several types of modified dysarthric speech signals are also supplied to a standard automatic speech recognition system. In that study, the proportion of words correctly recognized increased up to 121% (from 72.7% to 87.9%) relative to the original speech, across various parameterizations of the recognizer. This represents a significant advance towards human-to-human assistive communication software and human–computer interaction.
AuthorsRudzicz, F.
Year of Publication2013
PublicationComputer Speech & Language
Volume27
Issue6
Pages1163-1177
ISSN0885-2308 (print), 1095-8363 (online)
Publisher DOIhttp://www.sciencedirect.com/science/article/pii/S0885230812001003
Keywords (MeSH)adult, dysarthria, software, speech, voice

An augmentative and alternative communication tool for children and adolescents with cerebral palsy

TitleAn augmentative and alternative communication tool for children and adolescents with cerebral palsy
Publication TypeJournal Article
AbstractPeople with disabilities face many accessibility problems in their daily lives. One such group of disabled people, children who suffer from cerebral palsy, faces specific challenges, which arise due to motor dysfunction and communication disorders. In the effort to help these children, augmentative and alternative communication (AAC) systems emerge as solutions to supplement spoken communication or to replace it completely. But current AAC solutions do not always address specific communication needs. Determining how to collaborate directly with cerebral palsy children as partners in the assistive technology design process is still an open research problem. This work discusses the development of an AAC computer-based solution to act as a complement to therapist’s AAC activities in helping make the lives of children with disabilities better. The AAC tool is based on symbols located on a computer screen and uses symbol and phrase suggestion strategies aiming to increase communication efficiency. Two school-age students with cerebral palsy participated in this study. Quantitative reports and qualitative assessments from a speech and an occupational therapist are presented. This research deals with a set of accessibility guidelines that benefit researchers and practitioners, giving more evidence about the design of AAC computer-based solutions for people with limited speech or language skills.
AuthorsSaturno, C. E., Garcia Ramirez A. R., Conte M. J., Farhat M., and Piucco E. C.
Year of Publication2015
PublicationBehaviour and Information Technology
Volume34
Issue6
Pages632-645
ISSN0144-929X (print), 1362-3001 (online)
Publisher DOIhttp://www.tandfonline.com/doi/abs/10.1080/0144929X.2015.1019567?src=rec...
Keywords (MeSH)cerebral palsy, child, communication, communication disorders

Anticipatory other-completion of augmentative and alternative communication talk: A conversation analysis study

TitleAnticipatory other-completion of augmentative and alternative communication talk: A conversation analysis study
Publication TypeJournal Article
AbstractPurpose: The study described here investigates the practice of anticipatory completion of augmentative and alternative communication (AAC) utterances in progress. The aims were to identify and analyse features of this practice as they occur in natural conversation between a person using an AAC system and a family member. Method: The methods and principles of Conversation Analysis (CA) were used to video record conversations between people with progressive neurological diseases and a progressive speech disorder (dysarthria) and their family members. Key features of interaction were identified and extracts transcribed. Four extracts of talk between a man with motor neurone disease/amyotrophic lateral sclerosis and his mother are presented here. Results: Anticipatory completion of AAC utterances is intimately related to the sequential context in which such utterances occur. Difficulties can arise from topic shifts, understanding the intended action of an AAC word in progress and in recognising the possible end point an utterance. Conclusions: The analysis highlights the importance of understanding how AAC talk works in everyday interaction. The role of co-participants is particularly important here. These results may have implications for both AAC software design and clinical intervention.
AuthorsBloch, S.
Year of Publication2011
PublicationDisability and Rehabilitation: An International, Multidisciplinary Journal
Volume33
Issue3
Pages261-269
ISSN0963-8288 (print), 1464-5165 (electronic)
Publisher DOIhttp://informahealthcare.com/doi/abs/10.3109/09638288.2010.491574
Keywords (MeSH)adult, amyotrophic lateral sclerosis, communication, communication aids for disabled, comprehension, interpersonal relations, speech disorders

Augmentative and alternative communication in adolescents with severe intellectual disability: a clinical experience

TitleAugmentative and alternative communication in adolescents with severe intellectual disability: a clinical experience
Publication TypeJournal Article
AbstractAIM: Augmentative and alternative communication devices proved to be effective in patients with severe intellectual disability to overcome their communication impairments. In order to give a contribution for design of augmentative and alternative communication systems that better meet the needs of beginning communicators we decided to report our clinical experience about using augmentative and alternative communication in adolescents with severe intellectual disability. METHODS: Five patients who underwent a long time traditional speech rehabilitation program (at least 5 years) with scant improvements in linguistic function were recruited and evaluated by means of the Vineland Adaptive Behaviour Scale before and after a three years augmentative and alternative communication intervention carried out by a multidisciplinary team. RESULTS: After the rehabilitative intervention patients showed an improvement in communication, daily living skills and socialization as measured by the Vineland Adaptive Behaviour Scale. CONCLUSION: Augmentative and alternative communication is an effective rehabilitation approach to people with severe intellectual disability and impairments in linguistic expression. Moreover augmentative and alternative communication is a useful tool allowing these patients to increase their social participation also enhancing their self-esteem. Our clinical experience confirmed these topics also in adolescents who underwent a long time traditional speech rehabilitation program with scant improvements, providing practical information to clinicians.
AuthorsUliano, Domenico, Falciglia G., Del Viscio C., Picelli A., Gandolfi M., and Passarella A.
Year of Publication2010
Date PublishedJun
PublicationEuropean Journal of Physical & Rehabilitation Medicine.
Volume46
Issue2
Pages147-52
ISSNprint: ISSN 1973-9087, online: ISSN 1973-9095
Publisher DOIhttp://www.minervamedica.it/en/journals/europa-medicophysica/article.php...
Notes*FULL TEXT ARTICLE PROVIDED*
Keywords (MeSH)activities of daily living, adolescent, communication aids for disabled, communication disorders, intellectual disability, socialization

Building personalised synthetic voices for individuals with severe speech impairment

TitleBuilding personalised synthetic voices for individuals with severe speech impairment
Publication TypeJournal Article
AbstractFor individuals with severe speech impairment accurate spoken communication can be difficult and require considerable effort. Some may choose to use a voice output communication aid (or VOCA) to support their spoken communication needs. A VOCA typically takes input from the user through a keyboard or switch-based interface and produces spoken output using either synthesised or recorded speech. The type and number of synthetic voices that can be accessed with a VOCA is often limited and this has been implicated as a factor for rejection of the devices. Therefore, there is a need to be able to provide voices that are more appropriate and acceptable for users. This paper reports on a study that utilises recent advances in speech synthesis to produce personalised synthetic voices for 3 speakers with mild to severe dysarthria, one of the most common speech disorders. Using a statistical parametric approach to synthesis, an average voice trained on data from several unimpaired speakers was adapted using recordings of the impaired speech of 3 dysarthric speakers. By careful selection of the speech data and the model parameters, several exemplar voices were produced for each speaker. A qualitative evaluation was conducted with the speakers and listeners who were familiar with the speaker. The evaluation showed that for one of the 3 speakers a voice could be created which conveyed many of his personal characteristics, such as regional identity, sex and age.
AuthorsCreer, S., Cunningham S., Green P., and Yamagishi J.
Year of Publication2013
PublicationComputer Speech & Language
Volume27
Issue6
Pages1178-1193
ISSN0885-2308 (print), 1095-8363 (online)
Publisher DOIhttp://www.sciencedirect.com/science/article/pii/S0885230812000836
Keywords (MeSH)adult, communication aids for disabled, dysarthria, speech disorders, voice

Critical Issues Using Brain-Computer Interfaces for Augmentative and Alternative Communication

TitleCritical Issues Using Brain-Computer Interfaces for Augmentative and Alternative Communication
Publication TypeJournal Article
AbstractBrain-computer interfaces (BCIs) may potentially be of significant practical value to patients in advanced stages of amyotrophic lateral sclerosis and locked-in syndrome for whom conventional augmentative and alternative communication (AAC) systems, which require some measure of consistent voluntary muscle control, are not satisfactory options. However, BCIs have primarily been used for communication in laboratory research settings. This article discusses 4 critical issues that should be addressed as BCIs are translated out of laboratory settings to become fully functional BCI/AAC systems that may be implemented clinically. These issues include (1) identification of primary, secondary, and tertiary system features; (2) integrating BCI/AAC systems in the World Health Organization’s International Classification of Functioning, Disability and Health framework; (3) implementing language-based assessment and intervention; and (4) performance measurement. A clinical demonstration project is presented as an example of research beginning to address these critical issues.
AuthorsHill, K., Kovacs T., and Shin S.
Year of Publication2015
PublicationArchives of Physical Medicine and Rehabilitation
Volume96
Issue3 supplement 1
PagesS8-S15
ISSN0003-9993 (print) ,1532-821X (online)
Publisher DOIhttp://www.sciencedirect.com/science/article/pii/S0003999314003463
Keywords (MeSH)amyotrophic lateral sclerosis, brain-computer interfaces, communication aids for disabled, computer peripherals, language, user-computer interface

Designing interaction, voice, and inclusion in AAC research

TitleDesigning interaction, voice, and inclusion in AAC research
Publication TypeJournal Article
AbstractThe ISAAC 2016 Research Symposium included a Design Stream that examined timely issues across augmentative and alternative communication (AAC), framed in terms of designing interaction, designing voice, and designing inclusion. Each is a complex term with multiple meanings; together they represent challenging yet important frontiers of AAC research. The Design Stream was conceived by the four authors, researchers who have been exploring AAC and disability-related design throughout their careers, brought together by a shared conviction that designing for communication implies more than ensuring access to words and utterances. Each of these presenters came to AAC from a different background: interaction design, inclusive design, speech science, and social science. The resulting discussion among 24 symposium participants included controversies about the role of technology, tensions about independence and interdependence, and a provocation about taste. The paper concludes by proposing new directions for AAC research: (a) new interdisciplinary research could combine scientific and design research methods, as distant yet complementary as microanalysis and interaction design, (b) new research tools could seed accessible and engaging contextual research into voice within a social model of disability, and (c) new open research networks could support inclusive, international and interdisciplinary research.
AuthorsPullin, G., Treviranus J., Patel R., and Higginbotham J.
Year of Publication2017
PublicationAAC Augmentative and Alternative Communication
Volume33
Issue3
Pages139-148
ISSN0743-4618 (print) 1477-3848 (online)
Publisher DOIhttp://dx.doi.org/10.1080/07434618.2017.1342690
Keywords (MeSH)communication aids for disabled, equipment design, research, research design, technology

Personal storytelling: Using Natural Language Generation for children with complex communication needs, in the wild…

TitlePersonal storytelling: Using Natural Language Generation for children with complex communication needs, in the wild…
Publication TypeJournal Article
AbstractThis paper describes a Natural Language Generation system (NLG), HOW WAS SCHOOL TODAY? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibility study of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice re-cording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for NLG in augmented and alternative communication are also discussed.
AuthorsTintarev, N., Reiter E., Waller A., and Reddington J.
Year of Publication2016
PublicationInternational Journal of Human-Computer Studies
Volume92-93
Pages1-16
ISSN1071-5819 (print) 1095-9300 (online)
Publisher DOIhttp://dx.doi.org/10.1016/j.ijhcs.2016.04.005
Keywords (MeSH)child, communication disorders, technology

Preliminary investigation of visual attention to human figures in photographs: potential considerations for the design of aided AAC visual scene displays

TitlePreliminary investigation of visual attention to human figures in photographs: potential considerations for the design of aided AAC visual scene displays
Publication TypeJournal Article
AbstractPURPOSE Many individuals with complex communication needs may benefit from visual aided augmentative and alternative communication systems. In visual scene displays (VSDs), language concepts are embedded into a photograph of a naturalistic event. Humans play a central role in communication development and might be important elements in VSDs. However, many VSDs omit human figures. In this study, the authors sought to describe the distribution of visual attention to humans in naturalistic scenes as compared with other elements. METHOD Nineteen college students observed 8 photographs in which a human figure appeared near 1 or more items that might be expected to compete for visual attention (such as a Christmas tree or a table loaded with food). Eye-tracking technology allowed precise recording of participants' gaze. The fixation duration over a 7-s viewing period and latency to view elements in the photograph were measured. RESULTS Participants fixated on the human figures more rapidly and for longer than expected based on the size of these figures, regardless of the other elements in the scene. CONCLUSIONS Human figures attract attention in a photograph even when presented alongside other attractive distracters. Results suggest that humans may be a powerful means to attract visual attention to key elements in VSDs.
AuthorsWilkinson, K. M., and Light J. C.
Year of Publication2011
Date PublishedDec
PublicationJournal of Speech, Language, and Hearing Research
Volume54
Issue6
Pages1644-57
ISSN1558-9102 (print), 1092-4388 (electronic)
Publisher DOIhttp://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462357/
Notes*Public access Author Manuscript text*
Keywords (MeSH)adolescent, attention, communication aids for disabled, communication disorders, equipment design, eye movements, pilot projects, young adult

Supporting Personal Narrative for Children with Complex Communication Needs

TitleSupporting Personal Narrative for Children with Complex Communication Needs
Publication TypeJournal Article
AbstractChildren with complex communication needs who use voice output communication aids seldom engage in extended conversation. The “How was School today...?” system has been designed to enable such children to talk about their school day. The system uses data-to-text technology to generate narratives from sensor data. Observations, interviews and prototyping were used to ensure that stakeholders were involved in the design of the system. Evaluations with three children showed that the prototype system, which automatically generates utterances, has the potential to support disabled individuals to participate better in interactive conversation. Analysis of a conversational transcript and observations indicate that the children were able to access relevant conversation and had more control in the conversation in comparison to their usual interactions where control lay mainly with the speaking partner. Further research to develop an improved, more rugged system that supports users with different levels of language ability is now underway.
AuthorsBlack, R., Waller A., Turner R., and Reiter E.
Year of Publication2012
PublicationACM Transactions on Computer-Human Interaction (TOCHI)
Volume19
Issue2
Pages1-35
ISSN1073-0516 (print) 1557-7325 (online)
Publisher DOIhttp://dl.acm.org/citation.cfm?id=2240163
Keywords (MeSH)child, communication aids for disabled, narration

Supporting the Communication, Language, and Literacy Development of Children with Complex Communication Needs

TitleSupporting the Communication, Language, and Literacy Development of Children with Complex Communication Needs
Publication TypeJournal Article
AbstractChildren with complex communication needs (CCN) resulting from autism spectrum disorders, cerebral palsy, Down syndrome and other disabilities are severely restricted in their participation in educational, vocational, family, and community environments. There is a substantial body of research that demonstrates convincingly that children with CCN derive substantial benefits from augmentative and alternative communication (AAC) in their development of communication, language and literacy skills, with no risk to their speech development. Future research must address two significant challenges in order to maximize outcomes for children with CCN: (1) investigating how to improve the design of AAC apps/ technologies so as to better meet the breadth of communication needs for the diverse population of children with CCN; and (2) ensuring the effective translation of these evidence-based AAC interventions to the everyday lives of children with CCN so that the possible becomes the probable. This article considers each of these challenges in turn, summarizing the state of the science as well as directions for future research and development.
AuthorsLight, J. C., and McNaughton D B.
Year of Publication2012
PublicationAssistive Technology: The Official Journal of RESNA
Volume24
Issue1
Pages34-44
ISSN1040-0435 (print); 1949-3614 (online)
Publisher DOIhttp://www.tandfonline.com/doi/abs/10.1080/10400435.2011.648717#.UzXQTYW...
Keywords (MeSH)child, communication aids for disabled, communication disorders, language development, literacy

Towards the improvement of Augmentative and Alternative Communication through the modelling of conversation

TitleTowards the improvement of Augmentative and Alternative Communication through the modelling of conversation
Publication TypeJournal Article
AbstractNon-speaking people who use Augmentative and Alternative Communication (AAC) systems typically have low rates of communication which reduces their ability to interact with others. Research and development continues in the quest to improve the effectiveness of AAC systems in terms of communication rate and impact. One strategy involves making the basic unit of communication an entire utterance, and designing the AAC system to make the storage, retrieval and production of utterances as easy and efficient as possible. Some approaches take this further and include texts, narratives and/or multimedia material for use in conversation. AAC systems operating in such a manner require a structure for containing and managing conversational material and supporting the production of output during conversation. Ideally such a structure should be modelled on the way actual conversations proceed. A number of partial models for this have been presented thus far. These are reviewed in the paper and an integrated model is then proposed that includes both the structure of a conversation and the way in which an AAC system might produce conversational output (e.g. utterances, texts, multimedia items or combinations of these). Modelling the process in this way gives a structure with which an AAC system can organize the support and guidance that it offers to the person using the system. The paper concludes with consideration of three areas of development for further investigation.
AuthorsArnott, J. L., and Alm N.
Year of Publication2013
PublicationComputer Speech & Language
Volume27
Issue6
Pages1194-1211
ISSN0885-2308 (print), 1095-8363 (online)
Publisher DOIhttp://www.sciencedirect.com/science/article/pii/S0885230812000915
Keywords (MeSH)communication, communication aids for disabled, communication disorders