Automatic translator from Portuguese (voice and text) to Portuguese sign language

Abstract

One of the major focuses of technology, engineering, and computer science is to solve problems and improve the quality of life in health. The relationship between health and technology has made great advances in the last decades, and many cooperations are foreseen. Some citizens still have to deal with many obstacles daily to cope with the society as it is designed. One of the obstacles citizens deal with is deaf and hearing impairment. In Portugal, there are about 100,000 to 150,000 people with some level of hearing loss, and of these, around 30,000 people use Portuguese Sign Language as their mother tongue (1); National Institute for Rehabilitation, (2). The greatest difficulties are appropriate or amend if necessary. Encountered by them are poor communication with hearing people and the need of a translator person. Communication became even more complicated with the mandatory use of face mask, respect for the maximum number of people per division and social distancing due to the COVID-19 pandemic. To diminish or even solve this issue, the authors developed an automatic voice and text translation system to Portuguese sign language with captions. Sign language will be very useful for deaf and hearing impairment citizens, and captions will be useful for deaf and hearing impairment citizens who don’t understand sign language. For that, programming language tools were developed to create a translator with those requests. This article describes the developments and achievements. For the proof of concept, the translator started with 16 images in the database and reached a confidence level between 70 and 90%. This is an incentive to further developments that authors are continuing to produce more improvements in the developed tool.