Speech recognition technologies (Speech-to-Text (SET)) have been widely used in everyday and business life for many years. A person gives voice commands to a navigator in a car, asks questions to search engines on the Internet, and controls a smart home. STT has made life easier for journalists, students, doctors, lawyers, and managers. It is obvious that technology will continue to evolve, providing more and more assistance to people in performing routine daily tasks.
What is Speech-to-Text?
This is a technology that allows you to transform an oral conversation into a text format. From a technical point of view, this is a multi-level process of step-by-step audio processing and analysis. Artificial intelligence listens to a conversation, and then begins work on its recognition, first identifying sounds, then letters, from which it adds syllables, words, and then sentences.
The literal translation from English of the term Speech-to-Text means speech to text. But sometimes this technology is also called Automatic Speech Recognition (ASR), which can be translated as automatic speech recognition.
History of ASR development
The first developments date back to the mid-50s of the last century. In the 70s, attempts were made to modernize the achievements. However, it became possible to truly develop ASR only in the early noughties, when machine learning technology appeared. Programs based on artificial intelligence appeared, which could be trained in speech recognition and its transformation into text.
The first generations of ASR worked according to the scheme:
- The speech was dictated into a microphone.
- A sound recorder recorded the spoken text.
- The audio track is actually a complex wave. The recording was transformed into a two-dimensional amplitude-frequency model that was distinguishable by ear.
- Phonemes stood out from this model – very short fragments of recordings corresponding to individual sounds. However, you can’t make a text out of phonemes. They must be converted first to letters, then to syllables, and then to add words.
- Phoneme conversion. Three models have been developed for this purpose:
- acoustic, which converts sound into phonemes;
- linguistic, which allows you to match phonemes with letters;
- A language model that analyzed the developments of acoustic and linguistic models, and then formed words and sentences.
Machine learning has given an impetus to the development of technology, significantly increasing the efficiency of the recognition process. The new algorithms are called end-to-end, which means end-to-end testing, a method of checking the system from start to finish. In the case of AI, this means training the system to perform a task from the raw data to the desired result without intermediate steps.
The modern acoustic model immediately converts sounds into letters, bypassing the phoneme stage. The language model is able to select words based on the context, which is especially important when the words sound the same. These achievements have become possible because AI training is conducted on the basis of huge volumes of texts. Hundreds and thousands of training sessions develop the machine’s semantic understanding, thanks to which it selects the right words.
The acoustic model plays a leading role in the end-to-end algorithm. It consists of two devices:
- An encoder that translates sound into a language that the machine can understand.;
- the decoder that generates the text.
Decoders used for speech transcription are different:
- CTC is a well–trained model that can recognize each character at the moment of its utterance. However, it makes many mistakes.
- RNN Converter works in streaming mode. This algorithm was trained based on acoustic and language models, so it understands the context and makes fewer mistakes when transcribing.
- LAS transcribes long texts well and understands the general context.
After decoding, the text is erased, i.e.:
- the spelling of numbers is being corrected (in letters or, conversely, in numbers);
- Punctuation marks are applied;
- capitalize letters, for example, in proper names, at the beginning of sentences.
In different cases, different types of transcription are required, for example, not the full text, but only a fragment of it or only dialogues. Different technologies are used to solve such problems.
Where they are used
ASR has found application in many areas of a person’s personal and professional life, and new opportunities appear literally every day.
Smart Home management
The technology is used in home device control centers. The host can give voice commands to the virtual assistant, who executes them by interacting with the equipment that is connected to the system.:
- turn on or turn off electrical appliances;
- controls the microclimate in the home (temperature, humidity);
- ensure the security of the residential perimeter;
- it will select entertainment content or other information on the Internet;
- it will entertain the child.
Journalism, mass media
In this area, the advent of STT has brought literally revolutionary changes. If earlier journalists had to record interviews on dictaphones, and sometimes manually, and then translate audio into text for hours, now artificial intelligence is engaged in this routine. It has significantly accelerated the recognition process, and the most advanced systems understand foreign languages, can translate, edit, distinguish between several speakers, take into account accents, understand slang, and know professional terms. Although it is necessary to proofread the text before submitting it to the editor (machines can still make mistakes), nevertheless, the work of workers of the writing genre has become much easier and more comfortable.
Education
Students and teachers, as well as journalists, always had to write a lot, preparing for lectures and taking notes on them. At the same time, not everything can be written down in time at the lecture, semantic errors often crept into the notes, and due to the fact that you had to write quickly, many notes were difficult to recognize later. Now these problems are a thing of the past: AI does an excellent job of taking notes on lectures. It has also become easier for teachers to prepare for classes.: now, to prepare a written summary, you do not need to waste time typing it, everything can be dictated.
Medicine
There are services that allow you to record the speech of medical workers. Their special feature is that they are trained to recognize medical terms and diagnoses. The use of innovation has partially automated patient admission. The doctor no longer needs to write down complaints, conclusions, and appointments in detail – it’s enough to say all this, and the AI will write it down and give it out in a ready-made form. The doctor can spend the freed-up time on a more detailed conversation with the patient. In hospitals, it took a lot of time to fill out medical records, now this routine is also automated, and doctors have received additional time for self-education, research, and observations.
Business
ASR technologies are most widely used in this field. Artificial intelligence made it possible to record meetings and create sammaris, which significantly increased communication within companies, as well as employee productivity and responsibility. Managers whose work involves telephone conversations have been given the opportunity to partially automate their work. Now they do not lose calls, they can automatically conduct surveys, send out thanks, and create customer cards. The AI, trained to understand and respond to basic requests, took on the task of solving simple questions that customers most often address.
With the help of AI, the task of HR specialists, as well as recruiting agencies, has been simplified. The machine can conduct an initial consultation with applicants for vacancies, screening out those who definitely won’t fit.
Limitations for ASR technologies
- Lack of training materials. In order for the machine to better understand spoken language and discussions using special terminology, it should be trained not on book texts, but on live human speech. There is not enough such material yet.
- The machine can be trained to understand a conversation in several languages and it will recognize each one. However, if a person interferes with words from two languages, the recognition quality will be low: the machine will not understand part of the text. The problem is relevant in countries where there are several official languages and people are accustomed to using words from two or more languages in their conversation. For example, Indian residents include Hindi words in English, and in the CIS countries, where communication is mainly conducted in Russian, locals use a lot of vocabulary from the local language.
- Systems still don’t work well with terms, especially highly specialized ones.
The prospects
The evolution of ASR will go in the direction of overcoming the limitations that are currently hindering its development. For example, AI does not do a good job of recognizing a conversation with an accent or using complex terms.
New generations will better understand not only accents, but also emotions and intonations, and speech itself will cease to resemble mechanical speech, but will match human speech as much as possible, which is especially important for business.
The creation of services capable of delivering not only high-quality transcriptions, but also translations, will expand their implementation in government agencies, healthcare, medicine, the financial sector, and e-commerce.
A promising area is the development of accelerated machine learning methods, as well as improving human control over learning and model operation.
Since intelligent recognition innovations work with personal data, the problem of their security remains urgent. Ensuring confidentiality should be a key point in improving the technology.
How can our Follow Up service help?
Our company offers the development of AI-secretary for logging business meetings. The program can easily be combined with the user’s work calendar. The bot makes a full audio recording of the meeting and processes it within a few minutes.:
- transcribes the conversation;
- identifies and records agreements, tasks, responsible persons, and deadlines;
- Creates a sammari and sends it to all interested parties.
The recognition accuracy of the technology is 98%, and the quality of sammarisation is 100%.
It is possible to train a model for the specifics of the company’s activities.
The company provides the user with 100 free trial minutes to familiarize themselves with the product. A flexible pricing plan has been developed for regular customers and growing teams.
