Convert Speech To Text For Free

In this section, we will explore the implementation of real-time audio streaming using the Speech API, focusing on the architecture and the technologies involved in the process.

Architecture Implementation

This implementation serves as a reference for testing various algorithms and scenarios. It is designed to run independently of the operating system or computer architecture, utilizing open-source web technologies accessible through any web browser. This approach allows integration into various web applications and services, including video conferencing.

Audio Input

The audio input is captured through the microphone via a web browser that accesses the ASR (Automatic Speech Recognition) application. This setup ensures that the audio is directly sourced from the user’s environment, providing real-time interaction.

Audio Extraction

To access and process the audio, the MediaDevices API is employed. The audio input is captured using the getUserMedia method, which samples audio at 16 kHz with 16 bits per sample. This configuration is optimal for speech recognition tasks, ensuring clarity and accuracy in the transcription process.

Audio Splitter

The Web Audio API plays a crucial role in controlling audio on the web. It allows developers to create a processing graph composed of audio nodes. In this implementation, audio samples from the audio extractor are processed within an AudioContext, which represents a filtering graph of audio processing nodes. Different nodes are created to implement various audio splitting algorithms, which are essential for managing audio streams efficiently.

Client-Side Processing

To reduce the computational load on the server, audio splitting is offloaded to the client. However, the transcription process remains server-side due to its high computational requirements, necessitating specialized equipment like GPUs. The audio is received from the audio extractor and then transmitted to the Client-ASR connection.

Communication Protocols

For the connections between the audio splitter and the ASR cluster, as well as the ASR cluster and the display, WebSockets are utilized. This technology enables bidirectional communication between clients and servers, allowing for the efficient transfer of raw audio samples and the reception of transcribed text. Other protocols, such as RTP or WebRTC, were not considered due to their additional features being unnecessary for this implementation. The selected library for implementing WebSockets on the client side is socket.io, which simplifies the process of establishing these connections.

By leveraging these technologies, developers can create robust applications that convert speech to text for free, enhancing user interaction and accessibility in real-time scenarios.