Building an Automatic Speech Recognition (ASR) App using Streamlit and Whisper

## 1. Introduction Automatic Speech Recognition (ASR) is a core area of Artificial Intelligence that enables machines to convert spoken language into written text. It plays a crucial role in voice assistants, transcription services, accessibility tools, and real-time communication systems. The **Automatic Speech Recognition App** is a Streamlit-based application that converts uploaded audio files into text using OpenAI’s Whisper model. The application is optimized for speed and efficiency by using the lightweight **Whisper-Tiny** model and custom preprocessing techniques. This project demonstrates how modern deep learning models can be deployed in a simple web interface to perform real-time speech-to-text conversion without requiring external APIs. --- ## 2. Problem Statement In many real-world scenarios, spoken information must be converted into text for storage, search, or analysis. Manually transcribing audio is time-consuming, expensive, and error-prone. Challenges include: - Handling different audio formats and quality levels - Processing long audio files efficiently - Ensuring accurate transcription across accents and noise levels - Reducing computational cost for real-time applications The goal of this project is to build a lightweight, fast, and accurate speech recognition system that can run locally using open-source models. --- ## 3. Features The ASR application provides the following features: ### Audio Upload Support Users can upload `.wav` audio files directly into the application. ### Fast Speech-to-Text Conversion Uses Whisper-Tiny model optimized for speed and low memory usage. ### Audio Playback Uploaded audio can be played inside the Streamlit interface. ### Audio Preprocessing Includes: - Stereo to mono conversion - Audio trimming to 20 seconds - Resampling to 16kHz using optimized interpolation ### Lightweight Inference Uses Hugging Face Transformers without heavy pipeline overhead. ### Interactive UI Simple and user-friendly Streamlit interface. --- ## 4. Technologies Used | Technology | Purpose | |------------|----------| | Python | Core programming language | | Streamlit | Web application framework | | Hugging Face Transformers | ASR model implementation | | Whisper (OpenAI) | Speech recognition model | | PyTorch | Deep learning backend | | SoundFile | Audio file reading | | NumPy | Numerical processing | --- ## 5. How It Works The application uses OpenAI’s **Whisper-Tiny** model, which is a lightweight speech recognition model trained on large-scale multilingual audio data. ### Process Overview: 1. User uploads an audio file. 2. Audio is read using `soundfile`. 3. Audio is preprocessed: - Converted to mono - Trimmed to 20 seconds - Resampled to 16kHz 4. Audio is converted into model input features using Whisper Processor. 5. The model generates token predictions. 6. Tokens are decoded into readable text. The result is displayed instantly in the Streamlit UI. --- ## 6. Application Workflow ### Step 1: Upload Audio User uploads a `.wav` audio file. ### Step 2: Audio Playback The audio is played in the browser for verification. ### Step 3: Preprocessing Audio undergoes: - Channel normalization (mono conversion) - Duration limiting (max 20 seconds) - Resampling to 16kHz ### Step 4: Feature Extraction WhisperProcessor converts audio into model-compatible features. ### Step 5: Model Inference Whisper-Tiny generates predicted token IDs. ### Step 6: Decoding Tokens are decoded into final text transcription. ### Step 7: Output Display Final transcription is displayed in the UI. --- ## 7. Example Input ### Input Audio A sample `.wav` file containing speech: Audio: "Hello, welcome to the Automatic Speech Recognition demo. This system converts speech into text." --- ## 8. Example Output ### Transcription Result 📝 Hello, welcome to the Automatic Speech Recognition demo. This system converts speech into text. --- ## 9. Use Cases The Automatic Speech Recognition App can be used in multiple domains: ### Education Convert lectures and classroom recordings into text notes. ### Business Meetings Automatically transcribe meeting discussions. ### Call Centers Analyze customer support calls for quality monitoring. ### Accessibility Assist hearing-impaired users by converting speech into text. ### Voice Assistants Enable voice-controlled applications and commands. ### Data Analysis Convert spoken surveys and interviews into structured text data. --- ## 10. Future Improvements The application can be further enhanced in several ways: ### Multilingual Support Extend support for multiple languages and accents. ### File Format Expansion Support MP3, M4A, and FLAC formats. ### Real-Time Transcription Enable live speech-to-text streaming. ### Speaker Diarization Identify and separate multiple speakers in audio. ### Cloud Deployment Deploy on scalable cloud infrastructure for public access. ### Larger Whisper Models Upgrade to Whisper-Small or Whisper-Base for better accuracy. ### Confidence Scoring Display prediction confidence for each transcribed segment. --- ## 11. Conclusion The Automatic Speech Recognition App demonstrates how modern AI models like OpenAI Whisper can be efficiently integrated into lightweight web applications using Streamlit. By optimizing preprocessing and using a compact model variant, the system achieves a balance between speed and accuracy. This project highlights the practical use of speech recognition technology in real-world applications such as transcription services, accessibility tools, and voice-driven systems. It also showcases how deep learning models can be deployed without relying on external APIs. Overall, the application serves as a strong foundation for building advanced speech processing systems and demonstrates the growing capabilities of open-source AI models in handling complex audio tasks.