Building a Document Question Answering App with Streamlit and Donut AI

## Introduction Document understanding is a critical task in modern Artificial Intelligence. Organizations deal with a massive volume of documents such as invoices, receipts, contracts, forms, reports, and scanned records. Extracting information manually from these documents is time-consuming, expensive, and prone to human error. Recent advancements in Vision-Language Models (VLMs) have enabled AI systems to understand both visual and textual content simultaneously. One such model is Donut (Document Understanding Transformer), developed by NAVER CLOVA AI. Unlike traditional OCR-based systems, Donut directly processes document images and answers questions without requiring external Optical Character Recognition (OCR) tools. The **Document Question Answering App** is a Streamlit-based application that leverages the Donut model to allow users to upload document images and ask natural language questions about their content. The application provides an intuitive interface for extracting information from documents without manual reading or data entry. --- ## Problem Statement Businesses and organizations frequently process large collections of documents containing important information. Traditional document extraction methods typically rely on OCR engines such as Tesseract, followed by additional text-processing pipelines. These approaches often face challenges such as: * OCR inaccuracies in low-quality scans * Complex document layouts * Handwritten content * Multi-column formatting * Additional infrastructure requirements The challenge is to create a simple and efficient solution that can understand document content and answer user questions directly. The Document QA App addresses this challenge by utilizing the Donut model, which combines computer vision and natural language understanding into a single OCR-free architecture. --- ## Features The application offers several powerful features: ### Document Upload Users can upload document images in formats such as: * JPG * JPEG * PNG ### OCR-Free Processing The Donut model eliminates the need for external OCR installations or preprocessing pipelines. ### Natural Language Question Answering Users can ask questions in plain English about the uploaded document. Examples include: * What is the invoice number? * What is the total amount? * Who is the recipient? * What is the date? ### Interactive Streamlit Interface The application provides a clean and user-friendly interface accessible directly through a web browser. ### Real-Time Inference Answers are generated dynamically after users submit their questions. ### Cached Model Loading The model is loaded once and cached using Streamlit to improve application performance. --- ## Technologies Used The project integrates several modern AI technologies: | Technology | Purpose | | ------------ | ------------------------- | | Python | Core programming language | | Streamlit | Web application framework | | Transformers | Model inference framework | | Hugging Face | Pretrained AI models | | Donut Model | Document understanding | | Pillow (PIL) | Image processing | | PyTorch | Deep learning backend | The combination of these technologies enables efficient document analysis and question answering. --- ## How It Works The application uses the pretrained Donut model: ```python naver-clova-ix/donut-base-finetuned-docvqa ``` Unlike conventional OCR systems, Donut treats document understanding as a vision-language task. The workflow is as follows: 1. A user uploads a document image. 2. The image is processed by the Donut model. 3. The user enters a question. 4. The model analyzes both the image and question. 5. The model generates an answer directly from the document content. 6. The answer is displayed in the Streamlit interface. Because Donut performs OCR and reasoning internally, no external OCR software installation is required. --- ## Application Workflow ### Step 1: Upload Document The user uploads a document image through the Streamlit interface. ### Step 2: Display Document The uploaded image is displayed for verification. ### Step 3: Enter Question The user asks a question related to the document. Example: ```text What is the total amount? ``` ### Step 4: AI Processing The Donut model analyzes: * Visual structure * Text content * User question ### Step 5: Answer Generation The model generates the most relevant answer. ### Step 6: Display Result The answer is shown within the application. --- ## Example Input ### Uploaded Document Invoice image containing: ```text Invoice Number: INV-1025 Date: 15/06/2026 Customer: ABC Technologies Total Amount: $2,450 ``` ### User Question ```text What is the total amount? ``` --- ## Example Output ```text Answer: $2,450 ``` Another example: ### Question ```text Who is the customer? ``` ### Output ```text Answer: ABC Technologies ``` ### Question ```text What is the invoice number? ``` ### Output ```text Answer: INV-1025 ``` These results demonstrate how users can retrieve information directly from document images without manually searching through content. --- ## Use Cases The Document QA App has applications across multiple industries. ### Invoice Processing Automatically extract invoice numbers, dates, and payment details. ### Financial Services Retrieve information from bank statements and financial documents. ### Healthcare Extract patient information from medical forms and reports. ### Legal Industry Search contracts and legal documents using natural language questions. ### Human Resources Analyze employee forms and onboarding documents. ### Education Enable students and researchers to query scanned study materials. ### Government Records Simplify access to information stored in administrative documents. --- ## Future Improvements Several enhancements can further improve the application. ### Multi-Document Support Allow users to upload and query multiple documents simultaneously. ### PDF Support Enable direct PDF uploads without image conversion. ### Answer Highlighting Highlight regions in the document where answers were found. ### Multilingual Support Support document understanding in multiple languages. ### Export Functionality Allow users to download answers and extracted information. ### Batch Processing Process large collections of documents automatically. ### Advanced Analytics Generate summaries and insights from document collections. --- ## Conclusion The Document Question Answering App demonstrates the power of modern Vision-Language Models for intelligent document understanding. By leveraging the Donut model, the application eliminates the need for traditional OCR pipelines while providing accurate and efficient question-answering capabilities. The project showcases how Streamlit and Hugging Face Transformers can be combined to create practical AI-powered applications with minimal infrastructure requirements. Users can upload documents, ask natural language questions, and receive meaningful answers within seconds. As document intelligence continues to evolve, solutions like this will play an increasingly important role in automating information retrieval, improving productivity, and reducing manual effort across industries.