Overview
NoNotes was built to reduce the work between hearing useful information and turning it into organized notes. A user records audio in the browser, then the application transcribes the recording, restores punctuation, extracts the most important information, and presents both the full transcription and a summarized set of notes.
I led the backend and AI work, implementing the Flask API, transcription and summarization pipeline, and SQLite persistence that connected the browser experience to saved notes. The team completed a working end-to-end prototype during HappyHacks II, where it earned first place.
How it works
- RecordThe browser captures spoken audio through a simple recording interface.
- TranscribeThe server divides the recording into 10-second segments and submits them for speech recognition.
- RefineAn external punctuation service restores sentence boundaries before summarization.
- SummarizeA BERT-based extractive model identifies key sentences and formats them as point-form notes.
- SaveThe application stores the title, transcription, and summarized notes for later retrieval.
Backend and NLP architecture
I designed the Flask routes that coordinate audio upload, conversion, note creation, retrieval, and deletion. The transcription pipeline segments audio with PyDub, uses Google speech recognition, restores punctuation through the Tartu NLP service, and passes the result to an extractive BERT summarizer configured to retain the most relevant sentences.
Completed notes are stored in SQLite with URL-safe identifiers, allowing the JavaScript frontend to open and manage saved results. I also added application logging and command-line host and port configuration to make the prototype easier to run and troubleshoot during the event.
Trade-offs and next steps
The 48-hour deadline made scope control part of the engineering problem. SQLite provided fast, local persistence; extractive summarization preserved wording from the source transcript; and third-party speech and punctuation services made an end-to-end demo achievable within the event. For a production release, those integrations would require stronger privacy controls, authentication, upload validation, error recovery, and automated testing.
Outcome
The team delivered a working browser-to-notes workflow during the event: record audio, generate a transcript, extract key sentences, and save the result for later viewing. The project was awarded first place at HappyHacks II. For me, the most valuable engineering result was delivering a coherent workflow across audio processing, external NLP services, a Flask API, and persistent storage under a fixed deadline.