Projects / Audio ML engineering
← All projectsUnderstanding, generating,
and moving speech.
I built five audio implementations: transcript alignment and ASR scoring, a trained vector-quantized codec prototype, prosody-aware speech generation, streaming voice activity detection and model ablation studies. Each one has a measurable failure mode, which is where I start.
The implementations run locally in Python/NumPy, with offline speech rendering and repeatable tests. The codec results include both bitrate and reconstruction error; the classifiers include held-out clean and noisy scores. Context matters more than a flattering benchmark.
Five implemented projects
From a waveform to a measurable decision.
STT / ASR · WER / CER
Speech-to-Text Robustness & ASR Evaluation
I built a transcript-evaluation pipeline that aligns hypotheses to reference speech, separates substitution/deletion/insertion errors and reports corpus-weighted WER across noise, telephony and code-switching slices.
Python · Dynamic programming · Unicode normalization · JSONL · Unit tests
Implementation & measured behaviorNeural codec · VQ · Compression
Neural Audio Codec & Vector Quantization
I implemented and trained a small nonlinear vector-quantized audio autoencoder with a learned encoder, decoder and discrete codebook, then compared measured reconstruction error to an 8-bit companding baseline.
Python · NumPy · Learned VQ codebook · Straight-through gradients · PCM16 · SNR
Implementation & measured behaviorTTS · Prosody · Speech output
Text-to-Speech & Prosody Pipeline
I built an utterance segmentation and prosody-planning pipeline with per-chunk speaking rates and sentence-type F0 targets, connected to real offline macOS text-to-speech rendering.
Python · Offline macOS TTS · Pitch-contour plans · AIFF · Chunk scheduling
Implementation & measured behaviorReal-time · VAD · Telephony
Streaming Speech Inference & Endpointing
I implemented frame-based streaming speech activity detection with hysteresis and endpoint hangover, then exercised it against synthetic telephony filtering, noise, packet loss and packet-arrival jitter.
Python · NumPy · 20-ms framing · Streaming state machine · Jitter simulation
Implementation & measured behaviorTraining · Ablations · F1
Audio ML Training, Evaluation & Ablations
I trained a small audio-activity classifier with NumPy, built clean and noisy held-out sets, and ran controlled feature and augmentation ablations using repeatable seeds and explicit precision/recall/F1 results.
Python · NumPy · Supervised SGD · Acoustic features · Ablation studies · F1
Implementation & measured behaviorEvaluation before adjectives
I report WER and F1 with their corpus conditions, codec error alongside bitrate, and endpoint delays separately from inference time. A single attractive number is rarely the full story.
The constraints beyond a local experiment
The code establishes concrete algorithm and evaluation contracts. Model-scale claims, multilingual speech accuracy, GPU throughput and telephony performance require different datasets, hardware and measurement conditions; I don't conflate those numbers with local experiments.