Release 01 · A new research group within LAION ·
Humaneness Labs is a new research group within LAION, building the behavior layer for AI. Our first release is Humaneness Ears: open audio encoders that hear emotion, voice, and the small sounds between words — in a single forward pass.
Who we are
Humaneness Labs is a research group within LAION, the non-profit open AI research network. We study the behavior layer for AI: models of attention, emotion, perception, and social context that make interaction feel present.
We work the way LAION has always worked — in the open. Models, code, and datasets are shared so that others can test them, question them, and build on them.
Why listening comes first
Before a machine can respond as if it were present, it has to notice what people actually do: the hesitation before an answer, the laugh that breaks a sentence, the breath that says more than the words.
Voice reveals far more than words. Timing, breath, rhythm, effort, hesitation, and tiny shifts in articulation all expose what is happening beneath the surface. That is why our research starts here.
Humaneness Ears is the perception layer: it turns a recording into a rich, structured description of how something was said. It does not transcribe, caption, or generate speech.
Inside Humaneness Ears
Scroll through what happens between a sound arriving and the model’s answer. The shapes are illustrations of each output; the counts and specifications are exact.
01 · Hear
Ears takes 0.1 to 30 seconds of audio. Stereo is averaged to mono, the signal is resampled to 16 kHz, and a Whisper frontend turns it into 80-bin log-mel features.
02 · Encode
Base builds on a Whisper Base encoder, Medium on a Whisper Small encoder, both fully fine-tuned. There is no text decoder: every transformer layer contributes its own summary, and a learned gate mixes them for each recording.
03 · Feel
Ears predicts the intensity of all 40 EmoNet emotions simultaneously, on a 0–4 reference scale. A voice can be amused and embarrassed at the same time. A separate head rates perceived genuineness from 0 to 6.
04 · Describe
Next to emotion come 57 VoiceNet dimensions of delivery, register, timbre, and style, plus sound-quality and attribute scores: 192 scalars from one pass. Separate heads estimate speaking rate and two speaker-characteristic vectors.
The identity-style vector is not a person’s name or an identification guarantee.
05 · Locate
Every 20 milliseconds, Ears estimates whether a vocal burst is happening. It proposes events with a start and an end, up to 32 per clip, and gives each one its three most likely labels from 53 classes.
All 192 outputs
Each tile is one output. Its brightness is the Pearson correlation between the model’s predictions and held-out test targets. Hover or tap a tile to read it; filter by family or switch models.
Targets are machine annotations — Gemini labels and teacher models on 3,544 held-out clips, fewer for some outputs — not a fresh human panel. Source: the Humaneness Ears model card, which lists every output and metric.
Between the words
Laughter, sighs, gasps, a nervous gulp. Ears marks where these vocal bursts begin and end, and suggests what they are.
Event proposals reach high recall with many false positives, and fine labels are grouped into these 53 classes. Treat the labels as suggestions to listen to, not facts.
Measured, honestly
On public human-label benchmarks, both models agree moderately with people. Medium is slightly ahead on emotion intensity; the two are close elsewhere. Neither is one universal best model.
CREMA-D and RAVDESS use small supervised adapters trained on the model’s 192 scores with actor cross-validation, not zero-shot predictions. The full benchmark report compares every target and baseline.
Open weights · CC BY 4.0
Humaneness Ears Base
Whisper Base encoder · 6 layers · width 512
26.4M parametersHumaneness Ears Medium
Whisper Small encoder · 12 layers · width 768
97.1M parameters# one clip, local CPU
python inference.py clip.wav --model medium --device cpuNew weights, code, and documentation are released under CC BY 4.0; please credit Christoph Schuhmann and LAION. Upstream components keep their own terms: Whisper is MIT, and Orange’s identity teacher is CC BY-SA 3.0.