Commit Graph

34 Commits

Author SHA1 Message Date
mk
859db651e0 feat: add VAD threshold controls and smooth gate ramp
Replace the hard 0/1 VAD gate with a 20ms ramp in the worklet to prevent
clicks on open/close transitions. Expose positive and negative speech
probability thresholds as user-adjustable settings (defaults 0.5/0.35).
Sliders with restore-defaults button added to the VAD section of the
audio settings tab.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:57:35 -03:00
mk
edd1e1d34e fix: start VAD gate open to avoid permanent silence on model load failure
Starting the gate closed caused permanent silence if the ONNX model or
WASM files failed to load (onFrameProcessed never fired). Gate now starts
open so audio flows immediately; the first silence frame closes it. Also
ensures the gate is always reset to open when VAD is disabled.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:37:36 -03:00
mk
9f5b639190 fix: switch VAD gate to per-frame probability control
onSpeechStart/onSpeechEnd fire at segment boundaries — with constant
non-speech noise, onSpeechEnd never fires so the gate stayed open.
Switch to onFrameProcessed which fires every ~96ms and applies hysteresis
(open at >0.5, close at <0.35) matching Silero's own thresholds. Gate now
starts closed and opens only once the first speech frame is confirmed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:34:08 -03:00
mk
428b76db25 feat: add Silero VAD toggle to audio pipeline
Integrates @ricky0123/vad-web's MicVAD as an optional voice activity detector
alongside the noise gate. When enabled, the Silero ONNX model classifies each
audio frame as speech or silence; silence frames mute the worklet's output via
a new VAD gate message. VAD is wired into Publisher.ts alongside the existing
noise gate transformer. Vite is configured to copy the worklet bundle, ONNX
model, and ORT WASM files to /vad/ so they're reachable at runtime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:29:43 -03:00
mk
411e18c48a feat: add transient suppressor to audio pipeline
Implements a per-sample transient suppressor in the noise gate AudioWorklet
that instantly cuts gain when a sudden loud peak (desk hit, mic bump) exceeds
the slow background RMS by a configurable threshold, then releases over a
short window. Exposes enable, sensitivity, and release controls in the audio
settings tab.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:13:57 -03:00
mk
68d8bb1f92 feat: noise gate implementation 2026-03-23 20:56:58 -03:00
Robin
6b51b7dc58 Split MediaViewModel into multiple files 2026-02-25 22:41:28 +01:00
Timo K
83df152608 review 2026-01-16 13:20:03 +01:00
Timo K
9b40d7420a refactor to use destroy method 2026-01-15 19:04:32 +01:00
Timo K
f71e483080 logging updates 2026-01-15 18:13:34 +01:00
Valere
da55d84bde Add script to check that the tsdoc is correct and up-to-date 2025-12-30 17:02:44 +01:00
Valere Fedronic
6cfa95bcfa Merge pull request #3632 from element-hq/valere/double_pub_problems_highlevel
Fix: Races that could cause double publication or mute state desyncs
2025-12-17 08:50:00 +01:00
Valere
80e760ca55 review 2025-12-16 13:40:06 +01:00
Robin
6149dd2c9a Make the video behavior less confusing
There's no reason to allow it to take on placeholder values. It should be defined when the media has a published video track and undefined when not.
2025-12-15 15:26:42 -05:00
Valere
610af439a8 cleaning: just use LocalTrackPublished event to pause/unpause 2025-12-12 10:37:37 +01:00
Valere
f8310b4611 publisher: only use highlevel participant APIs 2025-12-12 10:31:08 +01:00
Timo K
6efce232f8 fix playwright tests 2025-12-10 18:50:19 +01:00
Timo K
bf801364a6 cleanup and tests 2025-12-09 15:23:30 +01:00
Timo K
2e646bfac1 Unify LiveKit and Matrix connection states 2025-12-02 19:49:13 +01:00
Timo K
47c6a17d1e review 2025-12-01 14:58:06 +01:00
Timo K
66dece98a5 add more test for publisher 2025-11-28 21:50:22 +01:00
Timo K
2011aef116 skip "Should show error screen if call creation is restricted" on ff 2025-11-28 18:00:04 +01:00
Timo K
c0913b6546 fix playwright test 2025-11-27 18:02:46 +01:00
Timo K
46f8fe4ec7 fix test errors 2025-11-27 16:43:28 +01:00
Timo K
e5117f705d More testing and cleaning up 2025-11-27 14:42:23 +01:00
Timo K
d22d7460fe Another larger refactor to fix sfu switches and in general proper
cleanup.
2025-11-25 20:18:34 +01:00
Valere
614762b5f8 fixup headers 2025-11-18 10:13:10 +01:00
Robin
bb746dc8dc Fix a resource leak in the track processor code 2025-11-14 18:45:02 -05:00
Robin
c7f50b53f5 Fix decryption errors
The code had regressed to a state where it was attempting to use one encryption worker for all LiveKit rooms, which does not currently work.
2025-11-12 15:41:41 -05:00
Timo K
85f659bcc9 Introduce MatrixMemberMetadata and use it to simplify username and
avatar computation This removes member from the tiles entirely!
2025-11-11 15:51:48 +01:00
Timo K
b8635b52d8 Working (no local feed) 2025-11-07 19:07:45 +01:00
Timo K
e741285b11 Fix lints, move CallViewModel.test.ts. Fix audio renderer 2025-11-07 14:04:40 +01:00
Timo K
28047217b8 Almost running
- NEVER use undefined as the default for behaviors (FOOTGUN)
2025-11-07 12:33:07 +01:00
Timo K
92fdce33ea pull out all screen share related logic. 2025-11-07 08:44:56 +01:00