react-native-nitro-realtime-audio

KO | EN

gitlite — search

#javascript #python #ai #typescript #llm #golang #android #linux #nodejs #java #vue #pytorch #security #ios

react-native-nitro-realtime-audio

(★ 9)

🎙️ A high-performance, real-time audio recording & streaming library for React Native powered by Nitro Modules. Emits raw 16-bit PCM ArrayBuffers with ultra-low latency—ideal for AI voice agents, Whisper streaming, and live speech-to-text.

react-native-nitro-realtime-audio 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# react-native-nitro-realtime-audio

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
![React Native](https://img.shields.io/badge/React%20Native-0.76+-61DAFB?logo=react)
![Expo](https://img.shields.io/badge/Expo-Development%20Build-000020?logo=expo)
![Nitro Modules](https://img.shields.io/badge/Nitro-Modules-00C853)

A high-performance real-time audio streaming library for React Native, powered by
[Nitro Modules](https://nitro.margelo.com/).

It provides native **16-bit Linear PCM (PCM16)** recording and playback with audio transferred between JavaScript and the native layer using standard `ArrayBuffer` objects.

The library is designed for real-time audio pipelines where audio needs to be captured, processed, transmitted, received, and played without relying on temporary audio files.

Designed for modern real-time audio applications such as:

- 🤖 AI Voice Assistants (OpenAI Realtime, Gemini Live)
- 🗣️ Speech-to-Text
- 🔊 Text-to-Speech playback
- 🌐 WebSocket audio streaming
- 🌐 WebRTC & VoIP
- 📈 Audio Visualizers
- 🎛️ Digital Signal Processing (DSP)
- 🎤 Voice Activity Detection (VAD)

Unlike traditional recording libraries that primarily save audio files, this library is built for **real-time PCM streaming**.

Audio can be captured from the microphone and streamed directly to JavaScript, while PCM audio received or generated in JavaScript can be streamed back to the native audio player.

Works with **Expo Development Builds**, **EAS Build**, and **React Native CLI** projects.

---

# Features

- ⚡ Powered by Nitro Modules for extremely low-overhead native ↔ JavaScript communication
- 🎙️ Real-time microphone audio streaming
- ▶️ Real-time PCM16 audio playback
- 🌊 Streaming playback for continuously arriving PCM chunks
- 📦 Streams raw 16-bit PCM audio using standard `ArrayBuffer`
- 🔄 Automatic native audio resampling for recording
- 🎚️ Configurable sample rate, channels, and recording chunk duration
- 📦 Native buffering for smooth PCM playback
- 🔄 Supports record → process/network → playback pipelines
- 🔊 Configurable native audio sessions for recording, playback, and duplex communication
- 🎧 Built-in speaker/receiver routing control for duplex voice workflows
- 📞 Optimized for AI voice assistants, VoIP, and real-time conversations
- 📱 Native support for Android and iOS
- 🚀 Compatible with Expo Development Builds and React Native CLI
- 🧩 Ideal for AI, WebRTC, DSP, speech processing, TTS, and custom audio pipelines
- 🗣️ Native Voice Activity Detection (VAD)
- ⚙️ Configurable speech detection threshold and debounce timing
- 🎛️ Native audio processing (Echo Cancellation, Noise Suppression, Automatic Gain Control)
- 📞 Optimized native voice processing for duplex communication

---

# Architecture

The library supports PCM audio in both directions.

## Recording

```text
Microphone
    │
    ▼
Native Audio Recorder
    │
    ▼
Chunk Accumulator
    │
    ▼
PCM16 AudioChunk
    │
    ▼
ArrayBuffer
    │
    ▼
JavaScript Callback
```

## Playback

```text
JavaScript / Network
    │
    ▼
ArrayBuffer
    │
    ▼
Native PCM Buffer
    │
    ▼
Native Audio Player
    │
    ▼
Speaker
```

This makes it possible to build pipelines such as:

```text
Microphone
    │
    ▼
Native Recorder
    │
    ▼
ArrayBuffer
    │
    ▼
JavaScript
    │
    ▼
WebSocket / AI / DSP
    │
    ▼
ArrayBuffer
    │
    ▼
Native Player
    │
    ▼
Speaker
```

---

# Why ArrayBuffer?

The library transfers raw PCM audio as `ArrayBuffer` instead of Base64 strings or temporary audio files.

This provides:

- Lower memory overhead
- Lower latency
- No Base64 encoding/decoding overhead
- Standard JavaScript binary format
- Easy interoperability with AI SDKs
- Easy WebSocket transmission
- Easy DSP processing
- Efficient native ↔ JavaScript audio transfer

For example, recorded audio can be sent directly to a WebSocket:

```ts
onAudioChunk((buffer) => {
  websocket.send(buffer);
});
```

Likewise, PCM received from a service can be passed directly to the player:

```ts
websocket.onmessage = (event) => {
  playChunk(event.data);
};
```

---

# Installation

Install the library together with its required peer dependency:

```bash
npm install @mindinventory/react-native-nitro-realtime-audio react-native-nitro-modules
```

or:

```bash
yarn add @mindinventory/react-native-nitro-realtime-audio react-native-nitro-modules
```

or for Expo:

```bash
npx expo install @mindinventory/react-native-nitro-realtime-audio react-native-nitro-modules
```

> **Note**
>
> `react-native-nitro-modules` is a required peer dependency.

Install iOS dependencies:

```bash
cd ios
pod install
```

---

# Expo Support

This library is compatible with:

- ✅ Expo Development Build
- ✅ Expo Prebuild
- ✅ Expo Bare Workflow
- ✅ React Native CLI
- ✅ EAS Build

> **Important**
>
> Since this library contains native code, it **cannot run inside Expo Go**.
>
> Use an Expo Development Build or EAS Build instead.

---

# iOS Setup

Add the microphone usage description to your **Info.plist**:

```xml
<key>NSMicrophoneUsageDescription</key>
<string>This app requires access to the microphone to record audio.</string>
```

If your application only uses PCM playback and does not use microphone recording, microphone permission is not required for playback itself.

---

# Android Setup

Add microphone permission to:

```text
android/app/src/main/AndroidManifest.xml
```

```xml
<uses-permission android:name="android.permission.RECORD_AUDIO" />
```

This permission is required for microphone recording.

PCM playback itself does not require microphone permission.

---

# API Reference

## Permissions & Device Information

### `getPlatformName(): string`

Returns the current platform.

```ts
'iOS';
'Android';
```

---

### `getNativeSampleRate(): number`

Returns the device's native hardware sample rate.

Example:

```text
48000
44100
```

---

### `getMicrophonePermissionStatus(): MicrophonePermissionStatus`

Returns one of:

```ts
'granted';
'denied';
'undetermined';
```

---

### `requestMicrophonePermission(): Promise<MicrophonePermissionStatus>`

Requests microphone permission from the user.

```ts
const permission = await requestMicrophonePermission();

if (permission === 'granted') {
  // Microphone can be used
}
```

---

# Audio Session

Real-time audio applications often need different native audio configurations depending on whether the application is recording, playing audio, or doing both simultaneously.

Configure the native audio session before starting recording or playback.

```ts
configureAudioSession({
  mode: 'record',
});

configureAudioSession({
  mode: 'playback',
});

configureAudioSession({
  mode: 'duplex',
  speaker: true,
});
```

## `configureAudioSession(config: AudioSessionConfig): void`

Configures the native audio session for the intended audio workflow.

| Property  | Description                                       |
| --------- | ------------------------------------------------- |
| `mode`    | `'record'`, `'playback'`, or `'duplex'`           |
| `speaker` | Optional speaker preference used with duplex mode |

### Choosing the Right Audio Session

| Scenario                       | Recommended Mode |
| ------------------------------ | ---------------- |
| Microphone recording only      | `record`         |
| PCM playback only              | `playback`       |
| Record and play simultaneously | `duplex`         |
| AI voice assistants            | `duplex`         |
| WebRTC / VoIP / voice chat     | `duplex`         |
| TTS / PCM player               | `playback`       |

### Record Mode

Use `record` when the application only needs microphone capture.

```ts
configureAudioSession({
  mode: 'record',
});
```

- ✅ Recording
- ❌ Playback

### Playback Mode

Use `playback` when the application only needs audio playback.

```ts
configureAudioSession({
  mode: 'playback',
});
```

- ❌ Recording
- ✅ Playback

### Duplex Mode

Use `duplex` when recording and playback need to operate together, such as conversational AI, WebRTC, VoIP, or voice-chat applications.

```ts
configureAudioSession({
  mode: 'duplex',
});
```

- ✅ Recording
- ✅ Playback
- ✅ Simultaneous recording and playback

### Speaker Routing

The optional `speaker` property can be used with duplex mode.

```ts
configureAudioSession({
  mode: 'duplex',
  speaker: true,
});
```

| Value   | Behavior                                                                                                      |
| ------- | ------------------------------------------------------------------------------------------------------------- |
| omitted | Uses the platform's default duplex routing behavior                                                           |
| `false` | Does not prefer the loudspeaker; the platform selects the appropriate non-speaker/default communication route |
| `true`  | Prefers the built-in loudspeaker                                                                              |

> Audio routing is ultimately controlled by the operating system. Connected Bluetooth devices, wired headsets, and other available audio routes can affect the final route selected by the platform.

## `deactivateAudioSession(): void`

Deactivates the currently configured native audio session.

```ts
deactivateAudioSession();
```

Call this when the application no longer needs the configured audio session.

---

# Recording

## `startRecording(config: AudioRecordingConfig): void`

Starts recording microphone audio.

Configure a `record` or `duplex` audio session before starting the recorder.

```ts
configureAudioSession({
  mode: 'record',
});

startRecording({
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 100,
});
```

| Property          | Description                                                                     |
| ----------------- | ------------------------------------------------------------------------------- |
| `sampleRate`      | Target output sample rate. Native audio is automatically resampled if required. |
| `channels`        | `1` = Mono, `2` = Stereo                                                        |
| `chunkDurationMs` | Duration of each emitted audio chunk in milliseconds                            |
| `vad`             | Optional Voice Activity Detection (VAD) configuration.                          |
| `processing`      | Optional native audio processing configuration.                                 |

---

## `stopRecording(): void`

Stops microphone recording and stops streaming new audio chunks.

```ts
stopRecording();
```

---

## `isRecording(): boolean`

Returns `true` if the native recorder is actively capturing audio.

```ts
const recording = isRecording();
```

---

## `getCapturedBufferCount(): number`

Returns the number of audio chunks captured during the current recording session.

```ts
const count = getCapturedBufferCount();

console.log('Captured chunks:', count);
```

Useful for debugging and validating streaming behaviour.

---

## `onAudioChunk(callback)`

Registers a callback that receives PCM audio chunks from the native recorder.

```ts
onAudioChunk((buffer: ArrayBuffer) => {
  // buffer contains signed 16-bit PCM samples
});
```

The callback is invoked whenever a chunk is completed by the native recorder.

For example:

```ts
onAudioChunk((buffer) => {
  const samples = new Int16Array(buffer);

console.log('Samples:', samples.length);
});
```

Or send the PCM directly to a real-time service:

```ts
onAudioChunk((buffer) => {
  websocket.send(buffer);
});
```

---

# Voice Activity Detection (VAD)

The library includes an optional native Voice Activity Detection (VAD) engine for detecting when speech starts and stops during recording.

VAD runs entirely on the native layer using RMS (Root Mean Square) audio analysis, avoiding additional JavaScript processing while recording.

## Enabling VAD

```ts
startRecording({
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 100,
  vad: {
    enabled: true,
    threshold: 0.02,
    minSpeechDurationMs: 150,
    minSilenceDurationMs: 500,
  },
});
```

## Configuration

| Property               | Default | Description                                                        |
| ---------------------- | ------- | ------------------------------------------------------------------ |
| `enabled`              | `false` | Enables Voice Activity Detection.                                  |
| `threshold`            | `0.05`  | Normalized RMS threshold used to classify speech.                  |
| `minSpeechDurationMs`  | `150`   | Continuous speech required before a speech start event is emitted. |
| `minSilenceDurationMs` | `500`   | Continuous silence required before a speech end event is emitted.  |

## Listening for Voice Activity

```ts
onVoiceActivity((event) => {
  console.log(event.isSpeaking);
  console.log(event.rms);
});
```

`VoiceActivityEvent`

```ts
{
  isSpeaking: boolean;
  rms: number;
}
```

### Understanding RMS

The reported RMS value is normalized between **0.0** and **1.0**.

| RMS         | Typical Meaning         |
| ----------- | ----------------------- |
| 0.00 – 0.02 | Silence / ambient noise |
| 0.02 – 0.06 | Quiet speech            |
| 0.06 – 0.20 | Normal speech           |
| > 0.20      | Loud speech             |

These values are approximate and may vary depending on the microphone and environment.

### Event Behavior

`onVoiceActivity()` is event-driven.

Unlike `onAudioChunk()`, it **does not fire for every audio chunk**.

Instead, events are emitted only when the speaking state changes:

- Speech started
- Speech stopped

This makes it suitable for AI assistants, speech recognition, push-to-talk interfaces, and other real-time voice applications without generating unnecessary JavaScript events.

---

# Native Audio Processing

The library optionally enables the operating system's built-in audio processing pipeline for real-time communication scenarios such as:

- 🤖 AI Voice Assistants
- 📞 Voice Chat
- 🌐 WebRTC
- ☎️ VoIP
- 🎙️ Speakerphone conversations

Native audio processing can improve microphone quality by reducing echo, suppressing background noise, and automatically adjusting microphone gain.

## Enabling Audio Processing

```ts
startRecording({
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 100,

processing: {
    echoCancellation: true,
    noiseSuppression: true,
    autoGainControl: true,
  },
});
```

## Configuration

| Property           | Default | Description                                         |
| ------------------ | ------- | --------------------------------------------------- |
| `echoCancellation` | `false` | Removes speaker playback from the microphone input. |
| `noiseSuppression` | `false` | Reduces stationary background noise.                |
| `autoGainControl`  | `false` | Automatically adjusts microphone input gain.        |

## Platform Behavior

### Android

Android enables each requested audio effect independently using the platform AudioEffects API.

- Acoustic Echo Canceler (AEC)
- Noise Suppressor (NS)
- Automatic Gain Control (AGC)

Support depends on the device hardware and Android version.

### iOS

iOS provides these features through Apple's native Voice Processing audio pipeline.

When `echoCancellation` is enabled, the library configures the audio session for voice communication and enables native voice processing.

Apple's voice processing pipeline includes:

- Acoustic Echo Cancellation
- Noise Suppression
- Automatic Gain Control

These effects are managed by the operating system and cannot be enabled or configured individually.

---

# Playback

The library supports native playback of raw **16-bit Linear PCM (PCM16)** audio.

Playback can be used for:

- AI voice responses
- Text-to-Speech audio
- WebSocket PCM streams
- Real-time voice applications
- Recorded PCM playback
- Custom audio processing pipelines

---

## `initializePlayer(config: AudioPlaybackConfig): void`

Initializes the native PCM audio player.

Call this before sending PCM chunks for playback.

```ts
initializePlayer({
  sampleRate: 24000,
  channels: 1,
  bufferSize: 4096,
});
```

| Property     | Description                                     |
| ------------ | ----------------------------------------------- |
| `sampleRate` | Sample rate of the PCM data that will be played |
| `channels`   | `1` = Mono, `2` = Stereo                        |
| `bufferSize` | Playback buffer configuration                   |

> **Important**
>
> The playback configuration must match the format of the PCM data being provided.
>
> For example, if your PCM stream is 24 kHz mono PCM16, initialize the player with `sampleRate: 24000` and `channels: 1`.

---

## `playChunk(buffer: ArrayBuffer): void`

Sends a PCM16 audio chunk to the native player.

```ts
playChunk(buffer);
```

Chunks can be sent continuously as they become available.

For example:

```ts
onPcmChunk((buffer) => {
  playChunk(buffer);
});
```

Or when receiving audio from a WebSocket:

```ts
websocket.onmessage = (event) => {
  playChunk(event.data);
};
```

PCM chunks are buffered on the native side and consumed by the native audio player.

JavaScript does not need to schedule every chunk according to the hardware playback clock.

---

## `finishPlayback(): void`

Signals that no more PCM chunks will be provided for the current playback stream.

```ts
finishPlayback();
```

Use this when the source stream has ended naturally.

For example:

```ts
for (const chunk of pcmChunks) {
  playChunk(chunk);
}

finishPlayback();
```

`finishPlayback()` allows already-buffered audio to complete playback.

It is different from `stopPlayback()`, which is used to stop the current playback.

---

## `stopPlayback(): void`

Stops the current playback.

```ts
stopPlayback();
```

Use this when playback needs to be cancelled or stopped by the application.

---

## `releasePlayer(): void`

Releases the native player resources.

```ts
releasePlayer();
```

Call this when the player is no longer needed.

For example:

```ts
useEffect(() => {
  initializePlayer({
    sampleRate: 24000,
    channels: 1,
    bufferSize: 4096,
  });

return () => {
    releasePlayer();
  };
}, []);
```

---

# PCMStreamer

`PCMStreamer` is a convenience helper for feeding PCM16 audio to the native player.

```ts
import {
  NitroRealtimeAudio,
  PCMStreamer,
} from '@mindinventory/react-native-nitro-realtime-audio';

const pcmStreamer = new PCMStreamer(NitroRealtimeAudio, {
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 20,
});
```

Send PCM chunks:

```ts
pcmStreamer.enqueue(chunk);
```

When no more chunks will arrive:

```ts
pcmStreamer.finish();
```

Stop playback:

```ts
pcmStreamer.stop();
```

You can also provide a complete PCM `ArrayBuffer`:

```ts
pcmStreamer.play(pcmBuffer);
```

The buffer is split into chunks based on the configured sample rate, channels, and chunk duration before being forwarded to the native player.

---

# Streaming Playback Example

A common use case is receiving PCM audio continuously from a WebSocket or AI service.

```ts
import {
  initializePlayer,
  playChunk,
  finishPlayback,
  releasePlayer,
} from '@mindinventory/react-native-nitro-realtime-audio';

initializePlayer({
  sampleRate: 24000,
  channels: 1,
  bufferSize: 4096,
});

websocket.onmessage = (event) => {
  const pcmBuffer = event.data;

playChunk(pcmBuffer);
};

websocket.onclose = () => {
  finishPlayback();
};

// When the player is no longer needed
releasePlayer();
```

The incoming chunks do not need to arrive exactly every 20 ms.

The native playback layer buffers incoming PCM and handles playback according to the native audio output.

---

# Record and Play Example

The following example captures PCM chunks from the microphone and plays them back after recording stops.

```tsx
import React, { useEffect, useRef, useState } from 'react';

import { View, Text, Button } from 'react-native';

import {
  getMicrophonePermissionStatus,
  requestMicrophonePermission,
  startRecording,
  stopRecording,
  isRecording,
  onAudioChunk,
  initializePlayer,
  playChunk,
  finishPlayback,
  stopPlayback,
  releasePlayer,
  configureAudioSession,
  deactivateAudioSession,
} from '@mindinventory/react-native-nitro-realtime-audio';

export default function AudioDemo() {
  const [recording, setRecording] = useState(false);
  const [chunksReceived, setChunksReceived] = useState(0);

const recordedPcmChunks = useRef<ArrayBuffer[]>([]);

useEffect(() => {
    onAudioChunk((buffer) => {
      recordedPcmChunks.current.push(buffer.slice(0));

setChunksReceived((count) => count + 1);
    });

return () => {
      stopPlayback();
      releasePlayer();
      deactivateAudioSession();
    };
  }, []);

const start = async () => {
    let permission = getMicrophonePermissionStatus();

if (permission !== 'granted') {
      permission = await requestMicrophonePermission();

if (permission !== 'granted') {
        return;
      }
    }

recordedPcmChunks.current = [];
    setChunksReceived(0);

configureAudioSession({
      mode: 'record',
    });

startRecording({
      sampleRate: 24000,
      channels: 1,
      chunkDurationMs: 100,
    });

setRecording(isRecording());
  };

const stop = () => {
    stopRecording();

setRecording(isRecording());

console.log(`Captured ${recordedPcmChunks.current.length} chunks`);
  };

const play = () => {
    stopPlayback();
    releasePlayer();
    deactivateAudioSession();

configureAudioSession({
      mode: 'playback',
    });

initializePlayer({
      sampleRate: 24000,
      channels: 1,
      bufferSize: 4096,
    });

for (const chunk of recordedPcmChunks.current) {
      playChunk(chunk);
    }

finishPlayback();
  };

return (
    <View>
      <Text>{recording ? 'Recording...' : 'Stopped'}</Text>

<Text>Chunks Received: {chunksReceived}</Text>

<Button title="Stop Playback" onPress={stopPlayback} />
    </View>
  );
}
```

---

# Duplex Streaming Example

Duplex mode is intended for applications that capture microphone PCM while also playing incoming PCM, such as AI voice assistants, WebRTC, VoIP, and real-time conversations.

```ts
configureAudioSession({
  mode: 'duplex',
  speaker: true,
});

initializePlayer({
  sampleRate: 24000,
  channels: 1,
  bufferSize: 4096,
});

onAudioChunk((buffer) => {
  websocket.send(buffer);
});

startRecording({
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 100,

processing: {
    echoCancellation: true,
    noiseSuppression: true,
    autoGainControl: true,
  },
});

websocket.onmessage = (event) => {
  playChunk(event.data);
};

// When the remote playback stream ends:
finishPlayback();

// When the audio experience is finished:
// stopRecording();
// stopPlayback();
// releasePlayer();
// deactivateAudioSession();
```

Use `speaker: true` when the application should prefer loudspeaker output during duplex communication. Omit it or use `false` when loudspeaker output should not be preferred.

---

# Recommended Configurations

## Audio Session

| Use Case                   | Mode       | Speaker  |
| -------------------------- | ---------- | -------- |
| Microphone capture only    | `record`   | —        |
| PCM / TTS playback only    | `playback` | —        |
| Conversational AI          | `duplex`   | Optional |
| WebRTC / VoIP / voice chat | `duplex`   | Optional |

Use `duplex` when recording and playback must remain available at the same time.

## Recording

| Use Case          | Sample Rate | Channels | Chunk Duration |
| ----------------- | ----------: | -------: | -------------: |
| Voice AI          |       24000 |     Mono |         100 ms |
| Whisper           |       16000 |     Mono |         100 ms |
| WebRTC            |       48000 |     Mono |          20 ms |
| General Recording |      Native |   Stereo |         100 ms |

The exact configuration should ultimately match the requirements of the service or audio pipeline consuming the PCM data.

## Playback

For playback, configure the player to match the incoming PCM format.

For example, for 24 kHz mono PCM16:

```ts
initializePlayer({
  sampleRate: 24000,
  channels: 1,
  bufferSize: 4096,
});
```

Do not initialize the player with a different sample rate or channel count than the PCM stream unless your application performs the required conversion.

---

# PCM Format

Audio chunks use:

```text
Encoding:     Linear PCM
Sample type:  Signed Int16
Bit depth:    16-bit
Byte order:   Little-endian
Transport:    ArrayBuffer
```

The number of bytes represented by a PCM duration can be calculated using:

```text
bytes =
  sampleRate
  × channels
  × bytesPerSample
  × durationSeconds
```

For PCM16:

```text
bytesPerSample = 2
```

For example, 20 ms of 24 kHz mono PCM16 contains:

```text
24000 × 1 × 2 × 0.020
= 960 bytes
```

---

# Example Application

The example application included in this repository demonstrates:

- 🎙️ Real-time microphone recording
- 📈 Live waveform visualization
- 📊 Live chunk statistics
- 📦 Raw PCM streaming
- 💾 WAV generation
- ▶️ PCM audio playback
- 🌊 Streaming PCM playback
- 🔄 Record → playback workflows
- 📋 Recording summaries
- 🎤 Voice Activity Detection (VAD)

It serves as both a demo application and a reference implementation for integrating the library.

---

# Notes

- Audio chunks contain **signed 16-bit Linear PCM** samples.
- Configure the appropriate native audio session before starting recording or playback.
- Use `duplex` mode for simultaneous recording and playback.
- Audio routing is ultimately controlled by the operating system and available audio devices.
- Samples are stored in **little-endian** format.
- Audio is transferred using standard JavaScript `ArrayBuffer`.
- Recording **does not automatically save audio files**.
- Generate WAV files or encode MP3/AAC yourself using the streamed PCM data if file output is required.
- Recording chunk size depends on the configured sample rate, channel count, and chunk duration.
- Playback expects PCM16 data.
- Playback sample rate and channel count should match the incoming PCM stream.
- Native playback buffering is used to handle PCM chunks before they are played.
- `finishPlayback()` should be used when a playback stream has naturally finished.
- `stopPlayback()` should be used when playback needs to be stopped.
- `releasePlayer()` should be used when native playback resources are no longer needed.
- Voice Activity Detection is optional and disabled by default.
- VAD processing occurs entirely on the native layer.
- `onVoiceActivity()` emits events only when speech starts or stops.
- Native audio processing relies on operating system capabilities.
- Audio effect availability varies by Android device.
- On iOS, Echo Cancellation, Noise Suppression, and Automatic Gain Control are provided together by Apple's Voice Processing pipeline.
- The library does not implement custom DSP algorithms for these audio effects.

---

# Best Practices

## Configure the audio session first

Configure the native audio session for the workflow before starting the recorder or initializing the player.

Recording only:

```ts
configureAudioSession({
  mode: 'record',
});

startRecording({
  sampleRate: 24000,
  channels: 1,
  chunkDurationMs: 100,
});
```

Playback only:

```ts
configureAudioSession({
  mode: 'playback',
});

initializePlayer({
  sampleRate: 24000,
  channels: 1,
  bufferSize: 4096,
});
```

Simultaneous recording and playback:

```ts
configureAudioSession({
  mode: 'duplex',
  speaker: true,
});
```

When the audio workflow is finished, stop/release the active recorder or player resources and deactivate the session when it is no longer needed.

---

## Process recording chunks immediately

The example application may store PCM chunks in memory so it can replay audio or generate a WAV file after recording.

For production applications such as AI assistants, speech recognition, WebRTC, or long-running streams, process each chunk immediately instead of storing every chunk in memory.

Example:

```ts
onAudioChunk((buffer) => {
  websocket.send(buffer);
});
```

Streaming chunks immediately keeps memory usage low even during long recording sessions.

---

## Stream playback chunks as they arrive

For live audio, send incoming PCM chunks to the player as soon as they are available.

```ts
websocket.onmessage = (event) => {
  playChunk(event.data);
};
```

Do not accumulate an entire response in JavaScript before starting playback unless your application specifically requires offline playback.

---

## Keep recording and playback formats consistent

When audio passes through an external service, make sure you know the format returned by that service.

For example:

```text
Recording

24000 Hz
Mono
PCM16

↓

AI / WebSocket

↓

Playback

24000 Hz
Mono
PCM16
```

Initialize playback using the format of the **incoming playback stream**, which may not always be the same as the recording format.

---

## Finish streams properly

When the source has finished producing PCM:

```ts
finishPlayback();
```

This indicates that no more chunks are expected for the current playback stream.

When playback needs to be cancelled:

```ts
stopPlayback();
```

---

## Release playback resources

When the player is no longer needed:

```ts
releasePlayer();
```

For React components, this can typically be performed during cleanup:

```ts
useEffect(() => {
  initializePlayer({
    sampleRate: 24000,
    channels: 1,
    bufferSize: 4096,
  });

return () => {
    releasePlayer();
  };
}, []);
```

## Enable audio processing only when needed

Audio processing is intended for real-time communication where microphone input and speaker playback occur simultaneously.

Typical use cases include:

- AI voice assistants
- Voice chat
- WebRTC
- VoIP
- Speakerphone conversations

For high-fidelity recording, music capture, or audio analysis, leave audio processing disabled.

---

# Roadmap

- ✅ Real-time PCM recording
- ✅ Configurable recording
- ✅ Native audio resampling
- ✅ Real-time PCM streaming
- ✅ PCM16 playback
- ✅ Streaming PCM playback
- ✅ Native audio session management
- ✅ Voice Activity Detection (VAD)
- ✅ Native Audio Processing
- ✅ Acoustic Echo Cancellation (AEC)
- ✅ Noise Suppression (NS)
- ✅ Automatic Gain Control (AGC)

---

# Contributing

Contributions, bug reports, and feature requests are welcome.

Please read the [Contributing Guide](CONTRIBUTING.md) before opening a pull request.

---

# License

MIT

---

# Let us know

If you use our open-source libraries in your project, please make sure to credit us and give a star to www.mindinventory.com.

<p>
<h4>Please feel free to use this component and let us know if you are interested in building apps or designing products.</h4>

---

Built with ❤️ using **React Native Builder Bob (create-react-native-library)**.

react-native-nitro-realtime-audio

react-native-nitro-realtime-audio 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

react-native-nitro-realtime-audio

react-native-nitro-realtime-audio 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README