SignTalk

KO | EN

gitlite — search

#privacy #claude #machine-learning #animation #chatbot #nodejs #mcp #bash #self-hosted #claude-code

(★ 9)

Android studio project

File Explorer

ZIP Download

- .gitignore
- AndroidProjectSystem.xml
- compiler.xml
- deploymentTargetSelector.xml
- deviceManager.xml
- gradle.xml
- migrations.xml
- misc.xml
- runConfigurations.xml
- .gitignore
- build.gradle.kts
- proguard-rules.pro
- - gradle-wrapper.jar
  - gradle-wrapper.properties
- libs.versions.toml
.gitignore
build.gradle.kts
gradle.properties
gradlew
gradlew.bat
LICENSE
README.md
settings.gradle.kts

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**An Android assistive communication application that converts hand gestures into readable text using real-time computer vision and on-device machine learning.**

[![Android](https://img.shields.io/badge/Platform-Android-3DDC84?logo=android&logoColor=white)](https://developer.android.com/)
[![Java](https://img.shields.io/badge/Language-Java-ED8B00?logo=openjdk&logoColor=white)](https://www.java.com/)
[![MediaPipe](https://img.shields.io/badge/AI-MediaPipe-4285F4?logo=google&logoColor=white)](https://developers.google.com/mediapipe)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://raw.githubusercontent.com/Ankit-ku-panda/SignTalk/main/LICENSE)

## Overview

SignTalk is an AI-powered Android application designed to make communication more accessible. It uses the device camera to detect hand landmarks in real time and translates recognized gestures into readable text.

The application performs processing directly on the Android device using MediaPipe, helping provide fast responses without relying on a continuous internet connection.

> SignTalk is an academic and experimental assistive-technology project. It is not a certified accessibility or medical communication system.

## Key Features

- Real-time camera preview using CameraX
- Hand landmark detection using MediaPipe
- Gesture-to-text conversion
- Support for forming readable words or messages
- On-device processing
- Offline operation after installation
- Simple and accessible Android interface
- Camera permission handling

## Technology Stack

| Technology | Purpose |
|---|---|
| Java 11 | Android application logic |
| Android SDK | Native Android development |
| XML | User-interface layouts |
| CameraX | Live camera capture and preview |
| MediaPipe Tasks Vision | Hand landmark detection |
| Google ML Kit | Vision and pose-processing support |
| Material Components | Android UI components |
| Gradle Kotlin DSL | Build configuration |

## How It Works

1. The user launches SignTalk and grants camera permission.
2. CameraX provides a live camera feed.
3. MediaPipe analyzes each frame and detects hand landmarks.
4. The application processes the landmark positions.
5. A corresponding gesture or character is displayed as text.
6. Recognized output can be combined into a readable message.

## Project Structure

```text
SignTalk/
├── app/
│   ├── src/main/
│   │   ├── assets/
│   │   │   └── hand_landmarker.task
│   │   ├── java/com/example/signtalk/
│   │   │   ├── CameraActivity.java
│   │   │   ├── EmojiInterpreter.java
│   │   │   └── MainActivity.java
│   │   ├── res/
│   │   │   ├── drawable/
│   │   │   ├── layout/
│   │   │   └── values/
│   │   └── AndroidManifest.xml
│   └── build.gradle.kts
├── gradle/
├── build.gradle.kts
├── settings.gradle.kts
├── LICENSE
└── README.md
```

## Requirements

Before building the project, install:

- Android Studio
- Java Development Kit 11 or newer
- Android SDK 36
- An Android device or emulator running Android 7.0 or newer
- Git

The application currently uses:

- Minimum SDK: 24
- Target SDK: 36
- Compile SDK: 36

## Installation and Setup

### 1. Clone the repository

```bash
git clone https://github.com/Ankit-ku-panda/SignTalk.git
cd SignTalk
```

### 2. Open the project

1. Start Android Studio.
2. Select **Open**.
3. Choose the cloned `SignTalk` directory.
4. Wait for Gradle synchronization to finish.

### 3. Run the application

1. Connect an Android device with USB debugging enabled or start an emulator.
2. Select the device in Android Studio.
3. Click **Run**.
4. Grant camera permission when requested.

A physical Android device is recommended for testing real-time gesture recognition.

## Build from the Command Line

On Windows:

```powershell
.\gradlew.bat assembleDebug
```

On Linux or macOS:

```bash
chmod +x gradlew
./gradlew assembleDebug
```

The generated debug APK should be available at:

```text
app/build/outputs/apk/debug/app-debug.apk
```

## Permissions

SignTalk requires camera access for real-time hand detection:

```xml
<uses-permission android:name="android.permission.CAMERA" />
```

Camera images should be processed locally by the application. Review the source code before using the app with sensitive information.

## Current Limitations

- Recognition accuracy can vary with lighting, camera angle and background.
- Similar hand gestures may be difficult to distinguish.
- Moving or two-handed signs may require additional temporal processing.
- The application is an experimental prototype and may not represent a complete standardized sign-language vocabulary.
- Performance depends on the Android device.

## Roadmap

- Improve recognition accuracy
- Add complete alphabet and word-level recognition
- Support dynamic and two-handed signs
- Add text-to-speech output
- Add sentence suggestions
- Support multiple spoken and sign languages
- Add recognition confidence scores
- Improve accessibility and UI feedback
- Add automated tests and model evaluation

## Contributing

Contributions are welcome.

1. Fork the repository.
2. Create a new branch:

```bash
git checkout -b feature/your-feature-name
```

3. Make and test your changes.
4. Commit the changes:

```bash
git commit -m "Add your feature"
```

5. Push the branch:

```bash
git push origin feature/your-feature-name
```

6. Open a pull request.

For major changes, open an issue first to discuss the proposed implementation.

## Author

**Ankit Kumar Panda**

- GitHub: [@Ankit-ku-panda](https://github.com/Ankit-ku-panda)
- Project: [SignTalk](https://github.com/Ankit-ku-panda/SignTalk)

## License

This project is licensed under the [MIT License](LICENSE).

Third-party libraries and model files included in or used by this project remain subject to their respective licenses and terms.

## Acknowledgements

- [Google MediaPipe](https://developers.google.com/mediapipe)
- [Android Developers](https://developer.android.com/)
- [CameraX](https://developer.android.com/media/camera/camerax)
- [Google ML Kit](https://developers.google.com/ml-kit)
- The open-source accessibility and machine-learning communities

---

If you find SignTalk useful, consider giving the repository a star.

SignTalk

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

SignTalk

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README