open-computer-use - GitLite

KO | EN

gitlite — search

#bash #python #mcp #git #docker #go #sql #shell #android #javascript

open-computer-use

(★ 2,218)

AI computer use powered by open source LLMs and E2B Desktop Sandbox

File Explorer

ZIP Download

- extensions.json
- settings.json
- architecture-light.png
- architecture.png
- - log.html
- browser.py
- config.py
- grounding.py
- llm_provider.py
- logging.py
- osatlas_provider.py
- providers.py
- sandbox_agent.py
- showui_provider.py
- streaming.py
- llm_provider.py
- sandbox_agent.py
- test_screenshot.png
.gitignore
LICENSE
main.py
poetry.lock
pyproject.toml
README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

`brew install`

View Details ▼

Install a Homebrew formula or cask.

brew install {{formula|cask}}

Install a formula/cask:

brew install {{[-s|--build-from-source]}} {{formula}}

Build and install a formula from source (dependencies will still be installed from bottles):

brew install {{[-n|--dry-run]}} {{formula|cask}}

Download the manifest, print what would be installed but don't actually install anything:

🔍

`git clone`

View Details ▼

Clone an existing repository.

git clone {{remote_repository_location}} {{path/to/directory}}

Clone an existing repository into a new directory (the default directory is the repository name):

git clone --recursive {{remote_repository_location}}

Clone an existing repository and its submodules:

git clone {{[-n|--no-checkout]}} {{remote_repository_location}}

Clone only the `.git` directory of an existing repository:

🔍

`poetry install`

View Details ▼

Install all dependencies for a Python project as defined in the pyproject.toml file.

poetry install

Install dependencies:

poetry install --no-root

Skip installing the project itself as a dependency:

poetry install --without dev

Install only production dependencies:

🔍

`poetry run`

View Details ▼

Run a command in the project's virtual environment.

poetry run {{command}}

Run a command inside the virtual environment:

poetry run {{command}} {{argument1 argument2 ...}}

Run a command with arguments:

poetry run {{script_name}}

Run a script defined in `pyproject.toml`:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

A secure cloud Linux computer powered by [E2B Desktop Sandbox](https://github.com/e2b-dev/desktop/) and controlled by open-source LLMs.

https://github.com/user-attachments/assets/3837c4f6-45cb-43f2-9d51-a45f742424d4

## Features

- Uses [E2B](https://e2b.dev/?utm_source=github&utm_medium=referral&utm_campaign=readme&utm_content=open-computer-use) for secure [Desktop Sandbox](https://github.com/e2b-dev/desktop)
- Operates the computer via the keyboard, mouse, and shell commands
- Supports 10+ LLMs, [OS-Atlas](https://osatlas.github.io/)/[ShowUI](https://github.com/showlab/ShowUI) and [any other models you want to integrate](#llm-support)!
- Live streams the display of the sandbox on the client computer
- User can pause and prompt the agent at any time
- Uses Ubuntu, but designed to work with any operating system

## Design

![Open Computer Use Architecture](https://raw.githubusercontent.com/e2b-dev/open-computer-use/master/assets/architecture.png)
![Open Computer Use Architecture](https://raw.githubusercontent.com/e2b-dev/open-computer-use/master/assets/architecture-light.png)

The details of the design are laid out in this article: [How I taught an AI to use a computer](https://blog.jamesmurdza.com/how-i-taught-an-ai-to-use-a-computer)

## LLM support

Open Computer Use is designed to make it easy to swap in and out new LLMs. The LLMs used by the agent are specified in [config.py](/os_computer_use/config.py) like this:

```
grounding_model = providers.OSAtlasProvider()
vision_model = providers.GroqProvider("llama3.2")
action_model = providers.GroqProvider("llama3.3")
```

The providers are imported from [providers.py](/os_computer_use/providers.py) and include:

- Fireworks, OpenRouter, Llama API:
  - Llama 3.2 (vision only), Llama 3.3 (action only)
- Groq:
  - Llama 3.2 (vision + action), Llama 3.3 (action only)
- DeepSeek:
  - DeepSeek (action only)
- Google:
  - Gemini 2.0 Flash (vision + action)
- OpenAI:
  - GPT-4o and GPT-4o mini (vision + action)
- Anthropic:
  - Claude (vision + action)
- HuggingFace Spaces:
  - OS-Atlas (grounding)
  - ShowUI (grounding)
- Moonshot
- Mistral AI (Pixtral for vision, Mistral Large for actions)

If you add a new model or provider, please [make a PR](../../pulls) to this repository with the updated providers.py!

## Get started

### Prerequisites

- Python 3.10 or later
- [git](https://git-scm.com/)
- [E2B API key](https://e2b.dev/dashboard?tab=keys&utm_source=github&utm_medium=referral&utm_campaign=readme&utm_content=open-computer-use)
- API key for an LLM provider (see above)

### 1. Install the prerequisites

In your terminal:

```sh
brew install poetry ffmpeg
```

### 2. Clone the repository

In your terminal:

```sh
git clone https://github.com/e2b-dev/open-computer-use/
```

### 3. Set the environment variables

Enter the project directory:

```
cd open-computer-use
```

Create a `.env` file in `open-computer-use` and set the following:

```sh
# Get your API key here: https://e2b.dev/?utm_source=github&utm_medium=referral&utm_campaign=readme&utm_content=open-computer-use
E2B_API_KEY="your-e2b-api-key"
```

Additionally, add API key(s) for any LLM providers you're using:
```
# You only need the API key for the provider(s) selected in config.py:
# Hugging Face Spaces do not require an API key.
FIREWORKS_API_KEY=...
OPENROUTER_API_KEY=...
LLAMA_API_KEY=...
GROQ_API_KEY=...
GEMINI_API_KEY=...
OPENAI_API_KEY=...
ANTHROPIC_API_KEY=...
MOONSHOT_API_KEY=...
# Required: Provide your Hugging Face token to bypass Gradio rate limits.
HF_TOKEN=...
```

### 4. Start the web interface

Run the following command to start the agent:

```sh
poetry install
```

```sh
poetry run start
```

The agent will open and prompt you for its first instruction.

To start the agent with a specified prompt, run:

```sh
poetry run start --prompt "use the web browser to get the current weather in sf"
```

The display stream should be visible a few seconds after the Python program starts.