where2act - GitLite

KO | EN

gitlite — search

#python #react #ai #golang #deep-learning #linux #nodejs #java #windows #awesome #awesome-list #pytorch #mcp #openai

where2act

(★ 143)

Where2Act: From Pixels to Actions for Articulated 3D Objects

파일 탐색기

최종 버전 다운로드 (.zip)

- - camera.blend
  - cube.obj
  - quaternion.py
  - render_blender.py
  - render_using_blender.py
- - .gitignore
  - README.md
- - model_3d.py
  - model_3d_critic.py
  - model_3d_critic_legacy.py
  - model_3d_legacy.py
- - .gitignore
  - __init__.py
  - quaternion.py
  - README.md
- - .gitignore
  - README.md
- - panda.urdf
  - panda_gripper.urdf
  - panda_robot.py
- - .gitignore
  - run_gen_offline_data.sh
  - run_train_3d.sh
  - run_train_3d_critic.sh
  - run_visu_action_heatmap_proposals.sh
  - run_visu_critic_heatmap.sh
- .gitignore
- camera.py
- checkcollect_data.py
- collect_data.py
- colors.py
- data.py
- datagen.py
- env.py
- gen_html_hierachy_local.py
- gen_offline_data.py
- README.md
- recollect_data.py
- replay_data.py
- requirements.txt
- train_3d.py
- train_3d_critic.py
- utils.py
- visu_action_heatmap_proposals.py
- visu_critic_heatmap.py
- .gitignore
- README.md
- teaser.png
- all_15cats.txt
- ins_cnt_15cats.txt
- ins_cnt_5cats.txt
- test_5cats_data_list.txt
- train_10cats_test_data_list.txt
- train_10cats_train_data_list.txt
.gitignore
README.md

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/daerduoCarey/where2act

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd where2act

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r code/requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

파이썬 스크립트(또는 모듈)를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Where2Act: From Pixels to Actions for Articulated 3D Objects

![Overview](https://raw.githubusercontent.com/daerduoCarey/where2act/main/images/teaser.png)

**The Proposed Where2Act Task.** Given as input an articulated 3D object, we learn to propose the actionable information for different robotic manipulation primitives (e.g. pushing, pulling): (a) the predicted actionability scores over pixels; (b) the proposed interaction trajectories, along with (c) their success likelihoods, for a selected pixel highlighted in red. We show two high-rated proposals (left) and two with lower scores (right) due to interaction orientations and potential robot-object collisions.

## Introduction
One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment. In this paper, we take a step towards that long-term goal -- we extract highly localized actionable information related to elementary actions such as pushing or pulling for articulated objects with movable parts. For example, given a drawer, our network predicts that applying a pulling force on the handle opens the drawer. We propose, discuss, and evaluate novel network architectures that given image and depth data, predict the set of actions possible at each pixel, and the regions over articulated parts that are likely to move under the force. We propose a learning-from-interaction framework with an online data sampling strategy that allows us to train the network in simulation (SAPIEN) and generalizes across categories.

## About the paper

Where2Act is accepted to ICCV 2021!

Our team: 
[Kaichun Mo](https://cs.stanford.edu/~kaichun),
[Leonidas J. Guibas](https://geometry.stanford.edu/member/guibas/),
[Mustafa Mukadam](http://www.mustafamukadam.com/),
[Abhinav Gupta](http://www.cs.cmu.edu/~abhinavg/),
and [Shubham Tulsiani](https://shubhtuls.github.io/)
from 
Stanford University and FaceBook AI Research.

Arxiv Version: https://arxiv.org/abs/2101.02692

Project Page: https://cs.stanford.edu/~kaichun/where2act

## Citations
    
    @inProceedings{mo2021where2act,
        title={Where2Act: From Pixels to Actions for Articulated 3D Objects},
        author={Mo, Kaichun and Guibas, Leonidas and Mukadam, Mustafa and Gupta, Abhinav and Tulsiani, Shubham},
        year={2021},
        booktitle={International Conference on Computer Vision (ICCV)}
    }

## About this repository

This repository provides data and code as follows.

```
    data/                   # contains data, models, results, logs
    code/                   # contains code and scripts
         # please follow `code/README.md` to run the code
    stats/                  # contains helper statistics
```

## Questions

Please post issues for questions and more helps on this Github repo page. We encourage using Github issues instead of sending us emails since your questions may benefit others.

## License

MIT Licence

## Updates

* [Jan 15, 2021] Preliminary version of Data and Code released. For more code on evaluation, stay tuned.
* [Aug 31, 2021] Fixed a bug: https://github.com/daerduoCarey/where2act/blob/main/code/models/model_3d.py#L147-L151. Thanks to Ruihai Wu for pointing this out!