smashscan

KO | EN

gitlite — search

#javascript #react #ai #typescript #llm #golang #rust #macos #java #windows #awesome #cli #awesome-list #pytorch

(★ 30)

Analyze SSBM footage with neural networks.

smashscan 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# SmashScan

This project is intended to perform video analysis on pre-recorded footage of the game Super Smash Bros Melee. Once the project has progressed enough, I plan to integrate it with the [SmashVods](http://smashvods.com/) video database.

## TODO List
If you are interested in helping out, join this repository's [Slack](https://join.slack.com/t/smashscan/shared_invite/enQtNDE5MjA5OTI0NDgwLTYwNGNkOWFmZjRjYjkwNDRkNzMzZGJjZjQwZTY5Y2YwZDhmNDJiYzEyZjk1OWJmMmU2YzYzNjRjMTIzYmM2YTI) and send a message.
+ Research ways to detect the percent counter (Either template matching, OCR, or neural networks)
+ Research ways to detect stock icons (Either template matching, feature matching, or neural networks)
+ Research ways to detect the timer (Either template matching, OCR, or neural networks)
+ Add additional game capabilities (SSB64, SSB4, PM, Rivals)

### Milestones
1. [Train a neural network (DarkNet) to search for the six legal SSBM stages, regardless of a stream overlay.](https://medium.com/@seft/smashscan-using-neural-networks-to-analyze-super-smash-bros-melee-a7d0ab5c0755)
2. [Improve video download speed and create a stage-timeline object to represent SSBM matches.](https://medium.com/@seft/smashscan-using-neural-networks-to-analyze-super-smash-bros-melee-part-2-b13ecfbf4e0d)
3. [Added non-stage training data to significantly improve stage-detection results.](https://medium.com/@seft/smashscan-using-neural-networks-to-analyze-super-smash-bros-melee-part-3-e057f07b79ff)
4. [Added percent template matching to segment the video for quicker stage-detection.](https://medium.com/@seft/smashscan-using-neural-networks-to-analyze-super-smash-bros-melee-part-4-78732b46a900)

## Instalation Guide
1. Create a Python 3.6 environment (required by tensorflow) locally or in a conda environment. Install the following python packages; The version numbers are not necessarily required, I just provided them as a reference for when I wrote this guide: numpy (1.11.0), matplotlib (1.5.1), cython (0.28.5), opencv-python (3.4.2.17), youtube-dl (2018.8.22 - 2018 version much faster than 2015-2017).
2. Install tensorflow if you don't have an NVIDIA GPU. Otherwise, install tensorflow-gpu. I followed [Mark Jay's tutorial](https://www.youtube.com/watch?v=vxjbL5iN1XY) for Ubuntu 18.04. 
    * CUDA 9.0 [Download](https://developer.nvidia.com/cuda-toolkit-archive) [Docs](https://docs.nvidia.com/cuda/archive/9.0/)
    * CUDNN 7.4.2 [Download](https://developer.nvidia.com/rdp/cudnn-download) [Docs](https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html)
    * tensorflow-gpu (>=1.9.0) [Download](https://www.tensorflow.org/install/pip) [Docs](https://www.tensorflow.org/install/gpu)
2. Install the [DarkFlow](https://github.com/thtrieu/darkflow) repository globally. I followed [Mark Jay's tutorials](https://www.youtube.com/watch?v=PyjBd7IDYZs&list=PLX-LrBk6h3wSGvuTnxB2Kj358XfctL4BM) to understand the basics of the DarkFlow. 
3. Clone this repo and create an `annotations`, `output`, and `videos` folder inside. The `.gitignore` doesn't like committing "empty" folders.

## Testing Guide

Obtain the YouTube video-id for the video you wish to test on. The video-id is the set of characters after the "v=" in a YouTube video URL. Afterwards, use `download.py` to save the video to the `videos` directory.

``` bash
# https://www.youtube.com/watch?v=dQw4w9WgXcQ
python download.py dQw4w9WgXcQ example-video-name.mp4
```

Use `test.py` to draw bounding boxes on the video. This uses the pre-trained weights (.pb) in the `cfg` folder.

``` bash
python test.py example-video-name.mp4
```

## Training Guide

Obtain the YouTube video-id for the video you wish to train on. The video-id is the set of characters after the "v=" in a YouTube video URL. Afterwards, use `download.py` to save the video to the `videos` directory.

``` bash
# https://www.youtube.com/watch?v=dQw4w9WgXcQ
python download.py dQw4w9WgXcQ example-video-name.mp4
```

Use `label.py` to save frames to the `images` directory and labels to the `annotations` directory. The key-bindings used for `label.py` are listed below.

``` bash
python label.py example-video-name.mp4
```

`,` - move backward 1 frame  
`.` - move forward 1 frame  
`direction` - move 10 frames  
`alt+direction` - move 100 frames  
`ctrl+direction` - move 1000 frames

`left-click -> release` - draw bounding box  
`r` - begin recording once you are satisfied with the box drawn  
`enter` - end recording and specify stage number in terminal  
`` ` `` - exit the labeler and save annotations & frames  
`q` - exit the labeler without saving annotations & frames

Opening the `images` folder freezes my computer because of how many images it tries to display at once. If your running Linux, a useful command to run to see the most recently updated file in a directory is as follows.

```bash
ls -t images | head -1

# Eye of GNOME graphics viewer. Just an image viewer program.
eog 123456.png
```

Once you are satisfied with the new labels, move the `images` and `annotations` folders into the `train` folder of your `darkflow` installation directory. Then run the following two commands in separate terminals.

```bash
# Training command.
flow --model cfg/tiny-yolo-voc-6c.cfg --load bin/tiny-yolo-voc.weights --train --annotation train/annotations --dataset train/images --epoch 100 --gpu 1.0 --summary summary

# Optional command to track the progress visually.
tensorboard --logdir summarytrain
```

After some time, you can cancel training by hitting `ctrl+c`. The DarkFlow trainer saves checkpoints every couple minutes, so you aren't losing much progress. If I plan on training for a long period of time (over 12 hours), I'll typically set the checkpoints to be saved over a longer time span. AKA, I will add the flag `--save 40000`. which will save checkpoints every 2500 iterations (40000/16, where batch size = 16).

This is because DarkFlow has an issue with gradient descent over long periods of time and training will get stuck at NaN. DarkFlow's `default.py` recommends rmsprop, but I tried adam and sgd with no success. It seems to be a common issue with the trainer from the issues I've read on the repository :/

Next, convert the most recent checkpoint into something usable by SmashScan.

``` bash
flow --model cfg/tiny-yolo-voc-6c.cfg --load -1 --savepb
```

The .pb (Protocol Buffer) format is a method for serializing structured data. The DarkFlow library saves the .meta and .pb files to the `built_graph` directory. Copy both of these files into the `cfg` directory of the SmashScan directory.

### Label History
Below are a list of tournaments I trained on for the initial release. The size of these annotation-image pairs is >10GB, so I don't plan on uploading them anytime soon. This list is mainly for referencing what tournaments have been used.

| Tournament | Annotation Range | Frames |
| :--------: | :----------: | :--------------: |
| GOML 2018 | 1 - 5019 | 5019 |
| EVO 2018 | 5020 - 8274 | 3255 |
| Low Tier City 6 | 8275 - 9246 | 972 |
| Smash Factor 7 | 9247 - 12485 | 3239 |
| Saints Gaming Live 2018 | 12486 - 14019 | 1534 |
| Smash N Splash 4 | 14020 - 15720 | 1701 |
| Dreamhack Austin 2018 | 15721 - 17299 | 1579 |
| Combo Breaker 2018 | 17300 - 19091 | 1792 |
| Momocon 2018 | 19092 - 20562 | 1471 |
| Pound Underground 2018 | 20563 - 22747 | 2185 |
| Smash Summit 6 | 22747 - 24852 | 2106 |
| Aegis 2018 | 24853 - 26005 | 1153 |
| Flatiron 3 | 26006 - 27383 | 1378 |
| Noods Oakland | 27384 - 29554 | 2171 |
| The Mango | 29555 - 30976 | 1422 |
| No Fun Allowed 2 | 30977 - 33133 | 2157 |
| The Gang | 33134 - 34861 | 1730 |
| Noods Noods Noods | 34862 - 36760 | 1899 |
| Valhalla | 36761 - 38503 | 1743 |
| Player Cams | 38503 - 41547 | 3044 |
| Holiday Bash 2017 | 41547 - 42628 | 1082 |
| Smash Valley VII | 42629 - 43944 | 1316 |
| Heir 5 | 43945 - 45062 | 1117 |
| Momocon 2018 (R) | 45063 - 46170 | 1108|
| No Fun Allowed 2 (R) | 46171 - 48285 | 2115 |
| Poi Poundaz | 48286 - 49782 | 1497 |

smashscan

smashscan 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

smashscan

smashscan 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README