sketchy - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #go #android #machine-learning #deep-learning #awesome #vue #pytorch #mcp #ios #open-source

sketchy

(★ 999)

A task based API for taking screenshots and scraping text from websites.

파일 탐색기

최종 버전 다운로드 (.zip)

- - capture.js
  - lodash.js
  - static.js
- - __init__.py
  - controller.py
  - static_upload.py
  - tasks.py
  - validators.py
- - __init__.py
  - capture.py
  - static.py
- __init__.py
- loggers.py
- supervisord.ini
.gitignore
AUTHORS
config-default.py
config-test.py
dev_requirements.txt
Dockerfile
fig.yml
LICENSE
manage.py
OSSMETADATA
README.md
setup-nflx.py
setup.py
tests.py
ubuntu_install.sh

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/Netflix-Skunkworks/sketchy

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd sketchy

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Docker

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.

docker build -t sketchy .

Dockerfile을 기반으로 실행 가능한 이미지를 빌드합니다.

docker run -p 8080:80 sketchy

빌드된 이미지를 실제 컨테이너로 실행합니다.

✅ 터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install .

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

python manage.py runserver

장고(Django) 개발 서버를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ----DEPRECATED----
![Sketchy](http://i.imgur.com/WvGJ8Ri.jpg)
# Overview
## What is Sketchy?

Sketchy is a task based API for taking screenshots and scraping text from websites.

## What is the Output of Sketchy?

Sketchy's capture model contains all of the information associated with screenshotting, scraping, and storing html files from a provided URL. Screenshots (sketches), text scrapes, and html files can either be stored locally or on an S3 bucket. Optionally, token auth can be configured for creating and retrieving captures. Sketchy can also perform callbacks if required.

## How Does Sketchy Do It?

Sketchy utilizes PhantomJS with [lazy-rendering](https://github.com/kimmobrunfeldt/url-to-image) to ensure Ajax heavy sites are captured correctly. Sketchy also utilizes [Celery](http://www.celeryproject.org/) task management system allowing users to scale Sketchy accordingly and manage time intensive captures.

## Release History ##

**Version 1.1.2** - *January 27, 2016*

This minor release addresses a bug and a new configuration option:

- A default timeout of 5 seconds was added to check_url task.  This should prevent workers from hanging [#26](https://github.com/Netflix/sketchy/issues/26).
- You can now specify a cookie store via an environment variable 'phantomjs_cookies' which will be used by PhantomJS.  This env variable simply needs to be a string of key/value cookie pairs.

**Version 1.1.1** - *June 16, 2015*

This minor release addresses a few bugs and some new configuration features:

- A new configuration option `PHANTOMJS_TIMEOUT` allows setting how long to wait for a capture to render before terminating the subprocess
- Celery retry functionality was added when PhantomJS fails to render a screenshot before the PhantomJS timeout occurs
- An incremental PhantomJS timeout was introduced to improve PhantomJS success at generating very large screenshots.  Each time PhantomJS retries to render a screenshot 5 seconds will be added to the previous `PHANTOMJS_TIMEOUT` configuration option.
- A number of typos have been fixed and comments have been added.

**Version 1.1** - *December 4, 2014*

A number of improvements and bug fixes have been made:

- A new model and API endpoint called "Static" was created.  This allows users to send Sketchy a static HTML file for text scraping and screenshotting.  See the [Wiki](https://github.com/Netflix/Sketchy/wiki) for usage information.
- New PhantomJS script called 'static.js' for creating screenshots of static html files.
- Creation of a new endpoint: api/v1.0/capture/last which shows the last capture that was taken.
- Creation of a new endpoint: api/v1.0/static/last which shows the last static capture that was taken.
- API list view is now reverse sorted so most recent capture is listed on the top of the page.
- For callback requests, capture status is now updated
- Task retry has been optimized to only retry on ConnectionErrors.  This should speedup errors that would never succeed during a retry.
- A new configuration setting "SSL\_HOST\_VALIDATION" can be set to scrape/screenshot webpages with SSL errors.
- A new configuration setting "CAPTURE_ERRORS" can be used to scrape/screenshot webpages that have 4xx or 5xx http status codes.

# Documentation #

Documentation is maintained in the Github [Wiki](https://github.com/Netflix/Sketchy/wiki)

# Docker #
Sketchy is also available as a [Docker](https://github.com/sbehrens/docker_sketchy) container.