appimages.scraper

KO | EN

gitlite — search

#python #typescript #android #machine-learning #deep-learning #docker #nodejs #java #awesome #cli #vue #pytorch #ios #openai

appimages.scraper

(★ 13)

Search for AppImage releases over the web.

appimages.scraper Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# appimages.scraper
Search for AppImage releases over the web.

### Dependencies
* Python 3.6
* Scrapy

### Run
* Normal run:`scrapy crawl generic.crawler -a project_file=./projects/org.appimage.appimaged.json`
* Output results to json:
`scrapy crawl appimage.github.io -o result.json -t json`

### Input 
The scraper should be feed with a `project_file` which will be a json formatted file like the following:

```
{
  "urls" : ["https://github.com/AppImage/AppImageKit/releases"]
}
```

**Missing fields?**

Sometimes authors doesnt provide good metadata about their project so we could help them by means of preset values. 
Take a look in the following example at the `presets` field and to the `decription` field inside. It will be use
as a fallback value in case that the author forgets to fill that field.

```
{
  "urls" : ["https://github.com/AppImage/AppImageKit/releases"]
  "presets": {
        "id" : "org.appimage.appimaged",
        "description" : {"null": "Daemon to monitor AppImage files in the user home dir."}
  }
}
```

**Multiple applications release in a single page ?**

No problem use the match field. It expects to be a python regex 
that will be used to match the right AppImage download links for the app you are scraping.

```
{
  "urls" : ["https://github.com/AppImage/AppImageKit/releases"],
  "match" : ".*\/appimagetool.*",
  "presets": {
    "id" : "org.appimagekit.appimaged",
    "description" : {"null": "Daemon to monitor AppImage files in the user home dir."}
  }
}
```

appimages.scraper

appimages.scraper Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

appimages.scraper

appimages.scraper Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README