torchgeo-bench

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #rust #windows #awesome #kubernetes #vue #awesome-list #security #mcp #ios #open-source

(★ 25)

No description available.

torchgeo-bench Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# torchgeo-bench

[![CI](https://github.com/torchgeo/torchgeo-bench/actions/workflows/ci.yaml/badge.svg)](https://github.com/torchgeo/torchgeo-bench/actions/workflows/ci.yaml)
[![PyPI version](https://img.shields.io/pypi/v/torchgeo-bench.svg)](https://pypi.org/project/torchgeo-bench/)
[![Python 3.12+](https://img.shields.io/pypi/pyversions/torchgeo-bench.svg)](https://pypi.org/project/torchgeo-bench/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](https://raw.githubusercontent.com/torchgeo/torchgeo-bench/main/LICENSE)

A lightweight benchmarking framework for evaluating **frozen** geospatial
foundation models on GeoBench V1/V2 and location encoders on CoordBench. Plug
in a backbone or coordinate encoder and run consistent downstream probes through
Hydra.

- **Frozen-backbone evaluation** — KNN-5, L-BFGS logistic regression, and
  linear / conv / FPN / DPT segmentation probes.
- **GeoBench V1 + V2 built in** — classification and segmentation, RGB or
  full multispectral / multi-modal stacks.
- **CoordBench built in** — coordinate-only regression and classification with
  random, spatial-block, and official holdouts.
- **Hydra-driven** — sweep models, datasets, partitions, image sizes, and
  bands without code changes.
- **Resumable** — `resume=true` skips already-computed `(dataset, method, model, …)` rows. Atomic CSV appends are safe across parallel jobs.
- **Bring your own model** — copy
  [`contrib_template.py`](src/torchgeo_bench/models/contrib_template.py),
  implement `_forward_patch_features`, and add a one-file Hydra config.
  See the [Stage 1 guide](https://torchgeo.org/torchgeo-bench/user/eval_own_model.html)
  for a full walkthrough, or the
  [Stage 2 guide](https://torchgeo.org/torchgeo-bench/user/contribute_model.html)
  to contribute the model back upstream.

## Installation

```bash
pip install torchgeo-bench
```

For development:

```bash
git clone https://github.com/torchgeo/torchgeo-bench
cd torchgeo-bench
uv sync --extra dev
```

Requires Python 3.12+ and runs on **Linux**, **macOS**, and **Windows**. On
Linux x86_64 the install automatically includes GPU-accelerated FAISS KNN
(CUDA 12, driver R525+, also works on GPU-less machines); other platforms get
CPU FAISS.

## Download a dataset

The runner expects datasets under `./data/`. To grab GeoBench V1:

```bash
torchgeo-bench download geobench_v1
```

To download only the datasets needed for a run:

```bash
torchgeo-bench download geobench_v1 --datasets m-eurosat
```

V2 (classification + segmentation) and torchgeo's EuroSAT downloader work the
same way (`torchgeo-bench download geobench_v2`, `torchgeo-bench download eurosat`). See the [documentation](https://torchgeo.org/torchgeo-bench/user/datasets.html)
for all options.

## Run a basic experiment

```bash
# Default: random convolutional features (RCF) on every available dataset
torchgeo-bench run

# A single dataset with a pretrained ImageNet ResNet-50
torchgeo-bench run model=timm/resnet50 dataset.names=[m-eurosat]
```

The default device is `cuda:0`. On a machine without a working CUDA GPU (or if
a GPU run crashes — see [troubleshooting](https://torchgeo.org/torchgeo-bench/user/troubleshooting.html)),
fall back to CPU:

```bash
torchgeo-bench run dataset.names=[m-eurosat] device=cpu
```

Results are appended to `results/all_results.csv`, which **ships pre-populated
with reference results** — to start from a clean slate, write to your own file
with `output=results/my_run.csv`. Re-run with `resume=true` to skip
already-completed rows.

## CoordBench — location encoders

`mode=coord` swaps the image pipeline for a **coordinate-only** track: point
`(lon, lat)` in, a downstream label out. Benchmarks are streamed directly from
the unified [`taylor-geospatial/coordbench`](https://huggingface.co/datasets/taylor-geospatial/coordbench)
HuggingFace dataset (PDFM, SatCLIP, SustainBench, CDC PLACES, MOSAIKS/USAVars,
the DeepMind/AlphaEarth suite, and more — no local download). A frozen encoder
is probed with **KNN** and a **ridge linear** head under **random** or
**spatial-block** cross-validation (regression → R², classification → accuracy).

```bash
# MIND location encoder on the whole suite, random + spatial CV
torchgeo-bench run mode=coord model=mind coord.split=both

# One family, linear probe only, trivial sin/cos baseline
torchgeo-bench run mode=coord model=sincos coord.names=pdfm coord.methods=[linear]
```

`model=mind` and `model=mind-small` ([MIND](https://huggingface.co/isaaccorley/MIND),
distilled from AlphaEarth/Climplicit/GeoCLIP/SINR) and `model=sincos` work with
the base install. The other pretrained encoders (SatCLIP / GeoCLIP / Climplicit /
SINR, via `rshf`) need the `coordbench` extra:
`pip install "torchgeo-bench[coordbench]"`,
then `model=climplicit` (etc.). Results land in
`results/coordbench_results.csv`. Add your own encoder by subclassing
`LocationEncoder` (implement `_encode`) and pointing a Hydra `model` config's
`_target_` at it. See the
[CoordBench guide](https://torchgeo.org/torchgeo-bench/user/coordbench.html)
and the runnable
[`FourierLocationEncoder` example](https://github.com/torchgeo/torchgeo-bench/blob/main/examples/coordbench_location_encoder.py).

## Learn more

- **[Documentation](https://torchgeo.org/torchgeo-bench/)** — full
  configuration reference, available models, dataset tables, multi-band
  experiments, evaluation methodology, output schema, dev / release
  workflow, and troubleshooting.
- **[AGENTS.md](https://github.com/torchgeo/torchgeo-bench/blob/main/AGENTS.md)**
  — contributor guide and house style.

## Citation

If you use this framework, please cite it (once the `torchgeo-bench` paper is
available):

```bibtex
@misc{torchgeobench,
  title  = {torchgeo-bench: A lightweight benchmarking framework for geospatial foundation models},
  author = {torchgeo-bench Contributors},
  year   = {TBD},
  note   = {Software},
  url    = {https://github.com/torchgeo/torchgeo-bench}
}
```

## License

[MIT](LICENSE).

torchgeo-bench

torchgeo-bench Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

torchgeo-bench

torchgeo-bench Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README