JALDRISTHI_SIH2026_INTERNAL

KO | EN

gitlite — search

#animation #llm #data-science #redis #self-hosted #linux #web #nlp #python #api

JALDRISTHI_SIH2026_INTERNAL

(★ 9)

Jal Drishti is an AI-assisted platform for monitoring and managing groundwater resources. It combines Digital Water Level Recorder (DWLR) observations, rainfall, and estimated local demand to forecast groundwater conditions, explain changes, identify high-risk locations, and compare possible interventions.

파일 탐색기

ZIP 다운로드

- app.py
- app.py
- create_pilot.py
- feature_engineering.py
- preprocess.py
- rainfall_extract.py
- requirements.txt
- station_quality.py
- train_xgboost.py
- app.py
- prediction.py
- README.md
- requirements.txt
- risk_engine.py
- shap_explanation.py
- train_model.py
- - __init__.py
  - recovery_score.py
- __init__.py
- - __init__.py
  - README.md
  - requirements.txt
.gitignore
_scan.py
MEMBER4_INTEGRATION.md
README.md
requirements.txt
run_integration.py

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

> **AI-powered groundwater monitoring, forecasting, and intervention planning**
> Team **Runtime Terrors** · Smart India Hackathon 2026

[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?logo=python)](https://python.org)
[![Streamlit](https://img.shields.io/badge/Dashboard-Streamlit-FF4B4B?logo=streamlit)](https://streamlit.io)
[![XGBoost](https://img.shields.io/badge/ML-XGBoost-orange)](https://xgboost.readthedocs.io)
[![License: MIT](https://img.shields.io/badge/License-MIT-green)](https://raw.githubusercontent.com/Swapnaneel07/JALDRISTHI_SIH2026_INTERNAL/main/LICENSE)

---

## 🚀 Quick Start (3 Commands)

```bash
# 1. Install all dependencies
pip install -r requirements.txt

# 2. Run the integration pipeline (generates all processed data)
python run_integration.py

# 3. Launch the dashboard
streamlit run dashboard/app.py
```

Open **http://localhost:8501** in your browser.

> **Note:** Place the trained model at `model/xgboost_gw_change_7d.pkl` before running the pipeline.  
> The pilot dataset lives at `trend-analysis/data/buldana_ml_features.csv`.

---

## 🗂️ Repository Structure

```
JALDRISTHI_SIH2026/
│
├── 📊 dashboard/
│   └── app.py                    ← Unified Streamlit dashboard (Member 4)
│
├── 🤖 risk_prediction/           ← XGBoost training + risk engine (Member 1)
│   ├── train_model.py
│   ├── prediction.py
│   ├── risk_engine.py
│   └── shap_explanation.py
│
├── 📈 trend-analysis/            ← Trend + anomaly pipeline (Member 2)
│   ├── member2_trend_anomaly/
│   │   ├── src/                  ← Pipeline, trend, anomaly, visualizations
│   │   ├── tests/                ← 40 pytest tests (all passing)
│   │   └── app/demo.py           ← Member 2 standalone Streamlit demo
│   └── data/
│       └── buldana_ml_features.csv   ← Pilot dataset (12,086 rows · 16 stations)
│
├── 🌧️  src/recovery/             ← Recovery score module (Member 3)
│   └── recovery_score.py
│
├── ⚙️  preprocessing/            ← Data preprocessing + feature engineering (Member 1)
│
├── 🗃️  model/
│   └── xgboost_gw_change_7d.pkl  ← Trained XGBoost model
│
├── 📁 data/processed/            ← Generated by run_integration.py
│   ├── stations.csv              ← Master table (24 columns, 16 stations)
│   ├── prediction_results.csv    ← XGBoost predictions (11,606 rows)
│   ├── trend_results.csv         ← Mann-Kendall + anomaly scores
│   └── recovery_results.csv      ← Rainfall response scores
│
├── 🔗 run_integration.py         ← Central integration script ← START HERE
├── 📋 requirements.txt           ← All dependencies (all 4 members)
└── 📖 README.md                  ← This file
```

---

## 🧩 What Each Member Built

| Member | Module | Key Output |
|--------|--------|-----------|
| **1** | XGBoost forecasting + Risk Engine + SHAP | 7-day GW predictions · Risk Score 0–100 · Feature importance |
| **2** | Trend Analysis + Anomaly Detection | Seasonal Mann-Kendall · Sen's Slope · Anomaly Score 0–100 |
| **3** | Rainfall Recovery Score | Station-level 0–100 recovery efficiency |
| **4** | Streamlit Dashboard + Integration | Full-stack dashboard with map, charts, exports |

---

## 📋 Prerequisites

| Requirement | Version |
|---|---|
| Python | 3.10 or higher |
| Git | Any |
| RAM | 4 GB+ recommended |

---

## 🔧 Detailed Setup Guide

### Step 1 — Clone the repository

```bash
git clone https://github.com/Swapnaneel07/JALDRISTHI_SIH2026_INTERNAL.git
cd JALDRISTHI_SIH2026
```

### Step 2 — (Recommended) Create a virtual environment

```bash
# Windows
python -m venv .venv
.venv\Scripts\activate

# macOS / Linux
python -m venv .venv
source .venv/bin/activate
```

### Step 3 — Install all dependencies

```bash
pip install -r requirements.txt
```

This installs everything needed across all 4 modules:
`pandas · numpy · xgboost · shap · scikit-learn · pymannkendall · plotly · streamlit · folium · streamlit-folium · openpyxl · reportlab · scipy · statsmodels`

### Step 4 — Verify the model file is in place

```
model/
└── xgboost_gw_change_7d.pkl   ← must exist
```

The model file is **not committed to git** (too large). Obtain it from the team shared drive or re-train:

```bash
cd preprocessing
python train_xgboost.py         # re-trains and saves the model
```

Then move the output to `model/xgboost_gw_change_7d.pkl`.

### Step 5 — Run the integration pipeline

```bash
python run_integration.py
```

This single script does everything:

```
[1/5] Loads  trend-analysis/data/buldana_ml_features.csv   (12,086 rows · 16 stations)
[2/5] Runs   Member 2 pipeline  →  trend_results.csv
[3/5] Runs   Member 3 pipeline  →  recovery_results.csv
[4/5] Runs   XGBoost model      →  prediction_results.csv  (11,606 predictions)
[5/5] Builds master             →  stations.csv            (24 columns)
```

Expected output:
```
============================================================
INTEGRATION COMPLETE
============================================================
Output files in data/processed/:
  prediction_results.csv   (1.4 MB)
  recovery_results.csv     (760 B)
  stations.csv             (3.8 KB)
  trend_results.csv        (1.2 KB)

To launch the dashboard:
  streamlit run dashboard/app.py
```

### Step 6 — Launch the unified dashboard

```bash
streamlit run dashboard/app.py
```

Open **http://localhost:8501** → you should see the full JALDRISTHI dashboard.

---

## 🖥️ Dashboard Pages

| Page | What you see |
|---|---|
| **🏠 Overview** | Executive metrics · Interactive Folium risk map · Station summary table |
| **📈 Prediction** | XGBoost 7-day forecast · Risk score + history chart · All-stations priority table |
| **📊 Trend & Anomaly** | Seasonal Mann-Kendall results · Sen's Slope · Anomaly scores · Colour-coded charts |
| **💧 Recovery** | Rainfall-response efficiency · 0–100 Recovery Score per station |
| **⚠️ Risk** | Risk level breakdown · Station prioritisation |
| **💡 Intervention & What-if** | Scenario simulation · Recommended actions |
| **📥 Download Center** | Export as CSV · Excel · PDF |

---

## ⚡ Running Individual Modules

### Member 2 standalone demo
```bash
streamlit run trend-analysis/member2_trend_anomaly/app/demo.py --server.port 8502
```

### Run Member 2 tests
```bash
cd trend-analysis/member2_trend_anomaly
pytest tests/ -v
# Expected: 40 passed
```

### Re-train the XGBoost model
```bash
cd preprocessing
python train_xgboost.py
# Outputs: xgboost_gw_change_7d.pkl + performance_metrics.txt
# Move pkl to model/
```

### Generate SHAP explanations
```bash
cd risk_prediction
python shap_explanation.py
# Outputs: shap_feature_importance.csv + shap_global_importance.png
```

---

## 📊 Key Results (Buldana Pilot)

| Metric | Value |
|---|---|
| DWLR Stations | 16 |
| Observations | 12,086 |
| Date Range | Jul 2023 – Dec 2025 |
| XGBoost R² | 0.91 |
| XGBoost MAE | 0.57 m |
| Trend: Declining | 13 stations |
| Trend: Stable | 3 stations |
| Valid rainfall events (M3) | 281 |
| Highest Recovery Score | Shindi_1 (100.0) |
| Highest Anomaly Score | Jamb_1 (91/100) |

---

## 🏗️ System Architecture

```
                    DWLR + Rainfall Data
                    (buldana_ml_features.csv)
                            │
          ┌─────────────────┼─────────────────┐
          ▼                 ▼                 ▼
      MEMBER 1          MEMBER 2          MEMBER 3
   XGBoost Model     Trend Analysis    Recovery Score
   Risk Engine       Mann-Kendall      Rainfall Events
   SHAP              Anomaly Detect.   0–100 Score
          │                 │                 │
          └─────────────────┼─────────────────┘
                            ▼
                  run_integration.py
                  (data/processed/)
                            │
                            ▼
                       MEMBER 4
                  Streamlit Dashboard
               Maps · Charts · Exports
```

---

## 🌊 How the Pipeline Works

### 1. XGBoost Forecast (Member 1)
Predicts **7-day groundwater level change** using 28 features including GW lags, rolling statistics, rainfall accumulation, and seasonal signals. Outputs `predicted_gw_7d` and a `risk_score` (0–100).

### 2. Trend Analysis (Member 2)
Runs **Seasonal Mann-Kendall test** per station. Reports trend direction (Improving/Declining/Stable), **Sen's slope** in m/year, and a residual-based **anomaly score** (0–100, 95th-percentile threshold).

### 3. Recovery Score (Member 3)
Detects significant rainfall events (≥20 mm), measures groundwater recovery ~7 days after each event, and derives a station-level **0–100 Recovery Score** (relative, min-max normalised across the 16 pilot stations).

### 4. Dashboard (Member 4)
Reads all processed CSVs via `station_id` key. Every page reads live data — no hard-coded values. Supports graceful fallbacks if any module output is missing.

---

## 🔌 Integration Key

All modules share a single join key:

```python
station_id   # e.g. "Ambhasi", "Jamb_1", "Shindi_1"
```

To merge outputs manually:
```python
import pandas as pd

stations   = pd.read_csv("data/processed/stations.csv")
trend      = pd.read_csv("data/processed/trend_results.csv")
recovery   = pd.read_csv("data/processed/recovery_results.csv")
prediction = pd.read_csv("data/processed/prediction_results.csv")

# All joinable on station_id
combined = stations.merge(trend, on="station_id", how="left") \
                   .merge(recovery, on="station_id", how="left")
```

---

## 🤝 Team

| Member | Role | Branch |
|---|---|---|
| **Member 1** | XGBoost Forecasting · Risk Engine · SHAP | `risk-prediction` · `preprocessing` |
| **Member 2** | Trend Analysis · Anomaly Detection | `trend-analysis` |
| **Member 3** | Groundwater Recovery Score | `feature/groundwater-recovery-score` |
| **Member 4** | Streamlit Dashboard · Integration | `member4-dashboard` |

All branches merged into **`main`** via `run_integration.py`.

---

## ❓ Troubleshooting

| Problem | Fix |
|---|---|
| `ModuleNotFoundError: pymannkendall` | `pip install pymannkendall` |
| `ModuleNotFoundError: streamlit_folium` | `pip install streamlit-folium folium` |
| `FileNotFoundError: xgboost_gw_change_7d.pkl` | Place model in `model/` or re-train with `preprocessing/train_xgboost.py` |
| `FileNotFoundError: buldana_ml_features.csv` | Data lives at `trend-analysis/data/buldana_ml_features.csv` |
| Dashboard shows demo data | Run `python run_integration.py` first to generate `data/processed/stations.csv` |
| Port already in use | `streamlit run dashboard/app.py --server.port 8502` |

---

## 📄 License

MIT License — see [LICENSE](LICENSE) for details.

---

<div align="center">
<b>💧 JALDRISTHI · Runtime Terrors · Smart India Hackathon 2026</b><br>
<i>From reactive monitoring to predictive groundwater management</i>
</div>

JALDRISTHI_SIH2026_INTERNAL

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

Download README