JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform

KO | EN

gitlite — search

#javascript #hacktoberfest #react #llm #go #android #machine-learning #rust #docker #nodejs #java #vue #ios #open-source

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform

(★ 10)

No description available.

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 💼 JobPulse AI
### *An Intelligent Job Market Analytics & Machine Learning Platform*

[![Python](https://img.shields.io/badge/Python-3.12%2B-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.110%2B-009688?style=for-the-badge&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
[![React](https://img.shields.io/badge/React-19.0-61DAFB?style=for-the-badge&logo=react&logoColor=black)](https://react.dev/)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.0%2B-3178C6?style=for-the-badge&logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
[![Tailwind CSS](https://img.shields.io/badge/Tailwind_CSS-v4.0-06B6D4?style=for-the-badge&logo=tailwindcss&logoColor=white)](https://tailwindcss.com/)
[![Plotly](https://img.shields.io/badge/Plotly.js-Interactive_Charts-3F4F75?style=for-the-badge&logo=plotly&logoColor=white)](https://plotly.com/)
[![License](https://img.shields.io/badge/License-MIT-green.svg?style=for-the-badge)](https://raw.githubusercontent.com/Omkar-narsale/JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform/main/LICENSE)

<p align="center">
  <a href="#-key-features">Key Features</a> •
  <a href="#%EF%B8%8F-system-architecture">System Architecture</a> •
  <a href="#-machine-learning--data-mining">Machine Learning</a> •
  <a href="#-rest-api-documentation">API Docs</a> •
  <a href="#-quick-start-guide">Quick Start</a> •
  <a href="#-dashboard-pages">UI Pages</a>
</p>

---

</div>

## 📌 Executive Summary

**JobPulse AI** is an end-to-end, enterprise-grade job market analytics platform powered by **Web Mining**, **Data Engineering ETL Pipelines**, **Machine Learning Intelligence**, and a high-performance **Glassmorphic React Analytics Dashboard**.

It extracts raw tech career listings from web job portals, transforms messy listings into clean structured entities, performs **K-Means clustering** and **Isolation Forest anomaly detection**, and provides an **AI Candidate Matcher & Skill Gap Analysis engine**.

---

## 🌟 Key Features

| Feature Component | Technical Highlights |
| :--- | :--- |
| 🕷️ **Automated Web Mining** | Multi-source scraping engine (`BeautifulSoup4`, `Requests`, `Selenium`) with custom header rotation, rate limiting, and automatic synthetic fallback. |
| 🧹 **Multi-Stage Data Engineering ETL** | Robust cleaning pipeline featuring HTML sanitization, regex salary range parsing, experience level extraction, and taxonomy matching. |
| 🗄️ **Relational Data Storage** | SQLAlchemy 2.0 ORM backed by MySQL / SQLite database schemas optimized for high-speed analytical queries and indexing. |
| 🤖 **K-Means Clustering** | Unsupervised ML segmentation grouping job market postings into distinct compensation vs. experience tiers. |
| 🛡️ **Isolation Forest Anomaly Engine** | Detects statistical market outliers, ghost jobs, non-standard salary listings, and suspicious recruitment patterns. |
| 🎯 **AI Job Matcher & Skill Gap Analysis** | TF-IDF & Cosine Similarity vector engine matching candidate profiles against job listings, generating precise match scores and missing skill recommendations. |
| 📊 **Interactive Plotly Visualizations** | Real-time interactive charts (Scatter plots, Treemaps, Heatmaps, Sunburst diagrams, Histograms, and Box plots). |
| 💎 **Glassmorphic Dark Mode UI** | Modern SaaS dashboard designed with React 19, TypeScript, Tailwind CSS, TanStack Query v5, and Framer Motion. |

---

## 🏗️ System Architecture

```
                                  [ WEB SOURCES ]
                       (RemoteOK Portal / Synthetic Engine)
                                        │
                                        ▼
                             [ SCRAPER ENGINE ]
                       (Rate Limiter, Retries, Raw CSV)
                                        │
                                        ▼
                         [ MULTI-STAGE ETL PIPELINE ]
                (Text Sanitizer, Salary & Exp Regex, Skill Parser)
                                        │
                                        ▼
                       [ SQLALCHEMY ORM & DATABASE ]
                     (SQLite / MySQL Relational Schema)
                                        │
             ┌──────────────────────────┼──────────────────────────┐
             ▼                          ▼                          ▼
   [ K-MEANS CLUSTERING ]    [ ISOLATION FOREST ANOMALIES ]   [ TF-IDF COSINE MATCH ]
  (Salary vs Experience Tiers)   (Outlier/Scam Detection)   (Skill Gap Matrix Generator)
             └──────────────────────────┬──────────────────────────┘
                                        │
                                        ▼
                                [ FASTAPI BACKEND ]
                         (REST Endpoints, Swagger Docs)
                                        │
                                        ▼
                           [ REACT 19 DASHBOARD ]
                     (Plotly, TanStack Query, Glass UI)
```

---

## 🤖 Machine Learning & Data Mining

<details>
<summary><b>🔍 1. K-Means Salary & Experience Clustering (Click to expand)</b></summary>

- **Objective**: Segments the job market into distinct career compensation tiers.
- **Features**: Scaled `min_salary`, `max_salary`, `avg_salary`, `min_experience`, and skill count vector.
- **Visualization**: Interactive 2D scatter plots mapping salary against experience with color-coded cluster overlays.
</details>

<details>
<summary><b>🛡️ 2. Isolation Forest Anomaly Detection (Click to expand)</b></summary>

- **Objective**: Identifies statistical outliers, suspicious salary figures, or skewed job descriptions.
- **Algorithm**: `sklearn.ensemble.IsolationForest` with configurable contamination parameters.
- **Output**: Anomaly decision scores (-1 for outliers, +1 for inliers) presented on dedicated threat analysis views.
</details>

<details>
<summary><b>🎯 3. Cosine Similarity Job Recommendation & Skill Gap Analysis (Click to expand)</b></summary>

- **Objective**: Computes semantic compatibility between candidate resumes and active job postings.
- **Pipeline**: TF-IDF vectorization across job titles, required skills, and descriptions.
- **Skill Gap Generator**: Identifies missing high-impact tech skills required to elevate match confidence.
</details>

---

## 📂 Project Directory Structure

```
jobpulse-ai/
├── backend/                        # FastAPI Application & ML Subsystem
│   ├── app/
│   │   ├── api/                    # REST API Endpoint Routers
│   │   │   ├── analytics.py        # Chart data APIs (Histogram, Heatmap, Sunburst)
│   │   │   ├── dashboard.py        # Executive KPI metrics
│   │   │   ├── jobs.py             # CRUD job management
│   │   │   ├── ml.py               # K-Means & Isolation Forest endpoints
│   │   │   ├── recommend.py        # Candidate matching & skill gap engine
│   │   │   ├── scraper.py          # On-demand web scraping triggers
│   │   │   └── search.py           # Multi-faceted parameter filtering
│   │   ├── config/                 # Pydantic BaseSettings environment config
│   │   ├── database/               # SQLAlchemy Session, Engine & Loaders
│   │   ├── etl/                    # Pandas data transformer & ETL pipeline
│   │   ├── ml/                     # Machine learning feature extractors & models
│   │   ├── models/                 # SQLAlchemy ORM models (Job, Skill, Company)
│   │   ├── preprocessing/          # Regex salary, experience & skill sanitizers
│   │   ├── scraper/                # Web scraper engine (RemoteOK + Mock portal)
│   │   └── tests/                  # PyTest suite for API, ETL & Scrapers
│   ├── data/                       # Timestamped raw & processed CSV stores
│   ├── requirements.txt            # Python dependencies
│   └── README.md                   # Backend documentation
│
├── frontend/                       # React 19 + TypeScript + Vite UI
│   ├── src/
│   │   ├── components/             # Reusable UI components & Navbar
│   │   ├── pages/                  # 11 Interactive Analytics Views
│   │   │   ├── LandingPage.tsx     # Hero section & interactive workflow
│   │   │   ├── Dashboard.tsx       # Executive KPI metrics & charts
│   │   │   ├── SalaryAnalytics.tsx # Salary distribution & box plots
│   │   │   ├── SkillsAnalytics.tsx # Skills heatmaps & treemaps
│   │   │   ├── ClusteringPage.tsx  # K-Means 2D scatter plots
│   │   │   ├── AnomaliesPage.tsx   # Isolation Forest anomaly engine
│   │   │   ├── RecommendPage.tsx   # AI Matcher & Skill Gap Analysis
│   │   │   └── SearchPage.tsx      # Multi-parameter live job search
│   │   ├── services/               # Axios API client integrations
│   │   └── types/                  # TypeScript data interfaces
│   ├── package.json                # Frontend dependencies
│   └── README.md                   # Frontend documentation
│
└── .gitignore                      # Git exclusion rules
```

---

## 📡 REST API Documentation

<details>
<summary><b>🌐 Expand complete REST API Endpoint Directory</b></summary>

| Category | Method | Endpoint | Description |
| :--- | :--- | :--- | :--- |
| **System** | `GET` | `/` | Health check & system version status |
| **Dashboard** | `GET` | `/api/dashboard` | Aggregated executive KPIs and metrics |
| **Jobs** | `GET` | `/api/jobs` | Paginated job postings with sorting |
| **Jobs** | `GET` | `/api/jobs/{id}` | Detailed job posting by ID |
| **Search** | `GET` | `/api/search` | Multi-faceted query (salary, location, exp, remote, skills) |
| **Analytics** | `GET` | `/api/analytics/salary-stats` | Overall salary min, max, median, and mean |
| **Analytics** | `GET` | `/api/analytics/salary-histogram` | Salary distribution histogram bins |
| **Analytics** | `GET` | `/api/analytics/skill-cooccurrence` | Skill co-occurrence correlation matrix |
| **Analytics** | `GET` | `/api/analytics/location-hierarchy` | City/Country location hierarchy sunburst data |
| **Machine Learning** | `GET` | `/api/ml/clusters` | K-Means market segmentation cluster centroids & points |
| **Machine Learning** | `GET` | `/api/ml/anomalies` | Isolation Forest outlier decision scores |
| **Recommendation**| `POST` | `/api/recommend/jobs` | Candidate profile matching & skill gap analysis |
| **Pipeline** | `POST` | `/api/scrape` | Trigger web scraper execution session |
| **Pipeline** | `POST` | `/api/etl` | Execute raw-to-processed ETL data pass |

> 💡 **Interactive Swagger UI** available at `http://127.0.0.1:8000/docs` when backend is running.

</details>

---

## 🎨 Dashboard Pages Overview

The frontend includes **11 dedicated analytical views**:

1. 🏠 **Landing Page (`/`)**: Interactive product tour, hero metrics counter, and pipeline visualization.
2. 📊 **Executive Dashboard (`/dashboard`)**: 8 KPI summary cards, salary distributions, and top demand trends.
3. 💰 **Salary Analytics (`/salary`)**: Detailed salary histograms, category breakdowns, and seniority box plots.
4. 🛠️ **Skills Analytics (`/skills`)**: Skill demand ranking, co-occurrence heatmaps, and technology treemaps.
5. 🏢 **Company Leaderboard (`/companies`)**: Top hiring employers and market share breakdown.
6. 🌍 **Location Intelligence (`/locations`)**: Geographical placement distribution and location sunburst chart.
7. 🧩 **K-Means Clustering (`/clusters`)**: Interactive 2D scatter plots of market compensation segments.
8. ⚠️ **Anomaly Detection (`/anomalies`)**: Isolation Forest decision plot flagging outlier postings.
9. 🎯 **AI Job Recommender (`/recommend`)**: Resume matching engine with dynamic missing skill gap analysis.
10. 🔍 **Multi-Parameter Search (`/search`)**: Live full-text search with instant dynamic filters.
11. ⚙️ **System Settings (`/settings`)**: Live API status monitoring and manual pipeline execution triggers.

---

## 🚀 Quick Start Guide

### Prerequisites
- **Python**: `v3.12+`
- **Node.js**: `v18.0+`
- **npm**: `v9.0+`

---

### 1️⃣ Backend Setup (FastAPI & ML Engine)

```bash
# Navigate to backend directory
cd backend

# Create virtual environment
python -m venv venv

# Activate virtual environment
# Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# macOS / Linux:
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Start FastAPI development server
uvicorn app.main:app --reload --host 127.0.0.1 --port 8000
```

> 🌐 Backend API running at: `http://127.0.0.1:8000`  
> 📖 OpenAPI Docs available at: `http://127.0.0.1:8000/docs`

---

### 2️⃣ Frontend Setup (React 19 & Vite)

```bash
# Open a new terminal and navigate to frontend directory
cd frontend

# Install Node modules
npm install

# Launch Vite development server
npm run dev
```

> 🚀 Frontend Dashboard running at: `http://localhost:3000`

---

## 🧪 Running Automated Tests

Run the complete backend PyTest suite to verify scrapers, ETL transformations, database models, and API routes:

```bash
cd backend
pytest app/tests/ -v
```

---

## 📄 License

Distributed under the **MIT License**. See `LICENSE` for more information.

---

Developed with ❤️ by **[Omkar Narsale](https://github.com/Omkar-narsale)**

⭐ *If you find JobPulse AI helpful, please give this repository a star!* ⭐

</div>

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform

JobPulse-AI-An-Intelligent-Job-Market-Analytics-Platform Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README