sicrograd

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #golang #go #deep-learning #windows #awesome #kubernetes #vue #awesome-list #security #open-source

(★ 28)

Scala implementation of Micrograd. A tiny scalar-valued autograd engine and a neural net implementation.

sicrograd 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Sicrograd

A Scala implementation of [Micrograd](https://github.com/karpathy/micrograd), approached with a pure and functional style. This project was inspired by and built while following Andrej Karpathy's YouTube video [Spelled-out intro to neural networks](https://www.youtube.com/watch?v=VMj-3S1tku0).

Inside, you'll discover:

* A domain-specific language for crafting mathematical expressions.
* A pure functional backpropagation engine.
* The foundational elements for a small neural network library.
* Utilities for visualizing expression graphs and observing neuron training.

### Prerequisites

To get started, you'll need:

* [SBT](https://www.scala-sbt.org/download/): For compiling and running the Scala code.
* [Graphviz](https://graphviz.org/download/): For rendering the mathematical expression visualizations.

Once you have SBT and Graphviz installed, you can dive into the demos:
```bash
# To visualize a math expression
sbt "runMain com.github.mouslihabdelhakim.sicrograd.VisualizeExpressions"

# To train a neural network to approximate the sin function
sbt "runMain com.github.mouslihabdelhakim.sicrograd.SinFunctionDemo"

# To train a neural network on the moon dataset 
sbt "runMain com.github.mouslihabdelhakim.sicrograd.MoonDataSetDemo"

# To visualize the training of a neural network
sbt "runMain com.github.mouslihabdelhakim.sicrograd.VisualizeNeuronTraining" 
```

### Crafting and Visualizing Expressions

Sicrograd's DSL allows you to intuitively define variables (like weights and biases) and combine them to construct mathematical expressions. These expressions can then be visualized.

Here’s a taste:

```Scala
    val expression = for {
      weight1 <- VariableScalar(1.0, "w1")
      weight2 <- VariableScalar(1.0, "w2")
      bias <- VariableScalar(1.0, "b")
    } yield weight1 * 2 + weight2 * 4 + bias

val drawingResults = for {
      exp <- expression
      _ <- derive(exp)
      _ <- draw(exp)
    } yield ()
```

The output of this code is the following diagram:

![math expression](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/w1_*_2_00_+_w2_*_4_00_+_b.png)

### Training neural networks

Sicrograd can train simple Multi-Layer Perceptrons (MLPs). The repository includes demos for tasks like function approximation and binary classification.

#### 1. Approximating a Sine Wave

The `SinFunctionDemo` trains a single-layer MLP to approximate the sin function for values between [0, 2π].

The network is trained on randomly generated data points:

![sin function training data](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/sin_vs_nn_training_testing_data.png)

And here's how the network's output compares to the actual sin function after training:

![nn vs sin](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/sin_vs_nn.png)

#### 2. Moon Dataset Classification

The `MoonDataSetDemo` provides a full example of training an MLP as a binary classifier on [the classic "moon" dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html).

The data used to train the neural network are shown below:

![training_test_data](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/moon_dataset.png)

Here is the achieved decision boundary on the moon dataset:
![classifier_output](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/classifier_output.png)

### Visualizing Neuron Training Dynamics
For a peek into the learning process, the VisualizeNeuronTraining class demonstrates how a neuron's weights are updated by the gradient descent algorithm. While the GIF aesthetics are a work-in-progress, it effectively illustrates the iterative nature of training:

![single_neuron_training](https://raw.githubusercontent.com/MouslihAbdelhakim/sicrograd/master/renders/tanh(n0_w0_*_I1_+_n0_b)_-_y**2.gif)

sicrograd

sicrograd 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

sicrograd

sicrograd 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README