freq

KO | EN

gitlite — search

#javascript #llm #golang #go #rust #deep-learning #java #cli #kubernetes #awesome-list #pytorch #security #mcp #ios

(★ 225)

This is a repository for freq.py and freq_server.py

File Explorer

Download Latest Version (.zip)

- freq.service
- sample_upstart.config
CONTRIBUTE.md
freq.exe
freq.py
freq_server.exe
freq_server.py
freq_sort.py
freqtable2018.freq
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# freq

This is a repository for `freq.py` and `freq_server.py`.

## Background:

While sitting in SANS SEC511 I listened to [`@sethmisenar`](https://twitter.com/sethmisenar) laement the difficulty in using existing tools to detect DGA (Domain Generation Algorithm) hostnames often used by malware.  There are lots of AI based tools out there that do this but some are rather complex. I thought I could quickly write a tool that would work.  In about 30 minutes I threw together some old code I had lying around from a SQL Injection tool I worked on and I had a working proof of concept.  `freq.py` was born and it worked pretty well.  A year later [`@securitymapper`](https://twitter.com/securitymapper) had me wrap it in a web interface so he could query it from a SIEM and then the tool took off.  It turns out to be a pretty effective technique and gained some popularity and wide use!   This is a rewrite of the tool that incorporates some lessons learned and performance enhancements.

## Recent Improvements:

- Only one table is required for case sensitve or insensitive lookups. The tables are all case sensitive.  You can turn off and on case sensitivity and the .probability lookups will do what is needed to make them case insensitive.
- Ignored characters with `--exclude` option.  Like `--ignore_case`, the characters are only ignored in the calculations of the probability. They are not ignored in the building of the table.
- Speed.  Like I said.  It was a proof of concept and never really built with any performance in mind.  This fixes that.
- Accuracy.  Some errors in calulations were identified by Pepe Berta (thanks!).  This fixes those and several others.  If you find others let me know.
- Two calculations - I've added a second frequency score that I've calculated differently.  It will requires some testing to see if it is more useful than the previous number in detecting random hosts.
- Multiple freq tables can be passed and loaded on the CLI then processed in the url.  For example: `python3 freq_server.py 9000 files.freq dns.freq`  Then query the url with the table name instead of measure like this `http://127.0.0.1:9000/dns.freq/hostname` or `http://127.0.0.1:9000/dns.freq1/hostname` for just measurement 1.

## Version Compatibility:

Both `freq.py` and `freq_server.py` will work in either Python2 or Python3, but ship as a python3 script.

## System-level Service Startup:

A systemd startup file is provided, although you will likely need to adjust paths to the script and `freqtable2018.freq` file. The provided sample assumes you've cloned this repository to /usr/local/share/freq/. Enable with something like the following, again substituting the appropriate paths:

```console
sudo systemctl enable /usr/local/share/freq/systemd/freq.service
sudo systemctl start freq.service
```

## Conntributing

If you're interested in contributing to the project, feel free to read through our [Contributing](CONTRIBUTE.md) document.

## To Do:

freq

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

freq

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README