pst-extraction

KO | EN

gitlite — search

#javascript #android #machine-learning #rust #deep-learning #nodejs #java #vue #awesome-list #pytorch #security #mcp #ios #open-source

(★ 37)

PST extraction and analytic pipeline

pst-extraction Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

### PST Extraction

place pst files in `pst-extract/pst/`

1. `bin/explode_psts.sh`  - runs readpst to convert pst to mbox
1. `bin/normalize_mbox.sh` - mbox files to json
1. `bin/run_spark_tika.sh` - tika extract text of attachments
1. `bin/run_tika_content_join.sh` - join attachment text with email json
1. `bin/run_spark_content_split.sh` - removes base64 encoded attachment from emails json and puts the json in to a separate directory 
1. `bin/run_spark_emailaddr.sh` - email address extraction and community assignment
1. `bin/run_spark_email_community_assign.sh` - assign communities to email json objects 
1. `bin/run_spark_topic_clustering.sh` - assign topic clustering to email json objects output by community assign 
1. `bin/run_spark_mitie.sh` - Run MITIE to generate entities for email and add to email json generated by topic clustering
1. `bin/run_spark_es_ingest_emailaddr.sh` - ingest emailaddrs to ES index 
1. `bin/run_spark_es_ingest_attachments.sh` - ingest attachments to ES index 
1. `bin/run_spark_es_ingest_emails.sh` - ingest emails with entities to ES index

### Extras

** Location Extraction **

__Locations extracted from text__ 
1. `bin/build_clavin_index.sh` setup location index (only needs to be
run once) 
2. `bin/run_location_extract.sh` extracts locations from text body
uses input from `bin/run_spark_content_split` task

__Locations extracted by IP__ 
1. `bin/setup_geo2ip.sh` setup geoip index 
2. `bin/run_spark_originating_location.sh` extracts location from ip address

![Workflow](https://raw.githubusercontent.com/Sotera/pst-extraction/master/etc/workflow.png)

This product includes GeoLite2 data created by MaxMind, available from [http://www.maxmind.com](http://www.maxmind.com).

pst-extraction

pst-extraction Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

pst-extraction

pst-extraction Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README