spark-nlp-workshop

(★ 1,089)

Public runnable examples of using John Snow Labs' NLP for Apache Spark.

File Explorer

Download Latest Version (.zip)

Showing a partial file list — download the zip above to see everything.

- - dict.txt
  - eng.train
  - streamlit_app.py
- - h_and_p.pdf
  - MT_OCR_00.pdf
  - MT_OCR_01.pdf
  - MT_OCR_02.pdf
  - ocr_ner.pdf
  - patterns.json
  - sample_pdf.pdf
- - rxnorm_sample.csv
  - snomed_sample.csv
- - default-sentiment-dict.txt
- - part-00000-08092d15-dd8c-40f9-a1df-641a1a4b1698.snappy.parquet
  - part-00001-08092d15-dd8c-40f9-a1df-641a1a4b1698.snappy.parquet
  - part-00002-08092d15-dd8c-40f9-a1df-641a1a4b1698.snappy.parquet
  - part-00003-08092d15-dd8c-40f9-a1df-641a1a4b1698.snappy.parquet
- - coca2017.txt
  - words.txt
- - entities.txt
- - amazon_cells_labelled.txt
  - imdb_labelled.txt
  - yelp_labelled.txt
- - NLU_BERT_Word_Embeddings_and_t-SNE_visualization_example.html
  - NLU_classifiers_demo.html
  - NLU_ELMo_Word_Embeddings_and_t-SNE_visualization_example.ipynb.html
  - NLU_Multiple_Word_Embeddings_and_t-SNE_visualization_example.ipynb.html
- benchmark.md
- Getting Started.html
- index.html
- README.md
- Spark NLP - Databricks.dbc
- - 10k_image.png
  - assertion_df.csv
  - cdns-20220101.html.txt
  - conll_noO.conll
  - earning_calls_sample.csv
  - finance_binary_clf.csv
  - finance_clf_data.csv
  - finance_data.csv
  - finance_pca_samples.csv
  - hipaa-table-001.txt
  - im1.png
  - im2.jpg
  - im4.png
  - img10.png
  - img5.jpg
  - img6.png
  - img7.png
  - invoice_01.pdf
  - invoice_01.png
  - relations.csv
  - res_report_page.pdf
  - sample_company_name.csv
  - sample_openedgar.json
  - sar.png
  - t01.jpg
  - t02.jpg
  - t03.jpg
  - tapas_example.pkl.bz2
  - test0.jpeg
  - tf2_contrib.py
- - README.md
- - 80.Financial_Graphs_Neo4j.ipynb
- - Jan 2023 - Finance NLP.pdf
  - README.md
- 01.Page_Splitting.ipynb
- 02.Sentence_Splitting_Tokenization.ipynb
- 03.Word_Sentence_Embeddings.ipynb
- 04.0.Document_Paragraph_Classification.ipynb
- 04.1.Training_Financial_Binary_Classifier.ipynb
- 04.2.Training_Financial_Multiclass_Classifier.ipynb
- 04.3.Training_Financial_Multilabel_Classifier.ipynb
- 05.0.NER_and_ZeroShotNER.ipynb
- 05.1.Training_Financial_NER.ipynb
- 05.2.Financial_NER_Additional_Examples.ipynb
- 05.3.ZeroShot_Financial_NER.ipynb
- 05.4.BertForTokenClassification_TrainEval.ipynb
- 05.5.BertForTokenClassification_TrainAndSave.ipynb
- 05.6.Contextual_Parser_Rule_Based_NER.ipynb
- 06.0.Relation_Extraction.ipynb
- 06.1.Additional_Relation_Extraction_Examples.ipynb
- 06.2.ZeroShot_Relation_Extraction.ipynb
- 06.3.Relation_Extraction_Training.ipynb
- 07.0.Understand_Entities_in_Context.ipynb
- 07.1.Training_Financial_Assertion.ipynb
- 08.0.Answering_Questions_Financial_Texts.ipynb
- 08.1.Automatic_Question_Generation_Financial_Texts.ipynb
- 08.2.Table_Question_Answering.ipynb
- 08.3.Finetuning_Table_Question_Answering.ipynb
- 09.0.Normalization_with_Entity_Resolution_Edgar.ipynb
- 09.1.Entity_Resolution_Edgar_unique_IDs_Tickers.ipynb
- 09.2.Entity_Resolution_NASDAQ.ipynb
- 09.3.Entity_Resolution_Training.ipynb
- 10.0.Data_Augmentation_with_ChunkMappers.ipynb
- 10.1.Data_Augmentation_with_ChunkMappers_Edgar.ipynb
- 10.2.Chunk_Mappers_Training.ipynb
- 11.0.Deidentification.ipynb
- 11.1.Pretrained_Deidentification_Pipeline.ipynb
- 11.2.Deidentification_Utility_Module.ipynb
- 12.1.Financial_Summarization.ipynb
- 13.0.Date_Normalizer.ipynb
- 14.0.Financial_ChunkKeyPhraseExtraction.ipynb
- 15.0.Financial_Text_Generation.ipynb
- 16.0.Vector_Store_Integration.ipynb
- 80.0.Use_case_Capital_Calls.ipynb
- 80.1.Augmenting_NER_With_Wikidata.ipynb
- 80.2.Suspicious_Activity_Reports_NER.ipynb
- 90.0.Financial_Visual_Classification.ipynb
- 90.1.Visual_and_Textual_Classification.ipynb
- 90.2.Financial_Visual_NER.ipynb
- 90.3.Financial_Table_Signature_Extraction.ipynb
- 90.4.Financial_Visual_and_Table_QA.ipynb
- 90.5.Financial_Visual_NER_Position_Finder.ipynb
- README.md
- Healthcare_NLP_Agents_with_LLMs.ipynb
- Implementing_RAG_in_Healthcare_with_SparkNLP_for_Healthcare.ipynb
- JSL_Healthcare_NLP_Agents_with_LLMs_Convergence_2024_May.ipynb
- Medical_Chatbot_RAG_JohnSnowLabs_Haystack.ipynb
- Medical_Chatbot_RAG_JohnSnowLabs_LangChain.ipynb
- README.md
- - age.json
  - annotations.json
  - AskAPatient.fold-0.test.txt
  - AskAPatient.fold-0.train.txt
  - AskAPatient.fold-0.validation.txt
  - blstm_3_200_128_83.pb
  - date.json
  - deid_surrogate_test_all_groundtruth_version2.xml
  - deid_surrogate_unmasked_text.csv
  - diabetes_txt_files.zip
  - Entity Resolution in Spark NLP for Healthcare.jpeg
  - faiss_retriever_db_diabetes.vs.zip
  - faiss_retriever_db_guidelines.vs.zip
  - gender.csv
  - gender.json
  - hipaa-table-001.txt
  - i2b2_assertion_sample_short.csv
  - i2b2_clinical_rel_dataset.csv
  - insurance_risk_factors_df.pickle
  - mt_data.csv
  - mt_oncology_10.zip
  - mt_sample_01.pdf
  - mt_samples.csv
  - mt_samples_10.csv
  - mtsamples_classifier.csv
  - NCBI_disease_official_test.conll
  - NCBI_disease_official_train_dev.conll
  - NER_label_list.md
  - NER_NCBIconlltest.txt
  - NER_NCBIconlltrain.txt
  - obfuscate.txt
  - obfuscate_es.txt
  - obfuscate_fr.txt
  - obfuscate_it.txt
  - obfuscate_pt.txt
  - obfuscate_ro.txt
  - oncology_label_description.csv
  - petfinder-mini.csv
  - pubmed_diabetes_1000_meta.csv
  - RE_relation_list.md
  - sample_ADE_dataset.csv
- - Basel_Worskhop_Oct_2023.pdf
  - Spark NLP Healthcare Training - April 2023.pdf
  - Spark NLP Healthcare Training - April 2024.pdf
  - Spark NLP Healthcare Training - Jan 2023.pdf
  - Spark NLP Healthcare Training - July 2023.pdf
  - Spark NLP Healthcare Training - October 2023.pdf
- - blstm_3_200_20_85.pb
- 00.SparkNLP_for_Healthcare_3h_Notebook.ipynb
- 01.0.Clinical_Named_Entity_Recognition_Model.ipynb
- 01.1.prepare_CoNLL_from_annotations_for_NER.ipynb
- 01.2.Resume_MedicalNer_Model_Training.ipynb
- 01.3.BertForTokenClassification_NER_ONNX_SparkNLP_with_Transformers.ipynb
- 01.4.ZeroShot_Clinical_NER.ipynb
- 01.5.Contextual_Parser_Rule_Based_NER.ipynb
- 01.6.Rule_Based_Entity_Matchers.ipynb
- 01.7.Text_Matcher_Internal.ipynb
- 01.8.Generative_AI_to_Ner_Training.ipynb
- 01.9.PretrainedZeroShotMultiTask.ipynb
- 02.0.Clinical_Assertion_Model.ipynb
- 02.1.Scope_window_tuning_assertion_status_detection.ipynb
- 02.2.FewShot_Assertion_Classifier.ipynb
- 02.3.Contextual_Assertion.ipynb
- 02.4.BertForAssertionClassification.ipynb
- 03.0.Clinical_Relation_Extraction.ipynb
- 03.1.Clinical_Relation_Extraction_BodyParts_Models.ipynb
- 03.2.Clinical_RE_Knowledge_Graph_with_Neo4j.ipynb
- 03.3.ZeroShot_Clinical_Relation_Extraction.ipynb
- 03.4.Resume_RelationExtractionApproach_Training.ipynb
- 04.0.Clinical_DeIdentification.ipynb
- 04.1.Clinical_Multi_Language_Deidentification.ipynb
- 04.10.Portuguese_Clinical_Deidentification.ipynb.ipynb
- 04.12.Deidentification_NER_Profiling_Pipeline.ipynb
- 04.13.Deidentification_Model_Evaluation.ipynb
- 04.14.End2End_Preannotation_and_Training_Pipeline.ipynb
- 04.15.CDA_DeIdentification.ipynb
- 04.2.Clinical_Deidentification_SparkNLP_vs_SpaCy_vs_Scrubadub_vs_Presidio_Comparison.ipynb
- 04.3.Clinical_Deidentification_SparkNLP_vs_Cloud_Providers_Comparison.ipynb
- 04.4.Clinical_Deidentification_Improvement.ipynb
- 04.5.Clinical_Deidentification_Utility_Module.ipynb
- 04.6.Light_Deidentification.ipynb
- 04.7.Deidentification_Custom_Pretrained_Pipelines.ipynb
- 04.8.Clinical_Deidentification_for_Structured_Data.ipynb
- 04.9.German_Clinical_Deidentification.ipynb
- 05.0.Clinical_Entity_Resolvers.ipynb
- 05.1.Clinical_Entity_Resolver_Model_Training.ipynb
- 05.10.CPT_Entity_Resolver.ipynb
- 05.2.Finetuning_Clinical_Entity_Resolver_Model.ipynb
- 05.3.Calculate_Medicare_Risk_Adjustment_Score.ipynb
- 05.4.Sentence_Entity_Resolvers_with_EntityChunkEmbeddings.ipynb
- 05.5.Improved_Entity_Resolvers_in_SparkNLP_with_sBert.ipynb
- 05.6.Improved_Entity_Resolution_with_SentenceChunkEmbeddings.ipynb
- 05.7.MedDRA_Models.ipynb
- 05.8.Resolving_Medical_Terms_to_Terminology_Codes_Directly.ipynb
- 05.9.Spanish_Healthcare_Models.ipynb
- 06.0.Chunk_Mapping.ipynb
- 06.1.Code_Mapping_Pipelines.ipynb
- 07.0.Pretrained_Clinical_Pipelines.ipynb
- 07.1.Pretrained_NER_Profiling_Pipelines.ipynb
- 07.2.Task_Based_Clinical_Pretrained_Pipelines.ipynb
- 07.4.PipelineTracer_and_PipelineOutputParser.ipynb
- 08.0.Clinical_Text_Classification_with_SparkNLP.ipynb
- 08.1.Text_Classification_with_DocumentMLClassifier.ipynb
- 08.2.Generic_Classifier.ipynb
- 08.3.MedicalBertForSequenceClassification_in_SparkNLP.ipynb
- 08.4.Gender_Classifier.ipynb
- 08.5.Text_Classification_with_Contextual_Window_Splitting.ipynb
- 08.6.Text_Classification_with_FewShotClassifier.ipynb
- 09.0.Spark_OCR.ipynb
- 09.1.Spark_OCR_Multi_Modals.ipynb
- 09.2.Spark_OCR_Deidentification.ipynb
- 09.3.Spark_OCR_Utility_Module.ipynb
- 09.4.PDF_Deidentification.ipynb
- 10.0.Clinical_NER_Chunk_Merger.ipynb
- 11.0.SentenceDetectorDL_Healthcare.ipynb
- 12.0.Clinical_Context_Spell_Checker.ipynb
- 13.0.Date_Normalizer.ipynb
- 14.0.Drug_Normalizer.ipynb
- 15.0.EntityRuler_with_Clinical_NER_Models.ipynb
- 15.1.Contextual_Entity_Ruler.ipynb
- 16.0.Coreference_Resolution_with_Clinical_NER_Models.ipynb
- 17.0.Graph_builder_for_DL_models.ipynb
- 18.0.Chunk_Sentence_Splitter.ipynb
- 18.1.Section_Header_Splitting_and_Classification.ipynb
- 19.0.Chunk_Key_Phrase_Extraction.ipynb
- 20.0.Named_Entity_Disambiguation.ipynb
- 21.0.Oncology_Model.ipynb
- 21.1.Oncology_Use_Cases.ipynb
- 22.0.Adverse_Drug_Event_ADE_NER_and_Classifier.ipynb
- 23.0.Medical_Question_Answering.ipynb
- 23.1.Porting_QA_Models_From_Text_Generator_Backbone.ipynb
- 24.0.Medical_Text_Summarization.ipynb
- 24.1.Medical_Text_Summarization_with_Abstractive_Approach.ipynb
- 24.2.Medical_Text_Summarization_with_Extractive_Approach.ipynb
- 24.3.Comparison_Medical_Text_Summarization.ipynb
- 25.0.Biogpt_Chat_JSL.ipynb
- 25.1.Medical_Text_Generation.ipynb
- 26.0.Voice_of_Patient_Models.ipynb
- 27.0.Social_Determinant_of_Health_Models.ipynb
- 28.0.Model_Download_Helpers.ipynb
- 29.0.Text2SQL_Generation.ipynb
- 30.0.InternalDocumentSplitter.ipynb
- 31.0.Structured_Streaming_with_SparkNLP_for_Healthcare.ipynb
- 32.0.Flattener_Convert_Annotations_to_DF.ipynb
- 33.0.Opioid_Models.ipynb
- 34.0.Clinical_Medication_Use_Case.ipynb
- 35.0.Analyse_Veterinary_Documents_with_Healthcare_NLP.ipynb
- 36.0.Loading_Medical_and_Open_Source_LLMs.ipynb
- 36.1.Multi_Modal_LLMs.ipynb
- 36.2.MedicalLLMEntityExtractor.ipynb
- 37.0.Human_Phenotype_Extraction_And_HPO_Code_Mapping.ipynb
- 38.0.Annotation_Converter.ipynb
- README.md
- Spark_NLP_Clinical_NER_Playground_Streamlit_app.ipynb
- - AssertionDLExample.java
  - AssertionFiltererDLExample.java
  - ChunkFiltererExample.java
  - DeidentificationExample.java
  - DrugNormalizerExample.java
  - MedicalNerModelExample.java
  - NerConverterInternalFiltererExample.java
  - readme.md
  - RelationExtractionModelExample.java
- jsl_installation_script_ubuntu.sh
- README.md
- SparkNLP_offline_installation.ipynb
- - agreement_page.pdf
  - assertion_df.csv
  - assertion_fin.csv
  - cdns-20220101.html.txt
  - commercial_lease_1.txt
  - commercial_lease_2.txt
  - commercial_lease_3.txt
  - conll_noO.conll
  - credit_agreement.txt
  - credit_agreement_2.txt
  - finance_clf_data.csv
  - finance_data.csv
  - hipaa-table-001.txt
  - intellectual_property_agreement.txt
  - legal_clf.csv
  - legal_pca_samples.csv
  - loan_agreement.txt
  - mnda_example.txt
  - mnda_sample.csv
  - nda_2.txt
  - non_disclosure_agreement.txt
  - relations.csv
  - sample_company_name.csv
  - sample_openedgar.json
  - t01.jpg
  - t02.jpg
  - t03.jpg
  - test0.jpeg
  - tf2_contrib.py
- - README.md
- - 80.1.Legal_Graphs_Neo4j.ipynb
- - Jan, 2023 - Legal NLP.pdf
  - README.md
- 01.Page_Splitting.ipynb
- 02.Sentence_Splitting_Tokenization.ipynb
- 03.Word_Sentence_Embeddings.ipynb
- 04.0.Clause_Document_Classification.ipynb
- 04.1.Training_Legal_Binary_Classifier.ipynb
- 04.2.Training_Legal_Multiclass_Classifier.ipynb
- 04.3.Training_Legal_Multilabel_Classifier.ipynb
- 04.4.Training_Legal_Multilabel_Classifier.ipynb
- 04.5.Classifying_with_WindowSplitting.ipynb
- 05.0.NER_and_ZeroShotNER.ipynb
- 05.1.Training_Legal_NER.ipynb
- 05.2.Clause_based_NER.ipynb
- 05.3.ZeroShot_Legal_NER.ipynb
- 05.4.BertForTokenClassification_TrainEval.ipynb
- 05.5.BertForTokenClassification_TrainAndSave.ipynb
- 05.6.Contextual_Parser_Rule_Based_NER.ipynb
- 06.0.Relation_Extraction.ipynb
- 06.1.Relation_Extraction_and_ZeroShotRE.ipynb
- 06.2.Relation_Extraction_Training.ipynb
- 06.3.Classification_NER_RE_on_Parties.ipynb
- 07.0.Understand_Entities_in_Context.ipynb
- 07.1.Training_Legal_Assertion.ipynb
- 08.0.Answering_Questions_Legal_Texts.ipynb
- 08.1.Automatic_Question_Generation_Legal_Texts.ipynb
- 08.2.NER_using_Question_Answering.ipynb
- 09.0.Normalization_with_Entity_Resolution_Edgar.ipynb
- 09.1.Entity_Resolution_Edgar_unique_IDs.ipynb
- 09.2.Entity_Resolution_Training.ipynb
- 10.0.Data_Augmentation_with_ChunkMappers.ipynb
- 10.1.Chunk_Mappers_Training.ipynb
- 11.0.Deidentification.ipynb
- 11.1.Deidentification_Utility_Module.ipynb
- 12.Coreference_Resolution.ipynb
- 13.0.Legal_Summarization.ipynb
- 14.0.Date_Normalizer.ipynb
- 14.0.Legal_ChunkKeyPhraseExtraction.ipynb
- 15.0.Date_Normalizer.ipynb
- 16.0.Legal_Text_Generation.ipynb
- 17.0.Vector_Store_Integration.ipynb
- 80.0.Legal_Contract_Understanding.ipynb
- 80.1.Legal_Contract_Understanding_NDA.ipynb
- 80.2.Legal_Subpoenas_NER.ipynb
- 90.0.Legal_Visual_Document_Understanding.ipynb
- 90.1.Layout_Classification_with_VisualNLP.ipynb
- 90.2.Legal_Visual_NER_Position_Finder.ipynb
- README.md
- - nlu_covid_emotion_showcase.ipynb
  - nlu_covid_sentiment_showcase.ipynb
  - nlu_emotion_airline_demo.ipynb
  - nlu_sentiment_airline_demo.ipynb
- - NLU1_1_2_Bengali_ner_Hindi_Embeddings_30_new_models.ipynb
  - NLU_3_0_2_release_notebook.ipynb
- - 01_dashboard.py
  - 02_NER.py
  - 03_text_similarity_matrix.py
  - 04_dependency_tree.py
  - 05_classifiers.py
  - 06_token_features.py
  - 07_token_embedding_manifolds.py
  - 08_sentence_embedding_manifolds.py
  - 09_entity_embedding_manifolds.py
  - README.md
- - amazon_alexa.tsv
  - appt-fakes.csv
  - audio_50secs.mp3
  - french_tweets.csv
  - holmes.txt
  - news_category_test.csv
  - news_category_test_multi_lingual.csv
  - news_category_train.csv
  - sample-sentences-en.txt
  - small_btc.csv
  - spam_ham_dataset.csv
  - title_conference.csv
- - Spark NLP Training - Public - April 2023.pdf
  - Spark NLP Training - Public - April 2024.pdf
  - Spark NLP Training - Public - Jan 2023.pdf
  - Spark NLP Training - Public - July 2023.pdf
  - Spark NLP Training - Public - October 2023.pdf
- - conll_eval.py
  - ner_highlighter.py
  - ner_image_log_parser.py
  - ner_log_parser.py
  - NLP Server.postman_collection.json
- 00.SparkNLP_for_OpenSource_3h_Notebook.ipynb
- 01.0.SparkNLP_Basics.ipynb
- 01.1.Reader2_Family_Native_File_Readers.ipynb
- 02.0.Text_Preprocessing_with_SparkNLP_Annotators_Transformers.ipynb
- 03.0.SparkNLP_Pretrained_Models.ipynb
- 03.1.SparkNLP_Pretrained_Models_Overview.ipynb
- 04.0.NERDL_Training.ipynb
- 04.1.NerDL_Graph.ipynb
- 04.2.Transformers_for_Token_Classification_in_SparkNLP.ipynb
- 04.3.Import_Transformers_Into_SparkNLP.ipynb
- 04.4.ZeroShot_NER.ipynb
- 04.5.EntityRuler.ipynb
- 05.0.Text_Classification_with_ClassifierDL.ipynb
- 05.1.Text_Classification_Examples_in_SparkML_SparkNLP.ipynb
- 05.2.Transformers_for_Sequence_Classification_in_SparkNLP.ipynb
- 05.3.Multi_Lingual_Training_and_Models.ipynb
- 05.4.ZeroShot_Text_Classification.ipynb
- 06.0.Text_Similarities_and_Dimension_Reduction_Visualizations_for_Embeddings.ipynb
- 06.1.Modern_Embeddings.ipynb
- 07.0.Context_Spell_Checker.ipynb
- 08.0.Keyword_Extraction_YAKE.ipynb
- 09.0.SentenceDetectorDL.ipynb
- 10.0.Graph_Extraction.ipynb
- 12.0.NLU_Crashcourse_Every_SparkNLP_Model_in_One_Line.ipynb
- 13.0.T5_Question_Answering_and_Summarization_with_SparkNLP.ipynb
- 13.1.T5_Workshop_with_SparkNLP.ipynb
- 13.2.T5_SQL_Code_Generation_and_Style_Transfer_with_SparkNLP.ipynb
- 14.0.GPT2_Transformer_In_SparkNLP.ipynb
- 15.0.Question_Answering.ipynb
- 15.1.Table_Question_Answering.ipynb
- 16.0.Image_Classification.ipynb
- 17.0.Speech_Recognition.ipynb
- 18.0.Summarization.ipynb
- 20.0.Translations.ipynb
- 21.0.Document_Splitters.ipynb
- 21.1.Document_Similarity_Ranker.ipynb
- 22.0.Llama2_Transformer_In_SparkNLP.ipynb
- 22.1.OpenAI_In_SparkNLP.ipynb
- 22.2.Retrieval_Augmented_Generation_with_Spark_NLP.ipynb
- 23.0.AutoGGUF_in_SparkNLP.ipynb
- 23.1.Native_GGUF_Models_in_SparkNLP.ipynb
- 24.0.Benchmark_Unstructured_Sparknlp_Html.ipynb
- 25.0.Table_Structural_Context_Unstructured_Sparknlp.ipynb
- 26.0.Benchmark_Unstructured_and_SparkNLP_Files_Ingestion.ipynb
- README.md
- - Airgaped.ipynb
- - installation.ipynb
  - readme.md
- - readme_deployment_azureml_endpoint.md
  - SparkNLP-JSL-AzureML-Endpoint-Demo.ipynb
- - README.md
  - SparkNLP_Dataproc_Setup.ipynb
- - image-1.png
  - image-2.png
  - image-3.png
  - image-4.png
  - image.png
  - jsl_emr_bootstrap.sh
  - jsl_OCR_test_notebook_for_emr.ipynb
  - jsl_test_notebook_for_emr.ipynb
  - NLP_EMR_Setup.ipynb
  - README.md
  - spark_nlp_healthcare_emr_serverless_guide.md
- - Download_Libraries_For_Airgapped_Environment.ipynb
  - image-1.png
  - image-2.png
  - image-3.png
  - image-4.png
  - image-5.png
  - image-6.png
  - image-7.png
  - image.png
  - jars.png
  - jsl_emr_bootstrap.sh
  - jsl_emr_bootstrap_from_s3.sh
  - python_libs.png
  - README.md
  - spark-nlp-jsl-test-notebook.ipynb
- - init_sparknlp_healthcare.sh
  - README.md
  - spark_healthcare_nlp_classical.py
  - step1.png
  - step10.png
  - step11.png
  - step12.png
  - step13.png
  - step14.png
  - step15.png
  - step2.png
  - step3.png
  - step4.png
  - step5.png
  - step6.png
  - step7.png
  - step8.png
  - step9.png
- - README.md
  - spark_healthcare_nlp_serverless.py
  - step1.png
  - step2.2.png
  - step2.png
  - step3.png
  - step4.png
  - step5.png
  - step6.png
  - step7.2.png
  - step7.3.png
  - step7.png
- - readme.md
- - Dockerfile
  - environment.yml
  - license.json
  - main.py
  - manifest.yaml
  - README.md
  - test.py
- - Healthcare&OCR_SageMaker_Setup.ipynb
  - NLP_SageMaker_Setup.ipynb
  - OCR_SageMaker_Setup.ipynb
  - README.md
- - app-image-config-input.json
  - Dockerfile
  - ecr_configure.sh
  - environment.yml
  - license.json
  - README.md
  - requirements.txt
  - SparkNLP_sagemaker.ipynb
- - README.md
- - Demo.ipynb
  - Integrating_Medical_LLM_with_Langchain.ipynb
  - Integrating_Medical_LLM_with_LlamaIndex.ipynb
  - Integrating_Medical_LLM_with_OpenAI.ipynb
- - AnnotationMerger.ipynb
  - AssertionChunkConverter.ipynb
  - AssertionDLApproach.ipynb
  - AssertionDLModel.ipynb
  - AssertionFilterer.ipynb
  - AssertionLogRegApproach.ipynb
  - AssertionLogRegModel.ipynb
  - AssertionMerger.ipynb
  - AverageEmbeddings.ipynb
  - BertSentenceChunkEmbeddings.ipynb
  - Chunk2Token.ipynb
  - ChunkConverter.ipynb
  - ChunkFilterer.ipynb
  - ChunkFiltererApproach.ipynb
  - ChunkKeyPhraseExtraction.ipynb
  - ChunkMapperApproach.ipynb
  - ChunkMapperFilterer.ipynb
  - ChunkMapperModel.ipynb
  - ChunkMergeModel.ipynb
  - ChunkMergeModel2.ipynb
  - ChunkSentenceSplitter.ipynb
  - ContextualAssertion.ipynb
  - ContextualEntityFilterer.ipynb
  - ContextualEntityRuler.ipynb
  - ContextualParserApproach.ipynb
  - ContextualParserModel.ipynb
  - DateNormalizer.ipynb
  - DeIdentification.ipynb
  - DeIdentificationModel.ipynb
  - Doc2ChunkInternal.ipynb
  - DocMapperApproach.ipynb
  - DocMapperModel.ipynb
  - DocumentFiltererByClassifier.ipynb
  - DocumentFiltererByNER.ipynb
  - DocumentHashCoder.ipynb
  - DocumentLogRegClassifier.ipynb
  - DocumentMLClassifierApproach.ipynb
  - DocumentMLClassifierModel.ipynb
  - DrugNormalizer.ipynb
  - EntityChunkEmbeddings.ipynb
  - EntityRulerInternal.ipynb
  - ExtractiveSummarization.ipynb
  - FeaturesAssembler.ipynb
  - FewShotClassifierModel.ipynb
  - Flattener.ipynb
  - GenericClassifierApproach.ipynb
  - GenericClassifierModel.ipynb
  - GenericLogRegClassifierApproach.ipynb
  - GenericLogRegClassifierModel.ipynb
  - GenericSVMClassifierApproach.ipynb
  - GenericSVMClassifierModel.ipynb
  - InternalDocumentSplitter.ipynb
  - IOBTagger.ipynb
  - LargeFewShotClassifierModel.ipynb
  - Mapper2Chunk.ipynb
  - Medical_Question_Answering.ipynb
  - MedicalBertForSequenceClassification.ipynb
  - MedicalBertForTokenClassifier.ipynb
  - MedicalDistilBertForSequenceClassification.ipynb
  - MedicalLLM.ipynb
  - MedicalNerApproach.ipynb
  - MedicalNerModel.ipynb
  - MedicalSummarizer.ipynb
  - MedicalTextGenerator.ipynb
  - MedicalVisionLLM.ipynb
  - MetadataAnnotationConverter.ipynb
  - NameChunkObfuscator.ipynb
  - NameChunkObfuscatorApproach.ipynb
  - NerChunker.ipynb
  - NerConverterInternal.ipynb
  - NerDisambiguator.ipynb
  - NerDisambiguatorModel.ipynb
  - NerQuestionGenerator.ipynb
  - PretrainedZeroShotNer.ipynb
  - REChunkMerger.ipynb
  - RegexMatcherInternal.ipynb
  - ReIdentification.ipynb
  - RelationExtractionApproach.ipynb
  - RelationExtractionDLModel.ipynb
  - RelationExtractionModel.ipynb
  - RENerChunksFilter.ipynb
  - Replacer.ipynb
  - Resolution2Chunk.ipynb
  - ResolverMerger.ipynb
  - Router.ipynb
  - SentenceEntityResolverApproach.ipynb
  - SentenceEntityResolverModel.ipynb
  - StructuredJsonConverter.ipynb
  - TextMatcherModel.ipynb
  - WindowedSentenceModel.ipynb
  - ZeroShotNerModel.ipynb
  - ZeroShotRelationExtractionModel.ipynb
- - 02.01.ContextSpellChecker.ipynb
  - 02.05.NorvigSweetingSpellchecker.ipynb
  - 02.06.SymmetricDeleteSpellchecker.ipynb
  - 03.01.DateMatcher_MultiDateMatcher.ipynb
  - 04.01.NGramGenerator.ipynb
  - 05.01.Lemmatizer_LemmatizerModel.ipynb
  - 06.01.Stemmer.ipynb
  - 06.02.SentenceDetectorDLModel.ipynb
  - 06.03.Normalizer.ipynb
  - 06.04.StopWordsCleaner.ipynb
  - 07.01.DocumentNormalizer.ipynb
  - 08.01.Tokenizer.ipynb
  - 08.02.RegexTokenizer.ipynb
  - 08.03.ChunkTokenizer.ipynb
  - 08.04.TokenAssembler.ipynb
  - 09.01.YakeKeywordExtraction.ipynb
  - 10.01.RegexMatcher.ipynb
  - 11.01.TextMatcher_BigTextMatcher.ipynb
  - 12.01.DependencyParser_TypedDependencyParser.ipynb
  - 12.02.POSTagger.ipynb
  - 12.03.Chunker.ipynb
  - 13.01.GraphExtraction_GraphFinisher.ipynb
  - 14.01.SpanBertCoref.ipynb
  - 15.01.Word2Vec.ipynb
  - 15.02.WordEmbeddings.ipynb
  - 16.01.Doc2Vec.ipynb
  - 16.02.ChunkEmbeddings.ipynb
  - 16.03.SentenceEmbeddings.ipynb
  - 16.04.UniversalSentenceEncoder.ipynb
  - 17.01.Transformers-based_Embeddings.ipynb
  - 18.01.ViveknSentiment.ipynb
  - 18.02.SentimentDL.ipynb
  - 18.03.SentimentDetector.ipynb
  - 19.01.BertForSequenceClassification.ipynb
  - 19.03.Sentence_Embeddings_with_Transformers.ipynb
  - 20.01.ClassifierDLApproach.ipynb
  - 20.02.ClassifierDLModel.ipynb
  - 20.03.MultiClassifierDL.ipynb
  - 21.01.EntityRuler.ipynb
  - 22.01.NerDLModel_NerConverter.ipynb
  - 22.02.NerOverwriter.ipynb
  - 23.01.NerVisualizer.ipynb
  - 24.01.NerDLApproach.ipynb
  - 24.02.TFNerDLGraphBuilder.ipynb
  - 24.03.CoNLL_Preparation_for_NER.ipynb
  - 25.01.NerCrf.ipynb
  - 26.01.BertForTokenClassification.ipynb
  - 27.01.QuestionAnswering_with_Transformers.ipynb
  - 27.02.MultiDocumentAssembler.ipynb
  - 28.01.TapasForQuestionAnswering.ipynb
  - 29.01.WordSegmenter.ipynb
  - 30.01.MarianTransformer.ipynb
  - 31.01.LanguageDetectorDL.ipynb
  - 32.01.ImageAssembler.ipynb
  - 32.02.ViTForImageClassification.ipynb
  - 33.01.T5Transformer.ipynb
  - 34.01.Wav2Vec2ForCTC.ipynb
  - 35.01.LightPipeline.ipynb
  - 35.02.Token2Chunk.ipynb
  - 35.03.PretrainedPipeline.ipynb
  - 35.04.DocumentAssembler.ipynb
  - 35.05.Finisher.ipynb
  - 35.06.Doc2Chunk.ipynb
  - 35.07.Chunk2Doc.ipynb
  - 35.08.GPT2Transformer.ipynb
  - links.xlsx
  - readme.md
- download_notebooks.py
- links.xlsx
.gitattributes
.gitignore
colab_setup.sh
ISSUE_TEMPLATE.md
jsl_colab_setup.sh
jsl_colab_setup_with_OCR.sh
jsl_sagemaker_setup.sh
jsl_sagemaker_setup_3.0.1.sh
jsl_sagemaker_setup_with_OCR.sh
LICENSE
patterns.json
README.md

# Installation Guide

1. Get the code

git clone https://github.com/JohnSnowLabs/spark-nlp-workshop

Downloads the entire project code from GitHub to your computer.

cd spark-nlp-workshop

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Docker Desktop Needed to build and run containers. Install it and keep it running in the background.

⚠️ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.

docker compose -f agents/mcp_servers/deidentification/deid-service/docker-compose.yml up -d --build

Runs the command against the services defined in the compose file.

✅ Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

3. Maven (Java)

Medium

Prerequisites

Git Needed to download the project code from GitHub.
JDK (Java) Required to build and run Java projects.
Maven The build tool used for the mvn command.

⚠️ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.

cd legal-nlp/platforms/grpc/java

This project's files live in a subfolder, so move into it first.

mvn clean install

Installs dependencies and builds the project using Maven.

✅ A BUILD SUCCESS message means it worked. The output is created under target/.

4. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

$ python3 -m venv .sparknlp-env

Runs the Python script (or module).

$ pip install pyspark==3.1.2

Installs the Python libraries listed in requirements.txt (or similar).

$ pip install spark-nlp

Installs the Python libraries listed in requirements.txt (or similar).

<img src="https://static.pepy.tech/personalized-badge/spark-nlp?period=total&units=international_system&left_color=grey&right_color=orange&left_text=pip%20downloads" /></a>

Type this command into your terminal and run it.

✅ If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)