HaluMem
HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.
File Explorer
- Framework.png
- HaluMem_EvalF.png
- HaluMem_vs_ExistHaluMem.png
- radar_questions.png
- eli5_messages.jsonl
- native_math_messages_sample.jsonl
- self_gene_unrelated_qa.jsonl
- stage0_persona.jsonl
- stage1_1_fixed.jsonl
- stage1_2_dynamic.jsonl
- stage1_3_preferences.jsonl
- stage2_1_extend_dynamic.jsonl
- stage2_2_extend_preference.jsonl
- stage2_3_profile2skeleton.jsonl
- stage3_1_narrative_arc.jsonl
- stage3_2_preference_events.jsonl
- stage3_3_init_events.jsonl
- stage3_4_merged_events.jsonl
- stage4_1_events2memories.jsonl
- stage5_1_dialogue_generation.jsonl
- stage5_2_token_calculation.jsonl
- .env-example
- eval_memobase.py
- eval_memos.py
- eval_memzero.py
- eval_memzero_graph.py
- eval_supermemory.py
- eval_tools.py
- eval_zep.py
- evaluation.py
- llms.py
- prompts.py
- README.md
- stage1_1_fixed_validation_prompt.txt
- stage1_2_dynamic_validation_prompt.txt
- stage1_3_preferences_validation_prompt.txt
- stage2_3_profile2skeleton_prompt.txt
- stage3_1_skeleton2core.txt
- stage3_2_skeleton2routine.txt
- stage4_1_events2memories.txt
- stage4_1_interference_memory_generation.txt
- stage5_1_dialogue_generation.txt
- stage5_1_dialogue_generation_with_interference.txt
- stage5_1_interference_memory_generation.txt
- stage5_1_memory_validation.txt
- stage6_1_question_generation.txt
- utils_dynamic_conversation_template.txt
- utils_fixed_conversation_template.txt
- utils_preferences_conversation_template.txt
- eli5_parquet_to_jsonl.py
- native_math_jsonl_to_messages_sample.py
- unrelated_dialogue_generate.py
- fix_token_cost.py
- llm_request.py
- parallel_utils.py
- run_all_stages.py
- stage1_1_fixed.py
- stage1_2_dynamic.py
- stage1_3_preferences.py
- stage2_1_extend_dynamic.py
- stage2_2_extend_preference.py
- stage2_3_profile2skeleton.py
- stage3_1_skeleton2core.py
- stage3_2_skeleton2routine.py
- stage3_3_init2events.py
- stage3_4_merge_all.py
- stage4_1_events2memories.py
- stage5_1_dialogue_generation.py
- stage5_2_token_calculation.py
- stage6_1_question_generation.py
- stage6_2_final_result_medium.py
- stage6_3_final_result_long.py
- utils.py
- .env copy
- .gitignore
- config.json
- poetry.lock
- pyproject.toml
- README.md
# Use via CDN
jsDelivrjsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.
Command Glossary
Commands referenced in this DOCs, explained below.
eval
View Details ▼
eval
Execute arguments as a single command in the current shell and return its result.
eval "{{echo foo}}"
Call `echo` with the "foo" argument:
eval "{{foo=bar}}"
Set a variable in the current shell:
poetry install
View Details ▼
poetry install
Install all dependencies for a Python project as defined in the pyproject.toml file.
poetry install
Install dependencies:
poetry install --no-root
Skip installing the project itself as a dependency:
poetry install --without dev
Install only production dependencies:
python
View Details ▼
python
Python language interpreter.
python
Start a REPL (interactive shell):
python {{path/to/file.py}}
Execute a specific Python file:
python -i {{path/to/file.py}}
Execute a specific Python file and start a REPL:
uuid
View Details ▼
uuid
Generate and decode Universally Unique Identifiers (UUID).
See also: `uuidgen`.
uuid
Generate a UUIDv1 (based on time and system's hardware address, if present):
uuid -v 4
Generate a UUIDv4 (based on random data):
uuid -v 4 -n {{number_of_uuids}}
Generate multiple UUIDv4 identifiers at once:
