research-library

KO | EN

gitlite — search

#large-language-models #list #animation #ios #chatbot #kubernetes #nextjs #containers #cli #open-source

research-library

(★ 9)

A personal research library that runs on your laptop. An agent writes you a paper, the paper goes into Oracle AI Database 26ai, and the next paper is cheaper because you can ask the library what you already know. No cloud account, no API keys.

File Explorer

ZIP Download

- README.md
- SKILL.md
- the-model-context-protocol.md
- ai-vector-search-in-oracle-26ai.md
- governing-agents-in-the-enterprise.md
- procedural-memory-in-agent-harnesses.md
- skill-induction-from-agent-traces.md
- the-autonomous-agent-sandbox.md
- why-agents-forget.md
- - SKILL.md
- index-full.html
- index.html
- talk-track.md
- 01-schema.sql
- 02-onnx.sql
- 03-index.sql
- - stacks.css
- - index.html
  - raw.html
- app.py
- - summary.md
- fixture.json
- grade.py
- personas.json
- topics.json
.gitattributes
.gitignore
ingest.py
README.md
requirements.txt
search.py
setup.sh
SKILL.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

A personal research library that runs on your laptop. An agent writes you a paper, the paper
goes into your library, and the next paper is cheaper to produce because you can ask the library
what you already know.

Everything runs locally: Oracle AI Database 26ai Free in a container, an embedding model loaded
into the database, and a small Flask front end. There is no cloud account, no API key and
nothing that costs money. The embeddings are generated inside the database, so the text you
search never leaves the machine.

```
About ten minutes, one command, and a library you can ask questions of.
```

## What you need

- A container runtime: Docker Desktop on macOS or Windows, Docker Engine or Podman on Linux.
- About 4GB of free memory. The database is capped at 2 CPU and 2GB inside the container.
- Python 3.10 to 3.13. Not 3.14, which is ahead of the driver.
- Around 4GB of disk. The database image is 1.59GB and the embedding model is 130MB.
- A bash shell to run `setup.sh` in. macOS and Linux have one. **Windows does not**: read
  [On Windows](#on-windows) before you start, because neither PowerShell nor `cmd` can run this
  script.

Apple Silicon is native. Do not add `--platform linux/amd64`.

## Quick start

```bash
git clone <this repo>
cd research-library
./setup.sh
```

Nine stages, each announced as it runs. The two slow ones are the 1.59GB database image and the
130MB embedding model, and both are cached, so a second run takes about a minute. When it finishes, a browser
opens on a library with six papers in it. That is the test: if you can see six papers, it worked.

On Windows this is `bash setup.sh` from Git Bash or WSL2, never `./setup.sh` in PowerShell. See
[On Windows](#on-windows).

To start again from nothing:

```bash
./setup.sh --fresh
```

## Using it

Every command below calls the interpreter inside the project's own virtual environment rather
than the system Python, because that is the one with `oracledb` in it. The path to it differs by
platform and there is no way around that:

| Platform | Interpreter |
|---|---|
| macOS, Linux, WSL2 | `.venv/bin/python` |
| Windows, PowerShell or `cmd` | `.venv\Scripts\python.exe` |
| Windows, Git Bash | `.venv/Scripts/python.exe` |

Ask a question from the terminal:

```bash
.venv/bin/python search.py "what did I already learn about forgetting"
```

```powershell
.venv\Scripts\python.exe search.py "what did I already learn about forgetting"
```

The top result is about context windows and desks, and it does not contain the word you searched
for. That is the whole argument for storing papers this way rather than in a folder.

Add a paper:

```bash
.venv/bin/python ingest.py papers/your-paper.md --topic agent-memory
```

The topic can also live in the markdown front matter as `topic: agent-memory`. Paragraphs shorter
than 200 characters are skipped, and long ones are split on sentence boundaries so that no chunk
is longer than the model can read.

Open Stacks:

```bash
.venv/bin/python stacks/app.py          # http://localhost:5050
STACKS_PORT=5051 .venv/bin/python stacks/app.py
```

```powershell
.venv\Scripts\python.exe stacks\app.py                        # http://localhost:5050
$env:STACKS_PORT=5051; .venv\Scripts\python.exe stacks\app.py
```

Port 5050 rather than 5000, because AirPlay Receiver holds 5000 on macOS. On Windows 5000 is
usually free, so that reason does not apply there, but the port stays 5050 so that everyone in
the room is typing the same URL. If 5050 itself is refused on Windows, it is almost never another
program: see [On Windows](#on-windows).

**Search** ranks chunks and prints the cosine distance on every hit. Smaller is closer.
**Is this new?** runs the same query and the same vector, and answers a different question:
whether the library already covers this. Two bands, and the nearest distance is always shown.
Under 0.45 is "You have this"; at or above is "Nothing close".

Read that second answer carefully, because distance compares wording as much as meaning. Ask it
about "forgetting" and it scores 0.584 and says nothing is close, even though three papers here
discuss exactly that, because the corpus says "forget" and never "forgetting". Query length is
not the variable: one word can score worse than a fourteen-word question. That is why there are
two bands rather than three, and why the matches are always shown underneath. The number is
evidence, not a verdict.

**What the database sees** is the second tab. Row counts, the model name, the index type, a
truncated embedding on every row, and the audit log underneath. Nothing on it is hardcoded.

## The manual path

If you would rather not run `setup.sh`, or you want to see what it does, here is the whole thing
by hand. Substitute `podman` for `docker` if that is what you
have; both work.

```bash
# 1. the database. This is the 1.59GB lite tag, the same one setup.sh uses.
#    :latest is the full 9.49GB image and pulls for about nine minutes.
docker run -d --name oracle26ai -p 1521:1521 \
  -e ORACLE_PWD=Research__2026 \
  -v oracle-research-data:/opt/oracle/oradata \
  container-registry.oracle.com/database/free:23.26.1.0-lite

# wait until the logs say DATABASE IS READY TO USE!
docker logs -f oracle26ai

# 2. the schema, and the vector memory setting the index needs
docker exec -i oracle26ai mkdir -p /opt/oracle/vec_dump /opt/oracle/rl-sql
docker cp sql/. oracle26ai:/opt/oracle/rl-sql/
docker exec -i oracle26ai sqlplus -s "/ as sysdba" \
  @/opt/oracle/rl-sql/01-schema.sql Research__2026

# 3. restart, because vector memory is reserved at startup
docker restart oracle26ai

# 4. the embedding model, 130MB, a few minutes.
#    onnx/ is gitignored, so it does not exist in a fresh clone. Make it first.
mkdir -p onnx
curl -fL -o onnx/all_MiniLM_L12_v2.onnx \
  https://objectstorage.us-ashburn-1.oraclecloud.com/n/adwc4pm/b/OML-Resources/o/all_MiniLM_L12_v2.onnx
docker cp onnx/all_MiniLM_L12_v2.onnx oracle26ai:/opt/oracle/vec_dump/
docker exec -i oracle26ai sqlplus -s research/Research__2026@localhost:1521/FREEPDB1 \
  @/opt/oracle/rl-sql/02-onnx.sql

# 5. the index
docker exec -i oracle26ai sqlplus -s research/Research__2026@localhost:1521/FREEPDB1 \
  @/opt/oracle/rl-sql/03-index.sql

# 6. python, papers, front end
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
for f in papers/*.md; do .venv/bin/python ingest.py "$f"; done
.venv/bin/python stacks/app.py
```

Step 6 is bash. In PowerShell it is:

```powershell
python -m venv .venv
.venv\Scripts\pip.exe install -r requirements.txt
Get-ChildItem papers\*.md | ForEach-Object { .venv\Scripts\python.exe ingest.py $_.FullName }
.venv\Scripts\python.exe stacks\app.py
```

### On Linux

The same commands work with Docker Engine or Podman. Two differences worth knowing:

- Rootless Podman maps ports above 1024 without help, and 1521 is above 1024, so nothing extra
  is needed. If you are using rootful Docker you may need `sudo` on every `docker` command.
- `xdg-open` is used to open the browser at the end. If it is missing, `setup.sh` finishes
  normally and you open `http://localhost:5050/` yourself.

SELinux systems sometimes refuse the volume mount. Add `:Z` to it:
`-v oracle-research-data:/opt/oracle/oradata:Z`.

## On Windows

`setup.sh` is a bash script. **PowerShell and `cmd` cannot run it**, and there is no PowerShell
port of it in this repository. You need a bash, and there are two, and one of them is clearly
better.

### WSL2, which is the recommended path

```powershell
wsl --install                     # once, then reboot if it asks
```

Then inside the Ubuntu window:

```bash
sudo apt update && sudo apt install -y python3-venv git
git clone <this repo>
cd research-library                # clone into the Linux home, not /mnt/c
./setup.sh
```

Everything in this README then works exactly as written, because WSL2 is Linux. Four things to
get right:

- **Clone into the Linux filesystem** (`~/research-library`), not `/mnt/c/Users/...`. Building on
  the Windows drive from WSL is slow enough to be visible on a call, and the executable bit on
  `setup.sh` does not survive there.
- **Turn on Docker Desktop's WSL integration**: Settings, Resources, WSL Integration, tick your
  distribution. Without it `docker` is not on the PATH inside WSL.
- **No browser opens at the end.** `xdg-open` is not there; `setup.sh` finishes normally and you
  open `http://localhost:5050/` in Windows yourself. `sudo apt install wslu` gives you `wslview`
  if you want the automatic open back.
- **Memory.** WSL2 takes half the machine's RAM by default, which is fine on 16GB. If a
  `%USERPROFILE%\.wslconfig` caps it lower, the database will not open. Raise it to at least 6GB.

### Git Bash, which works but has sharp edges

```
"C:\Program Files\Git\bin\bash.exe" setup.sh
```

or open Git Bash in the folder and run `bash setup.sh`. It works, and `setup.sh` now handles the
three things that used to break it, but know what they were:

- **Path mangling.** Git Bash rewrites any argument that looks like a Unix path before handing it
  to `docker.exe`, so `docker exec ... mkdir -p /opt/oracle/vec_dump` arrives inside the container
  as `C:/Program Files/Git/opt/oracle/vec_dump`, and `sqlplus "/ as sysdba"` arrives as
  `C:/Program Files/Git/ as sysdba`. `setup.sh` exports `MSYS_NO_PATHCONV=1` to switch it off. If
  you run any `docker` command by hand from Git Bash, put `MSYS_NO_PATHCONV=1` in front of it.
- **`python3` does not exist.** The python.org installer ships `python.exe` and no `python3.exe`,
  and typing `python3` on a stock Windows opens the Microsoft Store instead of running anything.
  Install Python 3.12 from python.org with **Add python.exe to PATH** ticked.
- **`.venv/bin/python` does not exist either.** On Windows a virtual environment puts its
  interpreter in `.venv/Scripts/python.exe`. `setup.sh` detects which one it made; every command
  you type by hand afterwards has to use the `Scripts` path.
- **ctrl-c does not always reach `python.exe`.** If a later run says port 5050 is in use, stop the
  leftover with `taskkill //F //IM python.exe` (two slashes, because Git Bash eats one).

### Ports on Windows

`-p 1521:1521` publishes on `localhost` under Docker Desktop exactly as it does on macOS, so
`localhost:1521/FREEPDB1` is the right DSN on both.

Two Windows-only clashes are worth knowing before they happen:

- **1521.** If this machine has ever had an Oracle client, XE, or a local listener installed, that
  listener owns 1521 and `docker run` fails with a bind error. `Get-Service | findstr -i oracle`
  finds it. Stop it, or move the published port with `-p 1522:1521` and set
  `RESEARCH_DSN=localhost:1522/FREEPDB1`.
- **5050, or any port, refused with "an attempt was made to access a socket in a way forbidden by
  its access permissions".** That is not another program. Hyper-V and WSL reserve blocks of ports
  at boot, and if 5050 falls inside one, nothing can bind it. List the blocks with:

```powershell
  netsh interface ipv4 show excludedportrange protocol=tcp
  ```

Then pick a port outside them: `STACKS_PORT=5051 ./setup.sh`.

### What is not tested

This repository has been built and run on macOS with Podman. The Windows paths above are
reasoned from the code and from how Docker Desktop and Git Bash behave, not measured on a Windows
machine. If you are about to demonstrate this live on Windows, do a full `./setup.sh --fresh` on
that machine first.

## How it works

One table:

```sql
CREATE TABLE papers (
  id          NUMBER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  title       VARCHAR2(300)  NOT NULL,
  topic       VARCHAR2(80)   NOT NULL,
  source_file VARCHAR2(300)  NOT NULL,
  chunk_ix    NUMBER         NOT NULL,
  chunk_text  CLOB           NOT NULL,
  embedding   VECTOR(384, FLOAT32),
  ingested_at TIMESTAMP DEFAULT SYSTIMESTAMP
);
```

One insert, which is the line the whole thing turns on:

```sql
INSERT INTO papers (title, topic, source_file, chunk_ix, chunk_text, embedding)
VALUES (:title, :topic, :source, :ix, :body,
        VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :body AS data));
```

The text and its vector are written in the same statement, so a row cannot exist with one and
not the other. One search:

```sql
SELECT title, chunk_text,
       VECTOR_DISTANCE(embedding,
                       VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :q AS data),
                       COSINE) AS distance
FROM   papers
ORDER  BY distance
FETCH FIRST 8 ROWS ONLY;
```

That is the entire architecture. There is no vector library, no embedding service, no agent
framework and no queue.

## The seed papers

Six papers ship with the library so that search works before you have written anything:

| Paper | Topic |
|---|---|
| Why Agents Forget | agent-memory |
| Procedural Memory in Agent Harnesses | agent-memory |
| Skill Induction from Agent Traces | agent-memory |
| The Autonomous Agent Sandbox | harness |
| AI Vector Search in Oracle 26ai | databases |
| Governing Agents in the Enterprise | governance |

Delete them once you have your own. `./setup.sh --fresh` puts them back.

## Two obvious next steps

Neither is built, both are about an hour, and both are drawn in the design mockup.

**Neighbours.** Paper-level similarity rather than chunk-level: average the vectors per paper,
or take the minimum pairwise distance, and show the three nearest papers under the top hit. It
answers "you have adjacent material you had forgotten about", which is the moment a folder of
markdown stops being enough.

**Recall.** Pull one chunk from something ingested more than a week ago, show the question it
answers, and hide the answer behind a reveal. Spaced repetition over your own library, which is
a different reason to keep one.

## Licence and credits

The embedding model is Oracle's augmented ONNX export of
[all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2), Apache 2.0.
Oracle AI Database 26ai Free is free to use under Oracle's Free Developer Licence, and the image
is on
[container-registry.oracle.com](https://container-registry.oracle.com/ords/ocr/ba/database/free).
Vector search reference:
[Oracle AI Vector Search documentation](https://docs.oracle.com/en/database/oracle/oracle-database/26/vecse/).

The database password in this repository is a local throwaway for a container on your own
machine. Change `RESEARCH_DB_PASSWORD` if you put this anywhere else.

research-library

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

research-library

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README