pastila

KO | EN

gitlite — search

#python #react #typescript #llm #go #machine-learning #linux #nodejs #macos #pytorch #security #mcp #openai #open-source

(★ 85)

Paste toy-service on top of ClickHouse

파일 탐색기

최종 버전 다운로드 (.zip)

CNAME
index.html
LICENSE
pastila.py
README.md
session.html

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Paste Service On Top Of ClickHouse

https://pastila.nl/

## About

[Read the blog post](https://clickhouse.com/blog/building-a-paste-service-with-clickhouse/).

This service is developed to demonstrate some features of ClickHouse
like asynchronous INSERTs, direct querying from HTML page, MATERIALIZED and EPHEMERAL columns,
CONSTRAINTS, custom HTTP handlers, quotas and user access control, etc.
in a toy service similar to "pastebin" or "gist".

Features:
- data is instantly saved in ClickHouse after pasting or editing - single page with no "save" button;
- after data is saved or loaded, you see the check mark in the bottom right corner;
- just copy the link from the address bar - it is a permanent link to the data;
- you can edit already saved data but the new link will be created and the old links remain permanent;
- you can host Markdown pages - just add `.md` to the URL before the `#` anchor and share it;
- you can host compressed content - to decompress on rendering add `.gz` to the URL before the `#` anchor and share it;
- you can host HTML pages - just add `.html` to the URL before the `#` anchor and share it;
  this is very insecure, see "open redirect";
  append `&sandbox` to the anchor to render without a confirmation dialog in an opaque-origin sandbox;
  scripts, forms, and top-level navigation are disabled in this mode;
- you can host a link shortener - just add `.link` to the URL before the `#` anchor and share it;
- you can view and share Claude Code sessions - just add `.claude.jsonl` to the URL before the `#` anchor;
  the session (JSONL, one record per line) is rendered like the viewer at https://github.com/ClickHouse/alexeyprompts;
- you can host terminal output with colors - just add `.terminal` to the URL before the `#` anchor and share it;
  ANSI/VT100 escape sequences (colors, bold, underline, etc.) are rendered on a black background like a real terminal;
  this also works for logs copied from `less` or `cat -v`, where ESC appears as literal `^[`;
- browser history is used while editing for easy undo;
- edit history is also saved in the database: after you load the data,
  previous version of the data is available by clicking the "back" button in bottom right corner;
- Added Encryption option to encrypt data in browser before inserting into ClickHouse database,
  encryption key is kept in anchor tag which never leaves the user's browser.

## Motivation

ClickHouse is analytical database, and the usage scenario of this service
is definitely not the most natural for ClickHouse.
Nevertheless, it is always important to test the system in corner cases
and unusual scenarios to find potential flaws and to explore new possibilities.

## Contributing

Please send a pull request to
https://github.com/ClickHouse/pastila

## Terms Of Use

This service can be used for helping ClickHouse development.
No other usages permitted.

## Warranty

No warranties or conditions of any kind, either express or implied.
The data can be removed or lost at any moment of time.

## Security And Privacy

This service may not provide any security or privacy.

## Cookie Policy

This service does not use cookies.

## Report Abuse

To report abuse, write to feedback@clickhouse.com. DMCA violations should be reported as well.

## Installation

```
CREATE DATABASE paste ENGINE = Replicated('/clickhouse/databases/paste/', '{shard}', '{replica}');

CREATE TABLE paste.data
(
    fingerprint UInt32 DEFAULT reinterpretAsUInt32(unhex(fingerprint_hex)),
    hash UInt128 DEFAULT reinterpretAsUInt128(unhex(hash_hex)),
    prev_fingerprint UInt32 DEFAULT reinterpretAsUInt32(unhex(prev_fingerprint_hex)),
    prev_hash UInt128 DEFAULT reinterpretAsUInt128(unhex(prev_hash_hex)),
    content String,

size UInt32 MATERIALIZED length(content),
    time DateTime64 MATERIALIZED now64(),
    query_id String MATERIALIZED queryID(),

fingerprint_hex String EPHEMERAL '',
    hash_hex String EPHEMERAL '',
    prev_fingerprint_hex String EPHEMERAL '',
    prev_hash_hex String EPHEMERAL '',
    is_encrypted UInt8,

CONSTRAINT length CHECK length(content) < 50 * 1024 * 1024,
    CONSTRAINT hash_is_correct CHECK sipHash128(content) = reinterpretAsFixedString(hash),
    CONSTRAINT not_uniform_random CHECK length(content) < 10000 OR arrayReduce('entropy', extractAll(content, '.')) < 7,
    CONSTRAINT not_constant CHECK length(content) < 10000 OR arrayReduce('uniqUpTo(1)', extractAll(content, '.')) > 1,

PRIMARY KEY (fingerprint, hash)
)
ENGINE = ReplicatedMergeTree;

CREATE USER paste IDENTIFIED WITH no_password
DEFAULT DATABASE paste
SETTINGS
    add_http_cors_header = 1 READONLY,
    async_insert = 1 READONLY,
    wait_for_async_insert = 1 READONLY,
    limit = 1 READONLY,
    offset = 0 READONLY,
    max_result_rows = 1 READONLY,
    force_primary_key = 1 READONLY,
    max_query_size = '10M' READONLY;

-- internal only user, no need to allow access from anywhere
CREATE USER paste_sys HOST LOCAL IDENTIFIED WITH no_password
DEFAULT DATABASE paste
SETTINGS
    add_http_cors_header = 1 READONLY,
    async_insert = 1 READONLY,
    wait_for_async_insert = 1 READONLY,
    limit = 1 READONLY,
    offset = 0 READONLY,
    max_result_rows = 1 READONLY,
    force_primary_key = 1 READONLY,
    max_query_size = '10M' READONLY;

CREATE QUOTA paste
KEYED BY ip_address
FOR RANDOMIZED INTERVAL 1 MINUTE MAX query_selects = 100, query_inserts = 1000,
FOR RANDOMIZED INTERVAL 1 HOUR MAX query_selects = 1000, query_inserts = 10000,
FOR RANDOMIZED INTERVAL 1 DAY MAX query_selects = 5000, query_inserts = 50000, written_bytes = '500M'
TO paste;

CREATE VIEW paste.data_view DEFINER = 'paste_sys' AS SELECT * FROM paste.data WHERE fingerprint = reinterpretAsUInt32(unhex({fingerprint:String})) AND hash = reinterpretAsUInt128(unhex({hash:String})) ORDER BY time LIMIT 1;

GRANT INSERT ON paste.data TO paste;
GRANT SELECT ON paste.data_view TO paste;

GRANT SELECT ON paste.data TO paste_sys;
```

pastila

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

pastila

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README