exmld

KO | EN

gitlite — search

#ai #typescript #go #golang #android #machine-learning #deep-learning #macos #nodejs #awesome #kubernetes #vue #ios #openai

(★ 44)

elixir interface to kinesis client library via MultiLangDaemon

exmld 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# exmld

This application allows Kinesis and DynamoDB streams to be processed using Elixir or
Erlang (by way of the KCL MultiLangDaemon).  It's particularly useful when aggregate
records are being used and items can be processed in approximate order (as opposed to
strict order within each shard), but that isn't a requirement.

Using [erlmld](https://github.com/AdRoll/erlmld), a normal Erlang Kinesis processing
application looks like this:

![Erlang - MultiLangDaemon processing](https://raw.githubusercontent.com/AdRoll/exmld/main/img/erlang-mld-workers.png)

Using this Elixir library (which uses erlmld), a processing application looks like this:

![Elixir - MultiLangDaemon processing](https://raw.githubusercontent.com/AdRoll/exmld/main/img/elixir-mld-pipeline.png)

This is done using the [Flow](https://hexdocs.pm/flow/Flow.html) framework to set up a
MapReduce-style processing pipeline within a single BEAM node.

By virtue of using the KCL, processing applications can horizontally scale across a group
of ([homogenous](https://github.com/awslabs/amazon-kinesis-client/issues/103)) worker
instances.

Unlike most applications using the KCL's MultiLangDaemon, an Erlang or Elixir processing
application using this library can easily make full use of a worker's processing power
(even if the stream contains a single shard) due to use of the Flow framework.

# Examples

See:

1. [example erlang processor](examples/erlang_processor/)
  2. [example elixir processor](examples/elixir_processor/)

exmld

exmld 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

exmld

exmld 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README