lang-comp

KO | EN

gitlite — search

#hacktoberfest #react #typescript #llm #golang #nodejs #macos #java #cli #kubernetes #vue #pytorch #security #openai

(★ 9)

Lang-comp is a benchmark suite for comparing the performance of AI coding agent effectivenes working in different programming languages

lang-comp 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Lang-comp

![approach](https://raw.githubusercontent.com/jovaneyck/lang-comp/main/assets/lang-comp-approach.png)

## Description

Lang-comp is a benchmark suite for comparing the performance of AI coding agent effectivenes working in different programming languages on a set of common programming tasks. The benchmark includes implementations of various algorithms and data structures, as well as a set of test cases to evaluate the performance of each implementation.

Main goal of the bench is to see how language affordances (e.g. static vs. dynamic typing) and availability in the training data affect token cost .

## Source code attribution

All source code used in this benchmark was adapted from the [exercism](https://github.com/exercism) project.

## Main harness

* These benchmarks use the Github copilot harness if not explicitly stated differently. Fresh context (system prompt, tool definitions etc): 15.200 tokens

## Benchmark results

![results](https://raw.githubusercontent.com/jovaneyck/lang-comp/main/assets/results.png)

lang-comp

lang-comp 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

lang-comp

lang-comp 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README