YOLOs-CPP-TensorRT
YOLOs-TRT is a header-only C++ library for running all YOLO models with all tasks with NVIDIA TensorRT on CUDA GPUs and Jetson. It features GPU preprocessing (letterbox/normalize/HWC→NCHW), CUDA Graph replay, FP16/INT8 support, and sub-2ms end-to-end latency with 530+fps.
File Explorer
- UbuntuGPU.Dockerfile
- main.yml
- .gitkeep
- benchmark_results.csv
- unified_benchmark_1768399342.csv
- auto_bench.sh
- CMakeLists.txt
- README.md
- yolo_unified_benchmark.cpp
- cover.png
- dog.jpg
- acknowledgments.md
- contributing.md
- development.md
- installation.md
- models.md
- quantization.md
- usage.md
- CMakeLists.txt
- example_camera_class.cpp
- example_camera_det.cpp
- example_camera_obb.cpp
- example_camera_pose.cpp
- example_camera_seg.cpp
- example_image_class.cpp
- example_image_depth.cpp
- example_image_det.cpp
- example_image_obb.cpp
- example_image_pose.cpp
- example_image_seg.cpp
- example_video_class.cpp
- example_video_det.cpp
- example_video_obb.cpp
- example_video_pose.cpp
- example_video_seg.cpp
- README.md
- run_camera_class.sh
- run_camera_det.sh
- run_camera_obb.sh
- run_camera_pose.sh
- run_camera_seg.sh
- run_image_class.sh
- run_image_depth.sh
- run_image_det.sh
- run_image_obb.sh
- run_image_pose.sh
- run_image_seg.sh
- run_video_class.sh
- run_video_det.sh
- run_video_obb.sh
- run_video_pose.sh
- run_video_seg.sh
- utils.hpp
- BoundedThreadSafeQueue.hpp
- Config.hpp
- cuda_preprocessing.cu
- cuda_preprocessing.hpp
- drawing.hpp
- nms.hpp
- preprocessing.hpp
- trt_session_base.hpp
- trt_utils.hpp
- types.hpp
- utils.hpp
- version.hpp
- classification.hpp
- depth.hpp
- detection.hpp
- obb.hpp
- pose.hpp
- segmentation.hpp
- yolos.hpp
- coco.names
- Dota.names
- export_onnx.py
- ImageNet.names
- imagenet_classes.txt
- .gitkeep
- prepare_release.sh
- batch_image_inference.cpp
- camera_inference.cpp
- class_image_inference.cpp
- depth_image_inference.cpp
- image_inference.cpp
- video_inference.cpp
- .gitkeep
- dog_bike_car.jpg
- .gitkeep
- .gitkeep
- download_test_models.sh
- export_onnx_yoloxx.py
- fintune_yoloxx.py
- labels.names
- results_cpp.json
- results_ultralytics.json
- .gitkeep
- compare_results.cpp
- inference_classification_cpp.cpp
- inference_classification_ultralytics.py
- requirements.txt
- .gitkeep
- .gitkeep
- .gitkeep
- .gitkeep
- compare_results.cpp
- inference_depth_cpp.cpp
- inference_depth_ultralytics.py
- requirements.txt
- .gitkeep
- dog_bike_car.jpg
- kitti_overlap.png
- .gitkeep
- .gitkeep
- .gitkeep
- download_test_models.sh
- export_onnx_yoloxx.py
- fintune_yoloxx.py
- voc.names
- results_cpp.json
- results_ultralytics.json
- compare_results.cpp
- inference_config.json
- inference_detection_cpp.cpp
- inference_detection_ultralytics.py
- requirements.txt
- image.png
- Dota.names
- download_test_models_obb.sh
- export_onnx_yolo_obb.py
- finetune_yolo_obb.py
- results_cpp.json
- results_ultralytics.json
- .gitkeep
- compare_obb_results.cpp
- compare_obb_results.py
- inference_config_obb.json
- inference_obb_cpp.cpp
- inference_obb_ultralytics.py
- test1.jpg
- test2.jpg
- coco.names
- download_test_models.sh
- export_onnx_yoloxx.py
- results_cpp.json
- results_ultralytics.json
- .gitkeep
- compare_results.cpp
- inference_config.json
- inference_pose_cpp.cpp
- inference_pose_ultralytics.py
- requirements.txt
- .gitkeep
- bus.jpg
- kitti_overlap.png
- .gitkeep
- .gitkeep
- .gitkeep
- coco.names
- download_test_models.sh
- export_onnx_yoloxx.py
- yolo11s-seg_car_peds_mask.png
- yolo11s-seg_kitti_overlap_mask.png
- yolo26n-seg_car_peds_mask.png
- yolo26n-seg_kitti_overlap_mask.png
- yolov8s-seg_car_peds_mask.png
- yolov8s-seg_kitti_overlap_mask.png
- yolov9c-seg_car_peds_mask.png
- yolov9c-seg_kitti_overlap_mask.png
- yolo11s-seg_car_peds_mask.png
- yolo11s-seg_kitti_overlap_mask.png
- yolo26n-seg_car_peds_mask.png
- yolo26n-seg_kitti_overlap_mask.png
- yolov8s-seg_car_peds_mask.png
- yolov8s-seg_kitti_overlap_mask.png
- yolov9c-seg_car_peds_mask.png
- yolov9c-seg_kitti_overlap_mask.png
- results_cpp.json
- results_ultralytics.json
- .gitkeep
- compare_results.cpp
- inference_config.json
- inference_segmentation_cpp.cpp
- inference_segmentation_ultralytics.py
- requirements.txt
- build_test.sh
- CMakeLists.txt
- README.md
- test_all.sh
- test_classification.sh
- test_depth.sh
- test_detection.sh
- test_obb.sh
- test_pose.sh
- test_segmentation.sh
- test_utils.sh
- onnx2trt.cpp
- calibration_0000.jpg
- calibration_0001.jpg
- calibration_0002.jpg
- calibration_0003.jpg
- calibration_0004.jpg
- calibration_0005.jpg
- calibration_0006.jpg
- calibration_0007.jpg
- calibration_0008.jpg
- calibration_0009.jpg
- calibration_0010.jpg
- calibration_0011.jpg
- calibration_0012.jpg
- calibration_0013.jpg
- calibration_0014.jpg
- calibration_0015.jpg
- calibration_0016.jpg
- calibration_0017.jpg
- calibration_0018.jpg
- calibration_0019.jpg
- calibration_0020.jpg
- calibration_0021.jpg
- calibration_0022.jpg
- calibration_0023.jpg
- calibration_0024.jpg
- calibration_0025.jpg
- calibration_0026.jpg
- calibration_0027.jpg
- calibration_0028.jpg
- calibration_0029.jpg
- calibration_0030.jpg
- calibration_0031.jpg
- calibration_0032.jpg
- calibration_0033.jpg
- calibration_0034.jpg
- calibration_0035.jpg
- calibration_0036.jpg
- calibration_0037.jpg
- calibration_0038.jpg
- calibration_0039.jpg
- calibration_0040.jpg
- calibration_0041.jpg
- calibration_0042.jpg
- calibration_0043.jpg
- calibration_0044.jpg
- calibration_0045.jpg
- calibration_0046.jpg
- calibration_0047.jpg
- calibration_0048.jpg
- calibration_0049.jpg
- ACCURACY_IMPROVEMENT_SOLUTION.md
- compare_all_precisions.py
- compare_models.py
- convert_models.sh
- convert_to_tensorrt.py
- deep_engine_analysis.py
- example_usage.py
- FP16_FIX_SUMMARY.md
- FP16_ISSUE_ANALYSIS.md
- generate_calibration_data.py
- improve_int8_accuracy.py
- INT8_ACCURACY_FIX.md
- INT8_IMPLEMENTATION_GUIDE.md
- quick_int8_test.py
- README_convert_to_tensorrt.md
- test_int8_accuracy.py
- COMPREHENSIVE_TENSORRT_GUIDE.md
- DOCUMENTATION_INDEX.md
- FINAL_SUMMARY.md
- QUICK_REFERENCE.md
- README_SCRIPTS.md
- TENSORRT_CONVERSION_ENHANCEMENT_SUMMARY.md
- USER_GUIDE_EXPORT_TO_TENSORRT.md
- .dockerignore
- .gitignore
- build.sh
- CHANGELOG.md
- CMakeLists.txt
- Dockerfile.tensorrt
- Dockerfile.tensorrt.jetson
- Dockerize.md
- LICENSE
- pyproject.toml
- README.md
- RELEASE.md
- requirements.txt
- run_camera.sh
- run_image.sh
- run_test.sh
- run_video.sh
# Use via CDN
jsDelivrjsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.
Command Glossary
Commands referenced in this DOCs, explained below.
cmake
View Details ▼
cmake
Cross-platform build automation system, that generates recipes for native build systems.
cmake {{path/to/project_directory}}
Generate a build recipe in the current directory with `CMakeLists.txt` from a project directory:
cmake --build {{path/to/build_directory}}
Use a generated recipe in a given directory to build artifacts:
cmake --install {{path/to/build_directory}} --strip
Install the build artifacts into `/usr/local/` and strip debugging symbols:
docker build
View Details ▼
docker build
Build an image from a Dockerfile.
docker build .
Build a Docker image using the Dockerfile in the current directory:
docker build {{github.com/creack/docker-firefox}}
Build a Docker image from a Dockerfile at a specified URL:
docker build {{[-t|--tag]}} {{name:tag}} .
Build a Docker image and tag it:
docker run
View Details ▼
docker run
This command is an alias of `docker container run`.
tldr docker container run
View documentation for the original command:
git checkout
View Details ▼
git checkout
Checkout a branch or paths to the working tree.
git checkout -b {{branch_name}}
Create and switch to a new branch:
git checkout -b {{branch_name}} {{reference}}
Create and switch to a new branch based on a specific reference (branch, remote/branch, tag are examples of valid references):
git checkout {{branch_name}}
Switch to an existing local branch:
git clone
View Details ▼
git clone
Clone an existing repository.
git clone {{remote_repository_location}} {{path/to/directory}}
Clone an existing repository into a new directory (the default directory is the repository name):
git clone --recursive {{remote_repository_location}}
Clone an existing repository and its submodules:
git clone {{[-n|--no-checkout]}} {{remote_repository_location}}
Clone only the `.git` directory of an existing repository:
pip install
View Details ▼
pip install
Install Python packages.
pip install {{package1 package2 ...}}
Install one or more packages:
pip install {{package1 package2 ...}} {{[-U|--upgrade]}}
Upgrade all specified packages to the latest version, installing any that are not already present:
pip install {{package}}=={{version}}
Install a specific version of a package:
python
View Details ▼
python
Python language interpreter.
python
Start a REPL (interactive shell):
python {{path/to/file.py}}
Execute a specific Python file:
python -i {{path/to/file.py}}
Execute a specific Python file and start a REPL:
apt
View Details ▼
apt
Package manager for Debian-based distributions.
Intended as a user-friendly alternative to `apt-get` for interactive use.
For equivalent commands in other package managers, see <https://wiki.archlinux.org/title/Pacman/Rosetta>.
sudo apt update
Update the list of available packages and versions (recommended before running other `apt` commands):
apt search {{package}}
Search packages by name or description:
apt list {{package}}
Search packages by name only (supports wildcards like `*`):
dpkg
View Details ▼
dpkg
Debian package manager.
Some subcommands such as `deb` have their own usage documentation.
For equivalent commands in other package managers, see <https://wiki.archlinux.org/title/Pacman/Rosetta>.
sudo dpkg {{[-i|--install]}} {{path/to/file.deb}}
Install a package:
sudo dpkg {{[-r|--remove]}} {{package}}
Remove a package:
dpkg {{[-l|--list]}} {{pattern}}
List installed packages:
