Uzu for 4D Version 20.0.5 Copy link to add as a dependency in 4D Uzu is a high-performance inference engine for AI models. It allows you to deploy AI directly in your app with zero latency, full data privacy, and no inference costs.
4D OxiBonsai Version 20.0.6 Copy link to add as a dependency in 4D oxibonsai is a pure Rust inference engine for PrismML’s Bonsai family.
4D ONNX Runtime GenAI Version 20.7.19 Copy link to add as a dependency in 4D ONNX Runtime GenAI is a library that provides optimised inference for generative AI models — particularly LLMs — built on top of the ONNX Runtime engine. This component provides a server CLI and a class-based generic API with callback functions to use ONNX Runtime GenAI from 4D.
4D LLaMA C++ Version 20.5.28 Copy link to add as a dependency in 4D llama.cpp is an open-source C/C++ library and CLI tool for running large language models (LLMs) locally on consumer hardware. This component provides a class-based generic API with callback functions to use llama.cpp from 4D.
