Skip to content

MLECO-6304: Support multiple concurrent LLM instances in JNI layer.

Jerry Collins requested to merge feature/multi-java-llm-support into main
  • Native LLM instance is now tracked in the Java LLM object. Handles are used in the Java LLM instance not pointers. Multiple LLM objects can now be created that point to their own native LLM object.

Limitations: Currently the build process only builds/links one LLM backend (e.g. llama.cpp / ONNX) so only one type of LLM backend is supported at runtime.

Change-Id: Iaeee9649c23bc4d1038e6c726779d5040e545ec7 Signed-off-by: Jerry Collins jerry.collins@arm.com

Merge request reports