Commits · v0.5.0 · Kleidi / KleidiAI

Dec 02, 2024

Use the same struct for int8 matmul with static and dynamic quantization · 7e8c4baf
Gian Marco Iodice authored 1 month ago and Jakub Sujak committed 1 month ago
```
Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>
```
2 tags

7e8c4baf

Optimize F32 <- QSI8D32 (LHS) x QSI4C32 (RHS) for SME · ba20def9

Gian Marco Iodice authored 1 month ago and

Jakub Sujak committed 1 month ago


- GEMM and GEMV Micro-kernels to compute the matrix multiplication of dynamically quantized symmetric signed 8-bit integer with per-block quantization (QSI8D32) LHS matrix and quantized symmetric 4-bit signed integer with per-block quantization (QSI4C32) RHS matrix and the accumulation of the result into a single-precision (F32) output, optimized for SME2 technology.

Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com>

Signed-off-by: Anitha Raj <anitha.raj@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Anitha Raj <anitha.raj@arm.com>
Reviewed-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>
Reviewed-by: Anton Bondarenko <anton.bondarenko@arm.com>
Approved-by: Jakub Sujak <jakub.sujak@arm.com>

ba20def9

Disable hanging FVP jobs · ef4ffe0b

Jakub Sujak authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

ef4ffe0b

Nov 29, 2024

Update CHANGELOG for v0.5.0 · d41854ec
Jakub Sujak authored 1 month ago and Anton Bondarenko committed 1 month ago
```
Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Anton Bondarenko <anton.bondarenko@arm.com>
```
v0.5.0-rc1

d41854ec
Bump KleidiAI version to v0.5.0 · b650c26d
Anton Bondarenko authored 1 month ago and Jakub Sujak committed 1 month ago
```
Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>
```
b650c26d

Run CI pipeline for protected tags · 248d37ed

Anton Bondarenko authored 1 month ago


Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

248d37ed

Align compile flags across build systems · 9678b2c1
Jens Elofsson authored 1 month ago and Anton Bondarenko committed 1 month ago
```
Signed-off-by: Jens Elofsson <jens.elofsson@arm.com>

Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>
```
9678b2c1

Add SME Int8 GEMM · 0063002d

Suhail M authored 1 month ago


* Add the SME Int8 GEMM set of microkernels:
  - LHS packing kernel.
  - Non-transposed RHS packing kernel.
  - Main kernel.
* Update the test framework to support static int8 GEMM.

Resolves: KLEIDIAI-171, KLEIDIAI-235, KLEIDIAI-39
Signed-off-by: Mohammed Suhail Munshi <MohammedSuhail.Munshi@arm.com>
Signed-off-by: Viet-Hoa Do <viet-hoa.do@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>
Reviewed-by: Mohammed Suhail Munshi <mohammedsuhail.munshi@arm.com>
Reviewed-by: Emil Ohlsson <emil.ohlsson@arm.com>
Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

0063002d

Disable unnecessary vectorization and SVE features for SME kernels · 02eb2af4

Anton Bondarenko authored 1 month ago


Current solution for SME microkernels support is to use
SME opcodes and SVE instruction in streaming mode. And a precondition
for compiler to understand SVE instructions is compilation with
-march=...+sve+sve2. However this allows compiler to generate own SVE
instructions for normal C/C++ code. And might cause illegal instruction
exception on CPUs where SME implemented w/o SVE. In this case we want
to disable usage of compiler generated SVE instructions.

Test no SVE instructions using FVP with disabled SVE support.

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

02eb2af4

Add transposed RHS packing for SME F16 GEMM · 4c80f73a

Viet-Hoa Do authored 1 month ago and

Jakub Sujak committed 1 month ago


Signed-off-by: Viet-Hoa Do <viet-hoa.do@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Emil Ohlsson <emil.ohlsson@arm.com>
Approved-by: Jakub Sujak <jakub.sujak@arm.com>

4c80f73a

Add SME2 FP16 GEMV micro kernel · 62eb5d3a

Felix Johnny Thomasmathibalan authored 1 month ago and

Jakub Sujak committed 1 month ago


RHS pack is required.
LHS pack is not required

Signed-off-by: Felix Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

Reviewed-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>
Reviewed-by: Emil Ohlsson <emil.ohlsson@arm.com>
Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

62eb5d3a

Contain SME2 assembly inside SMSTART/SMSTOP boundary · 16f74240

Jakub Sujak authored 1 month ago

Regenerate the SME2 GEMV micro-kernel assembly so that it is contained within the SMSTART/SMSTOP boundary, preventing illegal instruction faults when attempting to execute streaming SVE code on a system without SVE support.

Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

16f74240

Add fp16 in/out bf16 Gemm kernel and relevant packing functions · cd34c1bd

Gunes Bayir authored 1 month ago and

Emil Ohlsson committed 1 month ago


This commit

* Adds bf16 x bf16 = fp16 matmul microkernel with 8x12 output block size
* Lhs/Rhs packing functions that packs and converts the inputs from fp16 to bf16
* Corresponding tests, and modifications to the testing framework, and reference implementation

Signed-off-by: Gunes Bayir <gunes.bayir@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Jakub Sujak <jakub.sujak@arm.com>
Reviewed-by: Gunes Bayir <gunes.bayir@arm.com>
Reviewed-by: Emil Ohlsson <emil.ohlsson@arm.com>
Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

cd34c1bd

Enable RHS get_stride API · c4a22b2e

Felix Johnny Thomasmathibalan authored 1 month ago


Affected micro kernel: FP16 GEMM, SME2

Signed-off-by: Felix Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

c4a22b2e

Nov 28, 2024

Fix warnings introduced by -Wpedantic. · 01250019

Jens Elofsson authored 1 month ago


Signed-off-by: Jens Elofsson <jens.elofsson@arm.com>

Reviewed-by: Emil Ohlsson <emil.ohlsson@arm.com>
Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

01250019

Matmul int8 micro-kernels for F32 <- QA8DX (LHS) x QS8ICX (RHS) · f51cd7ec

Gian Marco Iodice authored 1 month ago and

Jakub Sujak committed 1 month ago


- Add GeMM-like micro-kernels
- Add GeMV-like micro-kernels

Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com>

Reviewed-by: Jakub Sujak <jakub.sujak@arm.com>
Approved-by: Jakub Sujak <jakub.sujak@arm.com>

f51cd7ec

Fix copy/paste error in kernel interfaces · 5b3de06b

Emil Ohlsson authored 1 month ago and

Jakub Sujak committed 1 month ago


This change fixes a minor copy paste error in the kernel interfaces.

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>

5b3de06b

Nov 27, 2024

Matmul int4 micro-kernel variants 16x4 and 1x4 for QSI8D32 (LHS) x QSI4C32 (RHS) -> F32 · 39ab4d69

Michael Kozlov authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


Signed-off-by: Michael Kozlov <michael.kozlov@arm.com>

Signed-off-by: Anitha Raj <anitha.raj@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Anitha Raj <anitha.raj@arm.com>
Reviewed-by: Anton Bondarenko <anton.bondarenko@arm.com>
Approved-by: Jakub Sujak <jakub.sujak@arm.com>
Approved-by: Anton Bondarenko <anton.bondarenko@arm.com>

39ab4d69

Nov 21, 2024

Add FP16 SME kernels · a40f5e59

Emil Ohlsson authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


Add fp16 kernels for LHS and RHS packing, and matmul. Also add related
unit tests for said kernels, and extend unit Matmul tests to support
calling fp16 kernels.

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

a40f5e59

Nov 20, 2024

Change required CMake version to 3.16 · 12f396f6

Emil Ohlsson authored 1 month ago


KleidiAI is intended to target certain build environments, this means
that KleidiAI should be buildable using CMake version 3.16

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>

12f396f6

Nov 19, 2024

Add support for odd shapes in Int4 Matmul Unit tests. · 9228bea5

Anitha Raj authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


* Round off the odd strides for the int4 RHS by padding with 0s

Signed-off-by: Anitha Raj <anitha.raj@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

9228bea5

Nov 18, 2024

Extract rhs packing parameter struct into kai_common.h · 247b3ec5

Michael Kozlov authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


Alias existing nxk/kxn packing parameter structs to new one.

To keep a consistent interface for the packing function(s) within a microkernel folder.

Signed-off-by: Michael Kozlov <michael.kozlov@arm.com>

Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com>
Reviewed-by: Michael Kozlov <michael.kozlov@arm.com>
Approved-by: Gian Marco Iodice <gianmarco.iodice@arm.com>

247b3ec5

Check for feature support before generating testdata. · e875fadd

Jens Elofsson authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago

Move the data generation in the unit tests to after feature support have been confirmed.

Signed-off-by: Jens Elofsson <jens.elofsson@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

e875fadd

Nov 14, 2024

Change f32 pack kernel to use RHS offset · e5d557c0

Emil Ohlsson authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago


One of the f32 kernel tests incorrectly uses matmtul functionality for
both main and rhs support functionality. This commit addresses this
issue

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

e5d557c0

Refactor quantize function · 9be70bf6

Anitha Raj authored 1 month ago and

Felix Johnny Thomasmathibalan committed 1 month ago

Add generic transpose function, use it for non-transposed (kxn) RHS packing tests.

Signed-off-by: Anitha Raj <anitha.raj@arm.com>

Signed-off-by: Michael Kozlov <michael.kozlov@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

9be70bf6

Disable build of tests for benchmark framework · 19280d69

Anton Bondarenko authored 1 month ago


Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Emil Ohlsson <emil.ohlsson@arm.com>

19280d69

Nov 13, 2024

Bug fixes for RHS and LHS pack testing · d854d865

Emil Ohlsson authored 1 month ago


This change addresses testing issues related to RHS and LHS packing
tests.

One issue related to reference data comparisson, which is addressed by
adding proper rounding for comparisons.

LHS packing tests did not correctly pass rolling parameters back to the
packing kernel

RHS packing had issues with blocking, which affected portioned testing

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

d854d865

Readability fixes for testing framework · d17183a1

Emil Ohlsson authored 1 month ago


This change makes some readability changes to the testing framework.
Which allows printing of rectangles and intermediate values for easy
dumping while debugging.

Signed-off-by: Emil Ohlsson <emil.ohlsson@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

d17183a1

Nov 11, 2024

Decrease timeout and add a retry to jobs in CI · 83a3ef3f

Anton Bondarenko authored 2 months ago


10 minutes should be enough for regular ones. Job with
external dependency has a bigger timeout of 1 hour
for better robustness.

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

83a3ef3f

Nov 06, 2024

Include cmath in quantize.cpp · 40a92683
Jens Elofsson authored 2 months ago and Felix Johnny Thomasmathibalan committed 2 months ago
```
Signed-off-by: Jens Elofsson <jens.elofsson@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>
```
2 tags

40a92683

Use relative path to fetch KleidiAI dependencies · 94e72a94

Jakub Sujak authored 2 months ago and

Felix Johnny Thomasmathibalan committed 2 months ago

The `CMAKE_SOURCE_DIR` variable always corresponds to the top level directory of the CMakefile being processed by CMake. This causes issues for CMake projects that fetch KleidiAI using `FetchContent()` as it incorrectly assumes KleidiAI's dependencies reside in that project's top level directory, rather than in KleidiAI's source tree. Resolve this issue by using the `CMAKE_CURRENT_SOURCE_DIR` variable to use relative paths to the KleidiAI project.

Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

94e72a94

Nov 04, 2024

Bump KleidiAI version to v0.4.0 · 6c4f42db

Jakub Sujak authored 2 months ago


Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

v0.4.0-rc1

6c4f42db

Add list of v0.4.0 changes to CHANGELOG · 7985c263

Jakub Sujak authored 2 months ago

Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Reviewed-by: Anton Bondarenko <anton.bondarenko@arm.com>
Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

7985c263

Use local third party components instead of downloading ones · 2ad44d03

Anton Bondarenko authored 2 months ago and

Felix Johnny Thomasmathibalan committed 2 months ago


Now when we have major third party components in repository
there is no need to download them anymore.

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>

2ad44d03

Add 16x4 variant of ASM matmul micro-kernel for F32 <- QSI8D32 x QSI4C32 · c9c64b90
Michael Kozlov authored 2 months ago and Felix Johnny Thomasmathibalan committed 2 months ago
```
- Add variant to examples
- Add unit test for variant

Signed-off-by: Michael Kozlov <michael.kozlov@arm.com>

Approved-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
```
c9c64b90

Add local third party components · 950505ab

Anton Bondarenko authored 2 months ago and

Felix Johnny Thomasmathibalan committed 2 months ago

Local third party components provides a better clarity for KleidiAI
library external dependencies.

Commands used to get files:
LICENSES/BSD-3-Clause.txt -> reuse download BSD-3-Clause
third_party/benchmark-v1.8.4.zip -> wget https://github.com/google/benchmark/archive/refs/tags/v1.8.4.zip
third_party/googletest-v.1.14.0.zip -> wget https://github.com/google/googletest/archive/refs/tags/v1.14.0.zip

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

950505ab

Disable some pre-commit hooks for licenses · 42f13f77

Anton Bondarenko authored 2 months ago


It's preferred to keep original license text untouched so
common project code style should not apply to it.

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Felix Johnny Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>
Approved-by: Jakub Sujak <jakub.sujak@arm.com>

42f13f77

Add RHS pack parameters to wide SME GEMV name · 84c351fc

Jakub Sujak authored 2 months ago and

Felix Johnny Thomasmathibalan committed 2 months ago


This helps distinguish it from the other SME GEMV kernel.

Signed-off-by: Jakub Sujak <jakub.sujak@arm.com>

Approved-by: Anton Bondarenko <anton.bondarenko@arm.com>

84c351fc

Revert "Disable some pre-commit hooks for licenses" · a1562425

Felix Johnny Thomasmathibalan authored 2 months ago

This reverts commit 6b3c6fad

.

Signed-off-by: Felix Thomasmathibalan <felixjohnny.thomasmathibalan@arm.com>

Approved-by: Anton Bondarenko <anton.bondarenko@arm.com>

a1562425

Disable some pre-commit hooks for licenses · 6b3c6fad

Anton Bondarenko authored 2 months ago


It's preferred to keep original license text untouched so
common project code style should not apply to it.

Signed-off-by: Anton Bondarenko <anton.bondarenko@arm.com>

Approved-by: Jakub Sujak <jakub.sujak@arm.com>

6b3c6fad