- The current build systems have the following behaviour
with regards to building "aocl_gemm" addon codebase(LPGEMM)
when giving "amdzen" as the target architecture(fat-binary)
- Make: Attempts to compile LPGEMM kernels using the same
compiler flags that the makefile fragments set for BLIS
kernels, based on the compiler version.
- CMake: With presets, it always enables the addon compilation
unless explicitly specified with the ENABLE_ADDON variable.
- This poses a bug with older compilers, owing to them not supporting
BF16 or INT8 intrinsic compilation.
- This patch adds the functionality to check for GCC and Clang compiler versions,
and disables LPGEMM compilation if GCC < 11.2 or Clang < 12.0.
- Make: Updated the configure script to check for the compiler version
if the addon is specified.
CMake: Updated the main CMakeLists.txt to check for the compiler version
if the addon is specified, and to also force-update the associated
cache variable update. Also updated kernels/CMakeLists.txt to
check if "aocl_gemm" remains in the ENABLE_ADDONS list after
all the checks in the previous layers.
AMD-Internal: [CPUPL-7850]
Signed-off by : Vignesh Balasubramanian <Vignesh.Balasubramanian@amd.com>
- Disabled rerouting to GEMV in BF16 APIs, in case of inputs
having m == 1. We would now use the GEMM path for
handling such inputs.
AMD-Internal: [CPUPL-7536]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
- In the current implementation of bf16 to f32 conversion for packed data
we handle both GEMM and GEMV conditions in the same function separated
with conditions.
- But, when n = (NC+1) the function would execute GEMV conversion logic
and write back the data inaccurately leading to accuracy issues.
- Hence, modified the convert function and reorder functions to have
separate conversion logic to make it cleaner and avoid confusions.
- Also, updated the API calls to adhere to the changes appropriately.
[AMD-Internal: CPUPL-7540]
- Fixed register assignment bug in lpgemv_m_kernel_f32_avx512 where zmm3
was incorrectly used instead of zmm4 in BF16_F32_BETA_OP_NLT16F_MASK macro.
- Replaced hardware-specific BF16 conversion intrinsics with manual
rounding, bit manipulation and F32 instruction set for compatibility on
hardware without native BF16 support.
- Added AVX512_BF16 ISA support checks for s8s8s32obf16 and u8s8s32obf16
GEMM operations to ensure processor compatibility before execution.
AMD-Internal: [CPUPL-7410]
Fix compiler warning messages in LPGEMM code:
- Removed extraneous parentheses in aocl_batch_gemm_s8s8s32os32.c
- Removed unused variables in lpgemv_{m,n}_kernel_s8_grp_amd512vnni.c
- Changed ERR_UBOUND in math_utils_avx2.h and math_utils_avx512.h
to match how it is specified in AOCL libm erff.c
AMD-Internal: [CPUPL-6579]
- AMD specific BLAS1 and BLAS2 franework: changes to make variants
more consistent with each other
- Initialize kernel pointers to NULL where not immediately set
- Fix code indentation and other other whitespace changes in DTL
code and addon/aocl_gemm/frame/s8s8s32/lpgemm_s8s8s32_sym_quant.c
- Fix typos in DTL comments
- Add missing newline at end of test/CMakeLists.txt
- Standardize on using arch_id variable name
AMD-Internal: [CPUPL-6579]
- Added conditional swapping of input matrices and their
strides for GEMV, based on whether transpose is toggled
specifically for the matrix, namely the B matrix when m=1
and the A matrix when n=1.
- This swapping ensures that we reroute the inputs to use the
alternative variant(code-path) in order to avoid packing cost
for the matrix, through logical transposition.
- Currently, this optimization is enabled only when no post-ops
are involved. With post-ops, there is a need to update the
incoming data(from the user) in some scenarios, which will be
dealt with later.
AMD-Internal: [CPUPL-7323]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
- Moved the `#ifdef BLIS_KERNELS_ZEN4` directive to encompass the relevant code block more effectively in `lpgemm_s8s8s32_sym_quant.c` and to remove unused variables warning.
AMD-Internal: [CPUPL-7320]
- Change begin_asm and end_asm comments and unused code in files
kernels/haswell/3/sup/s6x16/bli_gemmsup_rv_haswell_asm_sMx6.c
kernels/zen4/3/sup/bli_gemmsup_cd_zen4_asm_z12x4m.c
to avoid problems in clobber checking script.
- Add missing clobbers in files
kernels/zen4/1m/bli_packm_zen4_asm_d24xk.c
kernels/zen4/1m/bli_packm_zen4_asm_z12xk.c
kernels/zen4/3/sup/bli_gemmsup_cv_zen4_asm_z12x4m.c
- Add missing newline at end of files.
- Update some copyright years for recent changes.
- Standardize license text formatting.
AMD-Internal: [CPUPL-6579]
- Disabled topology detection as libgomp is not honoring
the standard function omp_get_place_proc_ids
- Added B prefetch in bf16 B packing kernels
AMD-Internal: SWLCSG-3761
- A matrix by default isn't expected to be packed for a normal row-stored
case. Hence the packing implementation is incomplete.
- But if the user explicitly enables packing, interface wasn't handling
the condition appropriately leading to data overwriting inside the incomplete
pack kernels, thereby leading to accuracy failure.
- As a fix, updated the interface to set the explicit PACK A to UNPACKED and
proceed with GEMM in cases where transpose of A is not necessary.
- Updated the batch gemm input file with additional test cases covering all the
APIs.
Bug Fixes:
- Fixed implementation logic for column major inputs with post-ops to be disabled
in S8 batch mat-mul. With the existing implementation, column-major inputs wouldn't
be executed in case of of32/os32 inputs.
- Fixed the Scale/ZP calculation in bench foru8s8s32ou8 condition, which was leading
to accuracy failures.
[AMD-Internal: CPUPL-7283 ]
- A matrix packing by default in isn't necessary for row-major matrix data. Also, it seems packing of A was
giving regressions and hence wasn't expected to be used.
- However, packA is necessary in column-major cases, where transpose has to be done. This path has been verified.
- Hence, when user sets pack A explicitly, it gets into the incomplete packA function, and overwrites the elements
in the buffer after subsequent iterations, leading to accuracy issues. As a fix to this the patch updates PACK
condition to UNPACKED at the interface while user explicitly sets one, ensuring seamless execution.
[ AMD-Internal : CPUPL - 7193 ]
Details:
- Fixed loading of matadd and matmul pointers in GEMV
lt16 kernel for AVX2 M=1 case.
- Hard-set row-stride of B to 1(inside GEMV), when it has
already been reordered.
AMD-Internal:CPUPL-7197, CPUPL-7221
Co-authored-by:Balasubramanian, Vignesh <Vignesh.Balasubramanian@amd.com>
Introduced support for GEMV operations with group-level symmetric quantization for the S8S8S32032 API.
Framework Changes:
- Added macro definitions and function prototypes for GEMV with symmetric quantization in lpgemm_5loop_interface_apis.h and lpgemm_kernels.h.
- LPGEMV_M_EQ1_KERN2 for the lpgemv_m_one_s8s8s32os32_sym_quant kernel, and
- LPGEMV_N_EQ1_KERN2 for the lpgemv_n_one_s8s8s32os32_sym_quant kernel.
- Implemented the main GEMV framework for symmetric quantization in lpgemm_s8s8s32_sym_quant.c.
Kernel Changes:
- lpgemv_m_one_s8s8s32os32_sym_quant for handling the case where M = 1 and implemented in lpgemv_m_kernel_s8_grp_amd512vnni.c.
- lpgemv_n_one_s8s8s32os32_sym_quant for handling the case where N = 1 and implemented in lpgemv_n_kernel_s8_grp_amd512vnni.c.
- Updated the buffer reordering logic for group quantization for N=1 cases in aocl_gemm_s8s8s32os32_utils.c.
Notes
- Ensure that group_size is a factor of both K (and KC when K > KC).
- The B matrix must be provided in reordered format (mtag_b == REORDERED).
AMD-Internal: [SWLCSG-3604]
- Removed duplicate calls to BATCH_GEMM_CHECK().
- Refactored freeing of post-op pointer in bench code and verified the
functionality.
- Modified indexing of the array to take the correct values.
Description:
1. Crated f32 intrinsic kernels without post-ops support f32 gemm
without post-ops optimally.
2. Initiated the no post-ops kernels from main kernel when post-ops
hander has no post-ops to do.
3. The kernels are redundant but added to get the best perf
for pure GEMM call.
AMD-Internal : SWLCSG-3692
- As part of rerouting to AVX2 code-paths on ZEN4/ZEN5(or similar)
architectures, the code-base established a contingency when
deploying fat binary on ZEN/ZEN2/ZEN3 systems. Due to this,
it was required that we always set AOCL_ENABLE_INSTRUCTIONS to
'ZEN3'(or similar values) to make sure we don't run AVX512
code on such architectures. This issue existed on FP32 and BF16
APIs.
- Added checks to detect the AVX512-ISA support to enable rerouting
based on AOCL_ENABLE_INSTRUCTIONS. This removes the incorrect
constraint that was put forth.
AMD-Internal: [CPUPL-7020]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
- For int8/uint8 reorder function, the k dimension is made multiple of 4 to
meet the alignment requirements.
- Modified the logic to update the k_updated to use multiples of 4.
[AMD - Internal : SWLCSG - 3686 ]
- Modified the correct variables to be passed for the batch_gemm_thread_decorator() for
u8s8s32os32 API.
- Removed commented lines in f32 GEMV_M kernels.
- Modified some instructions in F32 GEMV M and N Kernels to re-use the existing macros.
- Re-aligned the BIAS macro in the macro definition file.
[ AMD - Internal : CPUPL - 7013 ]
Implemented aocl_reorder_f32f32f32of32_reference( ) function and tested.
Implemented framework changes required and place holder for kernels for aocl_unreorder_f32f32f32of32_reference( ) function. It is not tested completely and will be taken care in subsequent commits.
[AMD-Internal: SWLCSG-3618 ]
- Added the correct strides to be used while unreorder/convert B matrix in m=1 cases.
- Modified Zero point vector loads to proper instructions.
- Modified bf16 store in AVX2 GEMV M kenrel
AMD Internal - [SWLCSG - 3602 ]
- Added new GEMV_AVX2 5-Loop for handling BF16 inputs, for n = 1 and m = 1 conditions.
- Modified Re-order and Un-reorder functions to cater to default n=1 reorder conditions.
- Added bf16 beta and store support in F32 GEMV N AVX2 and 256_512 kernels.
- Added bf16 beta support for F32 GEMV M kernels, and modified bf16 store conditions for
GEMV M kernels.
- Modified the n=1 re-order guards for reference bf16 re-order API.
- Added an additional path in the un-reorder case for handling n=1 vector conversion
AMD-Internal: [ SWLCSG - 3602 ]
* Updated aocl_batch_gemm_ APIs aligning to CBLAS batch API.
- Modified Batch-Gemm API to align with cblas_?gemm_batch_ API,
and added a parameter group_size to the existing APIs.
- Updated bench batch_gemm code to align to the new API definition.
- Modified the hardcoded number in lpgemm_postop file.
- Added necessary early return condition to account for group_count/group_size < 0.
AMD-Internal: [ SWLCSG - 3592 ]
Details:
- In s8 APIs with symmetric quantization, Existing kernels are
reused to avoid duplication of reorder code.
- Since the existing kernels are designed assuming that entire
KCxNC block is packed at once, to handle grouping in symmetric
quantization, we have to add JR and group loop outside the
function call to existing packB function.
- Though this was being done before, the cases where n_rem < 64
was not handled properly.
- Modified reorder and pack code to first divide the n_fringe part
into multiples-of-16 part and n_lt_16 part and then calling the
pack kernel twice to handle both parts separately.
- All the strides to access the reordered/pack buffer are updated
accordingly.
Details:
- In FP32 GEMM, when threading is disabled, rntm_pack_a and rntm_pack_b
were set to true by default. This leads to perf regression for smaller
sizes. Modified FP32 interface API to not overwrite the packA and
packB variables in rntm structure.
- In FP32 GEMV, Removed the decision making code based on mtag_A/B
and should_pack_A/B for packing. Matrices will be packed only
if the storage format of the matrices doesn't match the storage
format required by the kernel.
- Changed the control flow of checking the value of mtag to whether
matrix is "reordered" or "to-be-packed" or "unpacked". checking
for "reorder" first, followed by "pack". This will ensure that
packing doesn't happen when the matrix is already reordered even
though user forces packing by setting "BLIS_PACK_A/B"
-Modified python script to generate testcases based on block sizes
AMD-Internal: SWLCSG-3527
- The current implementation of the topology detector establishes
a contingency, wherein it is expected that the parallel region
uses all the threads queried through omp_get_max_threads(). In
case the actual parallelism in the function is limited(lower than
this expectation), the code may access unallocated memory section
(using uninitialized pointers).
- This was because every thread(having it's own pointer), sets its
initial value to NULL inside the parallel section, thereby leaving
some pointers uninitialized if the associated thread is not spawned.
- Also, the current implementation would use negative indexing(with -1)
if any associated thread was not spawned.
- Fix : Set every thread-specific pointer to NULL outside the parallel
region, using calloc(). As long as we have NULL checks for pointers
before accessing through them, no issues will be observed. Avoid
incurring the topology detection cost if all the reuqired threads
are not spawned(thereby avoiding potential negative indexing).
(when using core-group ID).
AMD-Internal: [SWLCSG-3573]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
Co-authored-by: Bhaskar, Nallani <Nallani.Bhaskar@amd.com>
Details:
- In FP32 GEMM interface, mtag_b is being set to PACK by default.
This is leading to packing of B matrix even though packing is not
absolutely required leading to perf regression.
- Setting mtag_b to PACK only if it is absolutely necessary to pack B matrix
modified check conditions before packing appropriately.
AMD-Internal - [SWLCSG-3575]
* Bug Fixes in FP32 Kernels:
- The current implementation lets m=1 tiny cases inside LPGEMV_TINY loop,
but the m=1 GEMV kernel call doesn't have the call to GEMV_M_ONE kernels.
Added the m=1 path in LPGEMV_TINY loop by handling the pack A/Pack B/reorder B
conditions.
- Added BF16 support for BIAS, Matrix-Add and Matrix-Mul for AVX512 F32
main and GEMV kernels
- Added BF16 Matrix-Add and Matrix-Mul support for AVX512_256 F32 kernels.
- Modified the condition check in FP32 Zero point in AVX512 kernels, and
fixed few bugs in Col-major Zero point evaluation.
AMD Internal: [ CPUPL - 6748 ]
* Bug Fixes in FP32 Kernels:
- The current implementation lets m=1 tiny cases inside LPGEMV_TINY loop,
but doesn't have the call to GEMV_M_ONE kernels. Added the m=1 path in
LPGEMV_TINY loop by handling the pack A/Pack B/reorder B conditions.
- Added BF16 support for BIAS, Matrix-Add and Matrix-Mul for AVX512 F32
main and GEMV kernels.
- Added BF16 Downscale, BIAS, Matrix-Add and Matrix-Mul support in AVX2 GEMV_N
and AVX512_256 GEMV kernels.
- Added BF16 Matrix-Add and Matrix-Mul support for AVX512_256 F32 kernels.
- Modified the condition check in FP32 Zero point in AVX512 kernels, and
fixed few bugs in Col-major Zero point evaluation and instruction usage.
AMD Internal: [ CPUPL - 6748 ]
* Bug Fixes in FP32 Kernels:
- The current implementation lets m=1 tiny cases inside LPGEMV_TINY loop,
but doesn't have the call to GEMV_M_ONE kernels. Added the m=1 path in
LPGEMV_TINY loop by handling the pack A/Pack B/reorder B conditions.
- Added BF16 support for BIAS, Matrix-Add and Matrix-Mul for AVX512 F32
main and GEMV kernels.
- Added BF16 Downscale, BIAS, Matrix-Add and Matrix-Mul support in AVX2 GEMV_N
and AVX512_256 GEMV kernels.
- Added BF16 Matrix-Add and Matrix-Mul support for AVX512_256 F32 kernels.
- Modified the condition check in FP32 Zero point in AVX512 kernels, and
fixed few bugs in Col-major Zero point evaluation and instruction usage.
AMD Internal: [ CPUPL - 6748 ]
* Bug Fixes in FP32 Kernels:
- The current implementation lets m=1 tiny cases inside LPGEMV_TINY loop,
but doesn't have the call to GEMV_M_ONE kernels. Added the m=1 path in
LPGEMV_TINY loop by handling the pack A/Pack B/reorder B conditions.
- Added BF16 support for BIAS, Matrix-Add and Matrix-Mul for AVX512 F32
main and GEMV kernels.
- Added BF16 Downscale, BIAS, Matrix-Add and Matrix-Mul support in AVX2 GEMV_N
and AVX512_256 GEMV kernels.
- Added BF16 Matrix-Add and Matrix-Mul support for AVX512_256 F32 kernels.
- Modified the condition check in FP32 Zero point in AVX512 kernels, and
fixed few bugs in Col-major Zero point evaluation and instruction usage.
AMD Internal: [ CPUPL - 6748 ]
---------
Co-authored-by: VarshaV <varshav2@amd.com>
* Implemented 6xlt8 AVX2 kernel for n<8 inputs
* Implemented fringe kernels for 6x16 and 6xlt16 AVX512 kernels for FP32
* Implemented m-fringe kernels for 6xlt8 kernel for AVX2
* Implemented m-fringe kernels for 6xlt8 kernel for AVX2
* Added the deleted kernels and fixed bias bug
AMD-Internal: SWLCSG-3556
* Bug Fixes in LPGEMM for AVX512(SkyLake) machine
- B-matrix in bf16bf16f32obf16/f32 API is re-ordered. For machines that
doesn't support BF16 instructions, the BF16 input is unre-ordered and
converted to FP32 to use FP32 kernels.
- For n = 1 and k = 1 sized matrices, re-ordering in BF16 is copying the
matrix to the re-ordered buffer array. But the un-reordering to FP32
requires the matrix to have size multiple of 16 along n and multiple
of 2 along k dimension.
- The entry condition to the above has been modified for AVX512 configuration.
- In bf16 API, the tiny path entry check has been modified to prevent
seg fault while AOCL_ENABLE_INSTRUCTIONS=AVX2 is set in BF16 supporting
machines.
- Modified existing store instructions in FP32 AVX512 kernels to support
execution in machines that has AVX512 support but not BF16/VNNI(SkyLake).
- Added Bf16 beta and store types in FP32 avx512_256 kernels
AMD Internal: [SWLCSG-3552]
* Bug Fixes in LPGEMM for AVX512(SkyLake) machine
- B-matrix in bf16bf16f32obf16/f32 API is re-ordered. For machines that
doesn't support BF16 instructions, the BF16 input is unre-ordered and
converted to FP32 to use FP32 kernels.
- For n = 1 and k = 1 sized matrices, re-ordering in BF16 is copying the
matrix to the re-ordered buffer array. But the un-reordering to FP32
requires the matrix to have size multiple of 16 along n and multiple
of 2 along k dimension.
- The entry condition to the above has been modified for AVX512 configuration.
- In bf16 API, the tiny path entry check has been modified to prevent
seg fault while AOCL_ENABLE_INSTRUCTIONS=AVX2 is set in BF16 supporting
machines.
- Modified existing store instructions in FP32 AVX512 kernels to support
execution in machines that has AVX512 support but not BF16/VNNI(SkyLake).
- Added Bf16 beta and store types, along with BIAS and ZP in FP32 avx512_256
kernels
AMD Internal: [SWLCSG-3552]
* Bug Fixes in LPGEMM for AVX512(SkyLake) machine
- Support added in FP32 512_256 kerenls for : Beta, BIAS, Zero-point and
BF16 store types for bf16bf16f32obf16 API execution in AVX2 mode.
- B-matrix in bf16bf16f32obf16/f32 API is re-ordered. For machines that
doesn't support BF16 instructions, the BF16 input is unre-ordered and
converted to FP32 type to use FP32 kernels.
- For n = 1 and k = 1 sized matrices, re-ordering in BF16 is copying the
matrix to the re-ordered buffer array. But the un-reordering to FP32
requires the matrix to have size multiple of 16 along n and multiple
of 2 along k dimension. The entry condition here has been modified for
AVX512 configuration.
- Fix for seg fault with AOCL_ENABLE_INSTRUCTIONS=AVX2 mode in BF16/VNNI
ISA supporting configruations:
- BF16 tiny path entry check has been modified to take into account arch_id
to ensure improper entry into the tiny kernel.
- The store in BF16->FP32 col-major for m = 1 conditions were updated to
correct storage pattern,
- BF16 beta load macro was modified to account for data in unaligned memory.
- Modified existing store instructions in FP32 AVX512 kernels to support
execution in machines that has AVX512 support but not BF16/VNNI(SkyLake)
AMD Internal: [SWLCSG-3552]
---------
Co-authored-by: VarshaV <varshav2@amd.com>
- Fixed the group size validation logic to correctly check if the
group_size is a multiple of 4.
- Previously the condition was incorrectly performing bitwise AND with
decimal 11 instead of binary 11 (decimal 3).
AMD-Internal: [CPUPL-6754]
* Fixed configuration issues in AOCL_GEMM addon
Description:
Fixed aocl_gemm addon initialization of kernels and block sizes
for machines which supports only AVX512 but not
AVX512_VNNI/VNNI_BF16.
Aligned NC, KC blocking variables between ZEN and ZEN4
AMD-Internal: [SWLCSG-3527]
* Implemented GEMV kernel for m=1 case.
Description:
- Added a new GEMV kernel for AVX2 where m=1.
- Added a new GEMV kernel for AVX512 with ymm registers where m=1.
Details:
- This implementation is picked form cntx when GEMM is invoked on
machines that support AVX512 instructions by forcing the
AVX2 path using AOCL_ENABLE_INSTRUCTIONS=AVX2 during run-time.
- This implementation uses MR=16 for GEMV.
AMD-Internal: [SWLCSG-3519]
Change-Id: I8598ce6b05c3d5a96c764d96089171570fbb9e1a
- Added a new GEMV kernel with MR = 8 which will be used
for cases where n=1.
- Modified GEMM and GEMV framework to choose right GEMV kernel
based on compile-time and run-time architecture parameters. This
had to be done since GEMV kernels are not stored-in/retrieved-from
the cntx.
- Added a pack kernel that packs A matrix from col-major to row-major
using AVX2 instructions.
AMD-Internal: [SWLCSG-3519]
Change-Id: Ibf7a8121d0bde37660eac58a160c5b9c9ebd2b5c
Details:
- These kernels are picked from cntx when GEMM is invoked
on machines that support AVX512 instructions by forcing the
AVX2 path using AOCL_ENABLE_INSTRUCTIONS=AVX2 during run-time.
- This path uses the same blocksizes and pack kernels as AVX512
path.
- GEMV is disabled currently as AVX2 kernels for GEMV are not
implemented.
AMD-Internal: [SWLCSG-3519]
Change-Id: I75401fac48478fe99edb8e71fa44d36dd7513ae5
Description
- To enable AVX512 VNNI support without native BF16 in BF16 kernels, the
BF16 C_type is converted to F32 for computation and then cast back to
BF16 before storing the result.
- Added support for handling BF16 zero-point values of BF16 type.
- Added a condition to disable the tiny path for the BF16 code path
where native BF16 is not supported.
AMD Internal : [CPUPL-6627]
Change-Id: I1e0cfefd24c5ffbcc95db73e7f5784a957c79ab9
- Updated the decision logic for taking the RD path for FP32.
- Since the 5-loop was designed specifically for RV kernels, added a
boolean flag to specify when RD path is to be taken, and set ps_b_use
to cs_b_use in case B matrix is unpacked.
AMD-Internal: [SWLCSG-3497]
Change-Id: I94ed28304a71b759796edcdd4edf65b9bad22bea
- For single-threaded configuration of BLIS, packing of A and B matrices
are enabled by default. But, packing of A is only supported for RV
kernels where elements from matrix A are being broadcasted. Since
elements are being loaded in RD kernels, packing of A results in
failures. Hence, disabled packing of matrix A for RD kernels.
- Fixed the issue where c_i index pointer was incorrectly being reset
when exceeding MC block thus, resulting in failures for certain
Post-Ops.
- Fixed the FP32 reoder case were for n == 1 and rs_b == 1 condition, it
was incorrectly using sizeof(BLIS_FLOAT) instead of sizeof(float).
AMD-Internal: [SWLCSG-3497]
Change-Id: I6d18afa996c253d79f666ea9789270bb59b629dd
Details:
- In reorder functions, validity of strides are being checked assuming
that the matrix to be reordered is always row-major. Modified the code
to take stor_order into consideration while checking for validity of
strides.
- This does not directly impact the functionality of GEMM as we don't
support GEMM on col-major matrices where A and/or B matrices are
reordered before GEMM computation. But this change makes sense when
reordering is viewed as an independent functionality irrespective of
what the reordered buffers will be used for.
Change-Id: If2cc4a353bca2f998ad557d6f128881bc9963330
- Support for Post-Ops has been added for all F32 RD AVX512 and AVX2
kernels.
AMD-Internal: [SWLCSG-3497]
Change-Id: Ia2967417303d8278c547957878d93c42c887109e
- Added FP32 RD (dot-product) kernels for both, AVX512 and AVX2 ISAs.
- The FP32 AVX512 primary RD kernel has blocking of dimensions 6x64
(MRxNR) whereas it is 6x16 (MRxNR) for the AVX2 primary RD kernel.
- Updatd f32 framework to accomodate rd kernels in case of B trans
with thresholds
- Updated data gen python script
TODO:
- Post-Ops not yet supported.
Change-Id: Ibf282741f58a1446321273d5b8044db993f23714
- Updated the S8 main, GEMV, m_, n_ and mn_ fringe kernels to support
multiple scale types for vector and scalar scales
- Updated the U8 main, GEMV, m_, n_, extMR_ and mn_ fringe kernels to
support multiple scale types for vector and scalar scales
- Updated the bench to accommodate multiple scale type input, and
modified the downscale_accuracy_check_ to verify with multiple scale
type inputs.
AMD Internal: [ SWLCSG-3304 ]
Change-Id: I7b9f3ec8ea830d3265f72d18a0aa36086e14a86e
- Updated the F32 tiny path to support column-major inputs.
- Tuned the tiny-path thresholds to redirect additional inputs to the
tiny path based on the m*n*k value.
AMD-Internal: [SWLCSG-3380]
Change-Id: If3476b17cc5eaf4f4e1cf820af0a32ede3e1742e
