```c
_Pragma( "omp parallel num_threads(n_threads)" )
{
// ... thread work ...
// Free the current thread's thrinfo_t structure.
bli_l3_thrinfo_free( rntm_p, thread ); // Line 183
}
// *** MISSING BARRIER HERE! ***
// Check the array_t back into the small block allocator...
bli_sba_checkin_array( array ); // Line 200
```
```c
// DANGEROUS execution timeline:
Thread 0 (chief):
completes func()
calls bli_l3_cntl_free()
calls bli_l3_thrinfo_free() → frees gl_comm ✓
exits OpenMP parallel region
calls bli_sba_checkin_array(array) → frees array ✗
Thread 1,2,3 (still executing):
still in func() or bli_l3_cntl_free()
trying to access freed gl_comm → CRASH!
trying to access freed array pools → CRASH!
```
This is **exactly the same issue** that PR #702 fixed in other files! The function needs a barrier before threads exit the parallel region to ensure:
1. **All threads complete their work** before any cleanup starts
2. **Global communicator isn't freed** while other threads are using it
3. **Array pools aren't freed** while other threads are accessing them
* Replaced omp barrier with bli_thread_barrier and added similar fix for pack compute routine
## **The Sequence is Critical:**
1. **All threads execute `func(...)`** - may access shared resources including communicators
2. **Barrier ** - ensures ALL threads finish their work
3. **Then each thread calls `bli_l3_sup_thrinfo_free()`** - only chief actually frees gl_comm
4. **Safe cleanup** - no use-after-free because all threads are done using gl_comm
## **What Would Happen Without the Barrier:**
```c
// Thread execution timeline WITHOUT barrier:
Time 1: Thread 0 finishes func() early → immediately calls bli_l3_sup_thrinfo_free()
Time 2: Thread 0 (chief) frees gl_comm
Time 3: Thread 1,2,3... still in func() → try to use freed gl_comm → CRASH!
```
## **With the Barrier (Current Safe Code):**
```c
// Thread execution timeline WITH barrier:
Time 1: All threads finish func() at different times
Time 2: ALL threads reach barrier → wait for slowest thread
Time 3: ALL threads proceed past barrier together
Time 4: Chief frees gl_comm → safe because no one using it anymore
```
- Remove redundant AOCL_DTL_LOG_NUM_THREADS calls from early return paths
- Update thread count logging to use AOCL_get_requested_threads_count() for early exits
- Clean up duplicate DTL logging in gemv_unf_var1 and gemv_unf_var2 implementations
- Remove thread count logging from bli_dgemv_n_zen4_int kernel variants
- Simplify aocldtl_blis.c AOCL_DTL_log_gemv_sizes by removing redundant conditional
- Standardize DTL trace exit patterns across axpy, scal, and gemv operations
- Remove commented-out DTL logging code in zen4 gemv kernel
This patch ensures thread count is logged only once per operation and uses
the correct API (AOCL_get_requested_threads_count) for early exit scenarios
where the actual execution thread count may differ from requested threads.
- Added `#pragma omp barrier` just before threads start releasing their threadinfo / global
communicator.
- This ensures all threads reach this sync point, preventing interleaved cleanup.
Co-authored-by: harsdave <harsdave@amd.com>
- Add explicit parentheses around (n <= 1520) && (k <= 128) to clarify
operator precedence and resolve compiler warning. The intended logic
is (m <= 1380) OR (n <= 1520 AND k <= 128).
- This change eliminates the compiler warning about mixing || and &&
operators without explicit grouping.
Adding SGEMM tiny path for Zen architectures.
Needed to cover some performance gaps seen wrt MKL
Only allowing matrices that all fit into the L1 cache to the tiny path
Only tuned for single threaded operation at the moment
Todo: Tune cases where AVX2 performs better than AVX512 on Zen4
Todo: The current ranges are very conservative, there may be scope to increase the matrix sizes that go into the tiny path
AMD-Internal: CPUPL-7555
Co-authored-by: Rohan Rayan rohrayan@amd.com
- In the current implementation of bf16 to f32 conversion for packed data
we handle both GEMM and GEMV conditions in the same function separated
with conditions.
- But, when n = (NC+1) the function would execute GEMV conversion logic
and write back the data inaccurately leading to accuracy issues.
- Hence, modified the convert function and reorder functions to have
separate conversion logic to make it cleaner and avoid confusions.
- Also, updated the API calls to adhere to the changes appropriately.
[AMD-Internal: CPUPL-7540]
- Revert the logical operator from OR (||) to AND (&&) in the DCOMPLEX
(ZGEMM) SUP threshold condition for k <= 128. The previous change to
OR logic was causing performance regressions for certain input sizes
by incorrectly routing cases to the SUP path when the native path
would be more optimal.
- Current kernel uses masked AVX512 instructions to handle fringe cases.
- These instructions are slow on genoa.
- To handle sizes less than 8, AVX2 and SSE code has been added.
- Existing masked AVX512 code is performing better when n > 8 therefore it is still kept for handling larger sizes where n % 8 != 0.
AMD-Internal: [CPUPL-7467]
Recent changes meant nt was defined but unused for serial builds
when DTL was not enabled. Restructure code to avoid this issue and
make the code more readable.
AMD-Internal: [CPUPL-6579]
Threshold tuning that determines whether SUP or native path should
be used for given input matrix size.
This tuning forces skinny matrices to take SUP path to ensure better
performance.
AMD-Internal: [CPUPL-7369]
Co-authored-by: harsh dave <harsdave@amd.com>
* DTL Log update
Updates logs with nt and AOCL Dynamic selected nt for axpy, scal and dgemv
Modified bench_gemv.c to able to process modified dtl logs.
* Updated DTL log for copy routine with actual nt and dynamic nt
* Refactor OpenMP pragmas and clean up code
Removed unnecessary nested OpenMP pragma and cleaned up function end comment.
* Fixed DTL log for sequential build
* Added thread logging in bla_gemv_check for invalid inputs
---------
Co-authored-by: Smyth, Edward <Edward.Smyth@amd.com>
1. Documented the process for handling external pull requests,
including validation, review, and notification steps.
2. Added a markdown flowchart illustrating the PR workflow.
3. Included communication and contributor attribution notes.
- Fixed register assignment bug in lpgemv_m_kernel_f32_avx512 where zmm3
was incorrectly used instead of zmm4 in BF16_F32_BETA_OP_NLT16F_MASK macro.
- Replaced hardware-specific BF16 conversion intrinsics with manual
rounding, bit manipulation and F32 instruction set for compatibility on
hardware without native BF16 support.
- Added AVX512_BF16 ISA support checks for s8s8s32obf16 and u8s8s32obf16
GEMM operations to ensure processor compatibility before execution.
AMD-Internal: [CPUPL-7410]
- In the TPSV APIs, operations such as (*n - 1) * *incx and *n * (*n + 1) may lead to overflow when using bla_integer. To mitigate this risk, these operations have been updated to use dim_t variables instead of bla_integer.
Some files have copyright statements but not details of the license.
Add this to DTL source code and some build and benchmark related
scripts.
AMD-Internal: [CPUPL-6579]
Current DAXPY kernel uses C code to solve cases when n %8 != 0.
This results in the compiled code using MUL+ADD instruction using SSE, instead of FMA instruction.
This causes inconsistency of numerical results.
To fix this, AVX2 and C code is replaced with masked AVX512 instructions to compute fringe cases.
AMD-Internal : [CPUPL-7315]
Memory is not freed for GEMV when MT kernel with called for NT = 1;
Fixed this by adding an extra check to make sure memory is freed.
AMD-Internal: [CPUPL-7352]
Fix compiler warning messages in LPGEMM code:
- Removed extraneous parentheses in aocl_batch_gemm_s8s8s32os32.c
- Removed unused variables in lpgemv_{m,n}_kernel_s8_grp_amd512vnni.c
- Changed ERR_UBOUND in math_utils_avx2.h and math_utils_avx512.h
to match how it is specified in AOCL libm erff.c
AMD-Internal: [CPUPL-6579]
* Fixed coverity issue in ztrsm small code path
* Fixed coverity issue in ztrsm small code path
---------
Co-authored-by: harsh dave <harsdave@amd.com>
* Fixes Coverity static analysis issue in the DTRSM
- Initializes ps_a_use variable and calls bli_auxinfo_set_ps_a() to set
pack stride in auxinfo structure.
* Fixed unintialized variable issue in the DTRSM
- Initializes ps_a_use variable and calls bli_auxinfo_set_ps_a() to set
pack stride in auxinfo structure.
---------
Co-authored-by: harsdave <harsdave@amd.com>
- Explicitly initialize all fields of mem_t structures in bli_znormfv_unb_var1 and bli_dnormfv_unb_var1 to prevent undefined behavior when memory is not allocated.
- Add a NULL check after bli_thread_broadcast() in bli_thrinfo_sup_create_for_cntl to ensure that the communicator is valid, and call bli_abort() if broadcast fails.
- AMD specific BLAS1 and BLAS2 franework: changes to make variants
more consistent with each other
- Initialize kernel pointers to NULL where not immediately set
- Fix code indentation and other other whitespace changes in DTL
code and addon/aocl_gemm/frame/s8s8s32/lpgemm_s8s8s32_sym_quant.c
- Fix typos in DTL comments
- Add missing newline at end of test/CMakeLists.txt
- Standardize on using arch_id variable name
AMD-Internal: [CPUPL-6579]
Fix compiler warnings for unused variables. Situations where they
occurred were:
- bla_gemm_amd.c: Builds not including AMD zen4 or zen5 codepaths.
- bla_trsm_amd.c: Builds where {M,N,K]=1 optimizations are disabled.
- Avoid warnings when DTL is not enabled.
AMD-Internal: [CPUPL-6579]
Previous commit (30c42202d7) for this problem turned off
-ftree-slp-vectorize optimizations for all kernels. Instead, copy
the approach of upstream BLIS commit 36effd70b6a323856d98 and disable
these optimizations only for the affected files by using GCC pragmas
AMD-Internal: [CPUPL-6579]
* Check for single thread or multi-thread execution
- Check for single or multi-threaded call to dgemm.
- If it is single threaded, dgemm is computed in tiny path
if inputs under threashold.
- If it is multi-threaded dgemm input, tiny path will evaluate
whether it is performant compute the given input with single
thread or not. Based on that input will be routed.
* Disable gemm_small on zen4/5 & single thread execution check for tiny path
- Check for single or multi-threaded call to dgemm.
- If it is single threaded, dgemm is computed in tiny path
if inputs under threashold.
- If it is multi-threaded dgemm input, tiny path will evaluate
whether it is performant compute the given input with single
thread or not. Based on that input will be routed.
- Architecture checks for zen3, zen2 and zen are placed
for gemm_small path.
---------
Co-authored-by: harsdave <harsdave@amd.com>
- Added conditional swapping of input matrices and their
strides for GEMV, based on whether transpose is toggled
specifically for the matrix, namely the B matrix when m=1
and the A matrix when n=1.
- This swapping ensures that we reroute the inputs to use the
alternative variant(code-path) in order to avoid packing cost
for the matrix, through logical transposition.
- Currently, this optimization is enabled only when no post-ops
are involved. With post-ops, there is a need to update the
incoming data(from the user) in some scenarios, which will be
dealt with later.
AMD-Internal: [CPUPL-7323]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
