* Adding data pool for random number generator
* Adopting pool in axpy, gemv, and ukr.gemm
* Using templates in data pool getter function to generate different pools for different ranges
* Addressing review comments
* Using data pool in more APIs
* Using data pool in more APIs
* Updating testing infra for trsv and trsm
* Addressing review comments
* Fixing numerical issues in trsm testing
* More review comments
* Small change to get trsm tests to pass
* Scalling tiny data pool
* Addressing review comments
- Disabled rerouting to GEMV in BF16 APIs, in case of inputs
having m == 1. We would now use the GEMM path for
handling such inputs.
AMD-Internal: [CPUPL-7536]
Co-authored-by: Vignesh Balasubramanian <vignbala@amd.com>
In earlier commits (e.g. a2beef3255) long running tests were broken
into separate executables for each data type. Standardize this approach
across all APIs, including separate executables for IIT_ERS tests, and
for each data type have separate executables for EVT tests.
AMD-Internal: [CPUPL-7386]
Co-authored-by: Vlachopoulou, Eleni <Eleni.Vlachopoulou@amd.com>
* GTestSuite: computediff improvements
- Using lazy evaluation to only create error strings when the comparison failed
- Check if increments are zero, then only check first element
Lots of the bench applications were not taking n_repeats as an argument on the command line (while others were).
Also adding the feature to skip the rest of the line (at the end) while reading bench inputs for all APIs
---------
Co-authored-by: Rayan <rohrayan@amd.com>
- If blis is compiled as multithreaded library, BLIS_NUM_THREADS is set to 1, and sizes are large enough for multithreaded path to be optimal, we take multithreaded path even though we can spawn only one thread. This adds openmp overhead.
- A check has been added inside the multithreaded kernels to check to use single threaded code path of only 1 thread can be spawned.
AMD-Internal : [SWLCSG-3408]
- When alpha == 0, we are expected to only scale y vector with beta and not read A or X at all.
- This scenario is not handled properly in all code paths which causes NAN and INF from A and X being wrongly propagated. For example, for non-zen architecture (default block in switch case) no such check is present, similarly some of the avx512 kernels are also missing these checks.
- When beta == 0, we are not expected to read Y at all, this also is not handled correctly in one of the avx512 kernel.
- To fix these, early return condition for alpha == 0 is added to bla layer itself so that each kernel does not have to implement the logic.
- DGEMV AVX512 transpose kernel has been fixed to load vector Y only when beta != 0.
AMD-Internal: [CPUPL-7585]
```c
_Pragma( "omp parallel num_threads(n_threads)" )
{
// ... thread work ...
// Free the current thread's thrinfo_t structure.
bli_l3_thrinfo_free( rntm_p, thread ); // Line 183
}
// *** MISSING BARRIER HERE! ***
// Check the array_t back into the small block allocator...
bli_sba_checkin_array( array ); // Line 200
```
```c
// DANGEROUS execution timeline:
Thread 0 (chief):
completes func()
calls bli_l3_cntl_free()
calls bli_l3_thrinfo_free() → frees gl_comm ✓
exits OpenMP parallel region
calls bli_sba_checkin_array(array) → frees array ✗
Thread 1,2,3 (still executing):
still in func() or bli_l3_cntl_free()
trying to access freed gl_comm → CRASH!
trying to access freed array pools → CRASH!
```
This is **exactly the same issue** that PR #702 fixed in other files! The function needs a barrier before threads exit the parallel region to ensure:
1. **All threads complete their work** before any cleanup starts
2. **Global communicator isn't freed** while other threads are using it
3. **Array pools aren't freed** while other threads are accessing them
* Replaced omp barrier with bli_thread_barrier and added similar fix for pack compute routine
## **The Sequence is Critical:**
1. **All threads execute `func(...)`** - may access shared resources including communicators
2. **Barrier ** - ensures ALL threads finish their work
3. **Then each thread calls `bli_l3_sup_thrinfo_free()`** - only chief actually frees gl_comm
4. **Safe cleanup** - no use-after-free because all threads are done using gl_comm
## **What Would Happen Without the Barrier:**
```c
// Thread execution timeline WITHOUT barrier:
Time 1: Thread 0 finishes func() early → immediately calls bli_l3_sup_thrinfo_free()
Time 2: Thread 0 (chief) frees gl_comm
Time 3: Thread 1,2,3... still in func() → try to use freed gl_comm → CRASH!
```
## **With the Barrier (Current Safe Code):**
```c
// Thread execution timeline WITH barrier:
Time 1: All threads finish func() at different times
Time 2: ALL threads reach barrier → wait for slowest thread
Time 3: ALL threads proceed past barrier together
Time 4: Chief frees gl_comm → safe because no one using it anymore
```
- Remove redundant AOCL_DTL_LOG_NUM_THREADS calls from early return paths
- Update thread count logging to use AOCL_get_requested_threads_count() for early exits
- Clean up duplicate DTL logging in gemv_unf_var1 and gemv_unf_var2 implementations
- Remove thread count logging from bli_dgemv_n_zen4_int kernel variants
- Simplify aocldtl_blis.c AOCL_DTL_log_gemv_sizes by removing redundant conditional
- Standardize DTL trace exit patterns across axpy, scal, and gemv operations
- Remove commented-out DTL logging code in zen4 gemv kernel
This patch ensures thread count is logged only once per operation and uses
the correct API (AOCL_get_requested_threads_count) for early exit scenarios
where the actual execution thread count may differ from requested threads.
- Added `#pragma omp barrier` just before threads start releasing their threadinfo / global
communicator.
- This ensures all threads reach this sync point, preventing interleaved cleanup.
Co-authored-by: harsdave <harsdave@amd.com>
- Add explicit parentheses around (n <= 1520) && (k <= 128) to clarify
operator precedence and resolve compiler warning. The intended logic
is (m <= 1380) OR (n <= 1520 AND k <= 128).
- This change eliminates the compiler warning about mixing || and &&
operators without explicit grouping.
Adding SGEMM tiny path for Zen architectures.
Needed to cover some performance gaps seen wrt MKL
Only allowing matrices that all fit into the L1 cache to the tiny path
Only tuned for single threaded operation at the moment
Todo: Tune cases where AVX2 performs better than AVX512 on Zen4
Todo: The current ranges are very conservative, there may be scope to increase the matrix sizes that go into the tiny path
AMD-Internal: CPUPL-7555
Co-authored-by: Rohan Rayan rohrayan@amd.com
- In the current implementation of bf16 to f32 conversion for packed data
we handle both GEMM and GEMV conditions in the same function separated
with conditions.
- But, when n = (NC+1) the function would execute GEMV conversion logic
and write back the data inaccurately leading to accuracy issues.
- Hence, modified the convert function and reorder functions to have
separate conversion logic to make it cleaner and avoid confusions.
- Also, updated the API calls to adhere to the changes appropriately.
[AMD-Internal: CPUPL-7540]
- Revert the logical operator from OR (||) to AND (&&) in the DCOMPLEX
(ZGEMM) SUP threshold condition for k <= 128. The previous change to
OR logic was causing performance regressions for certain input sizes
by incorrectly routing cases to the SUP path when the native path
would be more optimal.
- Current kernel uses masked AVX512 instructions to handle fringe cases.
- These instructions are slow on genoa.
- To handle sizes less than 8, AVX2 and SSE code has been added.
- Existing masked AVX512 code is performing better when n > 8 therefore it is still kept for handling larger sizes where n % 8 != 0.
AMD-Internal: [CPUPL-7467]
Recent changes meant nt was defined but unused for serial builds
when DTL was not enabled. Restructure code to avoid this issue and
make the code more readable.
AMD-Internal: [CPUPL-6579]
Threshold tuning that determines whether SUP or native path should
be used for given input matrix size.
This tuning forces skinny matrices to take SUP path to ensure better
performance.
AMD-Internal: [CPUPL-7369]
Co-authored-by: harsh dave <harsdave@amd.com>
* DTL Log update
Updates logs with nt and AOCL Dynamic selected nt for axpy, scal and dgemv
Modified bench_gemv.c to able to process modified dtl logs.
* Updated DTL log for copy routine with actual nt and dynamic nt
* Refactor OpenMP pragmas and clean up code
Removed unnecessary nested OpenMP pragma and cleaned up function end comment.
* Fixed DTL log for sequential build
* Added thread logging in bla_gemv_check for invalid inputs
---------
Co-authored-by: Smyth, Edward <Edward.Smyth@amd.com>
1. Documented the process for handling external pull requests,
including validation, review, and notification steps.
2. Added a markdown flowchart illustrating the PR workflow.
3. Included communication and contributor attribution notes.
- Fixed register assignment bug in lpgemv_m_kernel_f32_avx512 where zmm3
was incorrectly used instead of zmm4 in BF16_F32_BETA_OP_NLT16F_MASK macro.
- Replaced hardware-specific BF16 conversion intrinsics with manual
rounding, bit manipulation and F32 instruction set for compatibility on
hardware without native BF16 support.
- Added AVX512_BF16 ISA support checks for s8s8s32obf16 and u8s8s32obf16
GEMM operations to ensure processor compatibility before execution.
AMD-Internal: [CPUPL-7410]
- In the TPSV APIs, operations such as (*n - 1) * *incx and *n * (*n + 1) may lead to overflow when using bla_integer. To mitigate this risk, these operations have been updated to use dim_t variables instead of bla_integer.
Some files have copyright statements but not details of the license.
Add this to DTL source code and some build and benchmark related
scripts.
AMD-Internal: [CPUPL-6579]
Current DAXPY kernel uses C code to solve cases when n %8 != 0.
This results in the compiled code using MUL+ADD instruction using SSE, instead of FMA instruction.
This causes inconsistency of numerical results.
To fix this, AVX2 and C code is replaced with masked AVX512 instructions to compute fringe cases.
AMD-Internal : [CPUPL-7315]
Memory is not freed for GEMV when MT kernel with called for NT = 1;
Fixed this by adding an extra check to make sure memory is freed.
AMD-Internal: [CPUPL-7352]
