Commit Graph

3645 Commits

Author SHA1 Message Date
Devin Matthews
ec06b6a503 Add dependency on the "flat" blis.h file for the BLIS and BLAS testsuite objects.
This fixes a bug where "make -j<N> check" may fail after a change to one or more header files, or where testsuite code doesn't get properly recompiled after internal changes.
2021-08-12 19:27:31 -05:00
Field G. Van Zee
20a1c4014c Disabled sanity check in bli_pool_finalize().
Details:
- Disabled a sanity check in bli_pool_finalize() that was meant to alert
  the user if a pool_t was being finalized while some blocks were still
  checked out. However, this is exactly the situation that might happen
  when a pool_t is re-initialized for a larger blocksize, and currently
  bli_pool_reinit() is implemeneted as _finalize() followed by _init().
  So, this sanity check is not universally appropriate. Thanks to
  AMD-India for reporting this issue.
2021-08-12 14:44:04 -05:00
Field G. Van Zee
e366665cd2 Fixed stale API calls to membrk API in gemmlike.
Details:
- Updated stale calls to the bli_membrk API within the 'gemmlike'
  sandbox. This API is now called bli_pba (packed block allocator).
  Ideally, this forgotten update would have been included as part of
  21911d6, which is when the branch where the membrk->pba changes was
  introduced was merged into 'master'.
- Comment updates.
2021-08-12 14:06:53 -05:00
RuQing Xu
e38ca28689 Added Apple Firestorm (A14/M1) Subconfig
- Use the same bulk kernel as Cortex-A53 / ThunderX2;
- Larger block size;
- Use gemmsup kernels for double precision.
2021-08-13 03:21:19 +09:00
RuQing Xu
3df0e9b653 Arm64 8x4 Kernel Use Less Regs 2021-08-13 02:40:06 +09:00
RuQing Xu
4e7e225057 Armv8-A Supplimentary GEMMSUP Sizes for RD 2021-08-13 02:40:06 +09:00
RuQing Xu
c792d506ba Armv8-A Fix GEMMSUP-RD Kernels on GNU Asm
Suffixed NEON opcode is not supported by GNU assembler
2021-08-13 02:40:06 +09:00
RuQing Xu
ce44735209 Armv8-A Adjust Types for PACKM Kernels
GCC does not have full NEON intrinsics support.
2021-08-13 02:40:06 +09:00
RuQing Xu
8a32d19af8 Armv8-A GEMMSUP-RD 6x8m
Armv8-A now has a complete set of GEMMSUP kernels..
2021-08-13 02:40:06 +09:00
RuQing Xu
afd0fa6ad1 Armv8-A GEMMSUP-RD 6x8n 2021-08-13 02:40:06 +09:00
RuQing Xu
3c5f740514 Armv8-A s/d Packing Kernels Fix Typo
For GCC.
2021-08-13 02:40:06 +09:00
RuQing Xu
49b05df792 Armv8-A Introduced s/d Packing Kernels
Sizes according to the 2014 kernels.
2021-08-13 02:40:06 +09:00
RuQing Xu
c3faf93168 Armv8-A DGEMMSUP 6x8m Kernel
Recommended kernels set:
  ...
  BLIS_RRR, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8m, TRUE,
  BLIS_RCR, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8m, TRUE,
  BLIS_RCC, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8n, TRUE,
  BLIS_CRR, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8m, TRUE,
  BLIS_CCR, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8n, TRUE,
  BLIS_CCC, BLIS_DOUBLE, bli_dgemmsup_rv_armv8a_asm_6x8n, TRUE,
  ...
  bli_blksz_init     ( &blkszs[ BLIS_MR ],    -1,     6,    -1,    -1,
                                              -1,     8,    -1,    -1 );
  bli_blksz_init_easy( &blkszs[ BLIS_NR ],    -1,     8,    -1,    -1 );
  ...
2021-08-13 02:40:06 +09:00
RuQing Xu
3efe707b55 Armv8-A DGEMMSUP Adjustments 2021-08-13 02:40:06 +09:00
RuQing Xu
8ed8f5e625 Armv8-A Add More DGEMMSUP
- Add 6x8 GEMMSUP.
- Adjust prefetching.
- Workaround for Clang's disability to handle reg clobbering.
- Subproduct 6x8 row-major GEMM <- incomplete.
2021-08-13 02:40:06 +09:00
RuQing Xu
a9ba79ea14 Armv8-A Add GEMMSUP 4x8n Kernel
- Compile w/ both GCC & Clang.
- Edge cases use ref-kernels.
- Can give performance boost in some contexts.
2021-08-13 02:40:06 +09:00
RuQing Xu
df40efe8fb Armv8-A Add Part of GEMMSUP 8x4m Kernel
- Compile w/ both GCC & Clang
- Only block part is implement. Edge cases WIP
- Not Optimal kernel scheme. Should do 4x8 instead
2021-08-13 02:40:06 +09:00
RuQing Xu
6639999288 Armv8A DGEMM 4x4 Kernel WIP. Slow
Quite slow.
2021-08-13 02:40:06 +09:00
RuQing Xu
a29c16394c Armv8-A Add 8x4 Kernel WIP
Test result: a bit lower GFlOps than 6x8.
2021-08-13 02:40:04 +09:00
Dipal Madhukar Zambare
0cb552c8f8 Merge "Updated memory pool implementation to contain buffers of different sizes." into amd-staging-milan-3.1 2021-08-12 09:01:02 -04:00
Dipal M Zambare
2eede504b5 Updated memory pool implementation to contain buffers of different sizes.
-- In existing design memory pool supports buffers of only one
     size, This size is determined at compile time to support buffer
     needed for biggest block size and data type. However, the size
     calculation is not generic as it considers sizes only for GEMM.
     Also it assumes that the buffer will not be used for any other
     purpose than packing operations.

  -- If the new buffer is requested whose size is bigger than existing
     size, The pool is re-initialized to contain buffers of new size,
     however, this is done only if the pool is empty. If the pool is not
     empty the execution is aborted. This is undesirable as in
     mulithreaded scenarios it is possible that different threads needs
     buffers of different sizes at the same time (i.e. while other thread
     is still in middle of the operation).

  -- This commit removes the restriction of single size buffer, when
     new buffer is requested whose size if bigger than exiting one, no
     re-init is done. New buffer of required size is allocated, added
     to the pool and returned to the user.

Change-Id: I20acdb60eb06ab2e53366d51713aa83c4b2df0da
2021-08-12 10:46:27 +05:30
Devin Matthews
64a1f786d5 Implement proposed new function pointer fields for obj_t.
The added fields:
1. `pack_t schema`: storing the pack schema on the object allows the macrokernel to act accordingly without side-channel information from the rntm_t and cntx_t. The pack schema and "pack_[ab]" fields could be removed from those structs.
2. `void* user_data`: this field can be used to store any sort of additional information provided by the user. The pointer is propagated to submatrix objects and copies, but is otherwise ignored by the framework and the default implementations of the following three fields. User-specified pack, kernel, or ukr functions can do whatever they want with the data, and the user is 100% responsible for allocating, assigning, and freeing this buffer.
3. `obj_pack_fn_t pack`: the function called when a matrix is packed. This functions receives the expected arguments, as well as a mdim_t and mem_t* as memory must be allocated inside this function, and behavior may differ based on which matrix is being backed (i.e. transposition for B). This could also be achieved by passing a desired pack schema, but this would require additional information to travel down the control tree.
4. `obj_ker_fn_t ker`: the function called when we get to the "second loop", or the macro-kernel. Behavior may depend on the pack schemas of the input matrices. The default implementation would perform the inner two loops around the ukr, and then call either the default ukr or a user-supplied one (next field).
5. `obj_ukr_fn_t ukr`: the function called by the default macrokernel. This would replace the various current "virtual" microkernels, and could also be used to supply user-defined behavior. Users could supply both a custom kernel (above) and microkernel, although the user-specified kernel does **not** necessarily have to call the ukr function specified on the obj_t.

Note that no macros or functions for accessing these new fields have been defined yet. That is next once these are finalized. Addresses https://github.com/flame/blis/projects/1#card-62357687.
2021-08-11 18:11:47 -05:00
Kiran Varaganti
adfd569591 DGEMM Optimizations
Improve DGEMM performance for smaller sizes. AOCL DYNAMIC is incorporated at blas interface to enable
calling bli_dgemm_small when optimum number of threads implied is 1 for (n and k < 10).
Improved smart threading logic for dgemm,
Additional conditions at the blas interface added to invoke bli_dgemm_small.
Removed  N > 3  condition from bli_dgemm_small.

Change-Id: Id751528dfe9de37800b02ffaf765b6c82487093e
2021-08-10 12:34:43 -04:00
Meghana Vankadari
170719e647 Fixed few bugs in GEMMT for non-zen configs
Details:
- Added a check condition in GEMMT native path to choose
  update_triang routines based on whether the kernels are
  row-preferred or column preferred.
- Moved the zen-specific  SUP thresholds under BLIS_CONFIG_EPYC
  macro. For non-zen architectures, it falls back to L3 SUP
  thresholds.
- Modified SUP code path to always choose 2m variant. 1n variant
  is not implemented for GEMMT.

Change-Id: Ifdd55815c588f645e337de80b5b9d1864f6b5dd3
2021-08-10 02:38:17 -04:00
Nallani Bhaskar
29e7f08eb2 Updated aocldtl loging with fflush
Description:
1. Added fflush after fprintf in aocldtl so avoid mising any log
   info when application crashes
2. Removed one redundent if def statement

Change-Id: I79a645060fa36bd8c7e5eaca4d9183dc944329ea
2021-08-09 19:14:50 +05:30
Nallani Bhaskar
40a5a614b2 Updated vector loads/stores in reminder cases to avoid access beyond the matrix boundary
Description:
1. While processing reminder cases in bli_trsm_small algorithm
   there were few loads and stores which were accessing
   beyond the given matrix buffer because of vectorized instructions.

2. Modified 256bit vector loads at edges into 128bit or 64 bit loads/stores
   such that no read/write happens beyond the matrix boundary.

AMD-Internal: [CPUPL-1759] [SWLCSG-819]

Change-Id: Iba51d0ed9bb28d1b0948a219755b8dbcc86a7fa9
2021-08-09 10:51:26 +05:30
Field G. Van Zee
a32257eeab Fixed bli_init.c compile-time error on OSX clang.
Details:
- Fixed a compile-time error in bli_init.c when compiling with OSX's
  clang. This error was introduced in 868b901, which introduced a
  post-declaration struct assignment where the RHS was a struct
  initialization expression (i.e. { ... }). This use of struct
  initializer expressions apparently works with gcc despite it not
  being strict C99. The fix included in this commit declares a temporary
  variable for the purposes of being initialized to the desired value,
  via the struct initializer, and then copies the temporary struct (via
  '=' struct assignment) to the persistent struct. Thanks to Devin
  Matthews for his help with this.
2021-08-05 16:23:02 -05:00
Field G. Van Zee
c8728cfbd1 Fixed configure breakage on OSX clang.
Details:
- Accept either 'clang' or 'LLVM' in vendor string when greping for
  the version number (after determining that we're working with clang).
  Thanks to Devin Matthews for this fix.
2021-08-05 15:17:09 -05:00
Field G. Van Zee
868b90138e Fixed one-time use property of bli_init() (#525).
Details:
- Fixes a rather obvious bug that resulted in segmentation fault
  whenever the calling application tried to re-initialize BLIS after
  its first init/finalize cycle. The bug resulted from the fact that
  the bli_init.c APIs made no effort to allow bli_init() to be called
  subsequent times at all due to it, and bli_finalize(), being
  implemented in terms of pthread_once(). This has been fixed by
  resetting the pthread_once_t control variable for initialization
  at the end of bli_finalize_apis(), and by resetting the control
  variable for finalization at the end of bli_init_apis(). Thanks to
  @lschork2 for reporting this issue (#525), and to Minh Quan Ho and
  Devin Matthews for suggesting the chosen solution.
- CREDITS file update.
2021-08-04 18:31:01 -05:00
Madan mohan Manokar
4b90ae3112 single instance zgemm tuning
1. single instance case sup is enabled.
2. Env BLIS_SINGLE_INSTANCE should be set to 1 to enable single instance tuning.

AMD-Internal: [CPUPL-1743]
Change-Id: Iadb05a6e9313ac41271c0522da243fd47d80abec
2021-07-29 13:36:14 +05:30
Field G. Van Zee
8dba1e752c CREDITS file update. 2021-07-27 12:38:24 -05:00
Manideep Kurumella
33fd2f7398 Merge " BLIS : DGEMV performance improvement for incy/incx greater than 1" into amd-staging-milan-3.1 2021-07-21 10:23:47 -04:00
mkurumel
ce75a86e9b BLIS : DGEMV performance improvement for incy/incx greater than 1
Details :
      - Added packing Of Y for incy >1 cases for dgemv_unf_var2.
      - Added packing Of X for incx >1 cases for dgemv_unf_var1.

    AMD-Internal: [SWLCSG-735]

Change-Id: Ib395f478ba984a85533e4f79b3521d0b2500c30c
2021-07-21 17:55:05 +05:30
Chandrashekara K R
a906ffa8c6 AOCL-Windows: Update BLIS build system.
1. Added the compiler flags for the clang-cl compiler to build blis multithreading using openmp library.
2. Updated format of presenting version string.

AMD Internal : [CPUPL-1630]

Change-Id: I979de541fa57415c08c20b0d5b684ae6bd242d19
2021-07-20 18:56:52 +05:30
Field G. Van Zee
cc9206df66 Added Graviton2 Neoverse N1 performance results.
Details:
- Added single-threaded and multithreaded performance results to
  docs/Performance.md. These results were gathered on a Graviton2
  Neoverse N1 server. Special thanks to Nicholai Tukanov for
  collecting these results via the Arm-HPC/AWS hackaton.
- Corrected what was supposed to be a temporary tweak to the legend
  labels in test/3/octave/plot_l3_perf.m.
2021-07-16 15:48:37 -05:00
Devin Matthews
fab5c86d68 Merge pull request #516 from nicholaiTukanov/p10-sandbox-rework
P10 sandbox rework
2021-07-13 16:46:21 -05:00
Devin Matthews
84f9dcd449 Remove unnecesary windows/zen2 directory. 2021-07-13 16:45:44 -05:00
Field G. Van Zee
21911d6ed3 Merge branch 'dev' 2021-07-09 18:10:46 -05:00
Devin Matthews
17729cf449 Add vzeroupper to Haswell microkernels. (#524)
Details:
- Added vzeroupper instruction to the end of all 'gemm' and 'gemmtrsm' 
  microkernels so as to avoid a performance penalty when mixing AVX
  and SSE instructions. These vzeroupper instructions were once part 
  of the haswell kernels, but were inadvertently removed during a source 
  code shuffle some time ago when we were managing duplicate 'haswell' 
  and 'zen' kernel sets. Thanks to Devin Matthews for tracking this down 
  and re-inserting the missing instructions.
2021-07-09 14:59:48 -05:00
Devin Matthews
c9a7f59aa8 Merge pull request #522 from flame/windows-avx512
Fix Win64 AVX512 bug.
2021-07-08 14:00:38 -05:00
Devin Matthews
9a8e649c5a Fix Win64 AVX512 bug.
Use `-march=haswell` for kernels. Fixes #514.
2021-07-08 11:40:00 -05:00
Devin Matthews
75f03907c5 Add comment about make checkblas on Windows
[ci skip]
2021-07-07 15:44:11 -05:00
Devin Matthews
4651583b12 Merge pull request #520 from flame/travis-ci-install
Test installation in Travis CI
2021-07-07 01:11:20 -05:00
Field G. Van Zee
69205ac266 CREDITS file update.
Details:
- Thanks to Chengguo Sun for submitting #515 (5ef7f68).
- Thanks to Andrew Wildman for submitting #519 (551c6b4).
- Whitespace update to configure (spaces to tabs).
2021-07-06 20:39:22 -05:00
Devin Matthews
174f7fc9a1 Test installation in Travis CI 2021-07-06 19:35:55 -05:00
Devin Matthews
551c6b4ee8 Merge pull request #519 from awild82/oot_build_bugfix
Fix installation from out-of-tree builds
2021-07-06 19:32:53 -05:00
Andrew Wildman
f648df4e55 Add symlink to blis.pc.in for out-of-tree builds 2021-07-06 16:35:12 -07:00
Devin Matthews
78eac6a0ab Revert "Always run make check."
This reverts commit a201a53440.
2021-07-06 11:05:43 -05:00
Devin Matthews
a201a53440 Always run make check.
I'm concerned that problems may lurk for `x86_64` builds on Windows which may be uncovered by a fuller `make check`.
2021-07-05 21:39:18 -05:00
Devin Matthews
5ef7f684dc Merge pull request #515 from chengguosun/bug-fix
Fixed configure script bug.
2021-07-05 21:35:07 -05:00