amd/blis
1
0
Fork 0
mirror of https://github.com/amd/blis.git synced 2026-05-13 10:35:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
c19db2ff826e2ea6ac54569e8aa37e91bdf7cabe
blis/kernels/zen/bli_kernels_zen.h
Field G. Van Zee 88ad841434 Squash-merge 'pr' into 'squash'. (#457) 
Merged contributions from AMD's AOCL BLIS (#448).
  
Details:
- Added support for level-3 operation gemmt, which performs a gemm on
  only the lower or upper triangle of a square matrix C. For now, only
  the conventional/large code path will be supported (in vanilla BLIS).
  This was accomplished by leveraging the existing variant logic for
  herk. However, some of the infrastructure to support a gemmtsup is
  included in this commit, including
  - A bli_gemmtsup() front-end, similar to bli_gemmsup().
  - A bli_gemmtsup_ref() reference handler function.
  - A bli_gemmtsup_int() variant chooser function (with variant calls
    commented out).
- Added support for inducing complex domain gemmt via the 1m method.
- Added gemmt APIs to the BLAS and CBLAS compatiblity layers.
- Added gemmt test module to testsuite.
- Added standalone gemmt test driver to 'test' directory.
- Documented gemmt APIs in BLISObjectAPI.md and BLISTypedAPI.md.
- Added a C++ template header (blis.hh) containing a BLAS-inspired
  wrapper to a set of polymorphic CBLAS-like function wrappers defined
  in another header (cblas.hh). These two headers are installed if
  running the 'install' target with INSTALL_HH is set to 'yes'. (Also
  added a set of unit tests that exercise blis.hh, although they are
  disabled for now because they aren't compatible with out-of-tree
  builds.) These files now live in the 'vendor' top-level directory.
- Various updates to 'zen' and 'zen2' subconfigurations, particularly
  within the context initialization functions.
- Added s and d copyv, setv, and swapv kernels to kernels/zen/1, and
  various minor updates to dotv and scalv kernels. Also added various
  sup kernels contributed by AMD to kernels/zen/3. However, these
  kernels are (for now) not yet used, in part because they caused
  AppVeyor clang failures, and also because I have not found time to
  review and vet them.
- Output the python found during configure into the definition of PYTHON
  in build/config.mk (via build/config.mk.in).
- Added early-return checks (A, B, or C with zero dimension; alpha = 0)
  to bli_gemm_front.c.
- Implemented explicit beta = 0 handling in for the sgemm ukernel in
  bli_gemm_armv7a_int_d4x4.c, which was previously missing. This latent
  bug surfaced because the gemmt module verifies its computation using
  gemm with its beta parameter set to zero, which, on a cortexa15 system
  caused the gemm kernel code to unconditionally multiply the
  uninitialized C data by beta. The C matrix likely contained
  non-numeric values such as NaN, which then would have resulted in a
  false failure.
- Fixed a bug whereby the implementation for bli_herk_determine_kc(),
  in bli_l3_blocksize.c, was inadvertantly being defined in terms of
  helper functions meant for trmm. This bug was probably harmless since
  the trmm code should have also done the right thing for herk.
- Used cpp macros to neutralize the various AOCL_DTL_TRACE_ macros in
  kernels/zen/3/bli_gemm_small.c since those macros are not used in
  vanilla BLIS.
- Added cpp guard to definition of bli_mem_clear() in bli_mem.h to
  accommodate C++'s stricter type checking.
- Added cpp guard to test/*.c drivers that facilitate compilation on
  Windows systems.
- Various whitespace changes.
2020-11-14 09:39:48 -06:00

202 lines
8.2 KiB
C
Raw Blame History

/*
BLIS
An object-based framework for developing high-performance BLAS-like
libraries.
Copyright (C) 2014, The University of Texas at Austin
Copyright (C) 2020, Advanced Micro Devices, Inc.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
- Neither the name(s) of the copyright holder(s) nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
// -- level-1m --
PACKM_KER_PROT(double, d, packm_8xk_gen_zen)
PACKM_KER_PROT(double, d, packm_6xk_gen_zen)
PACKM_KER_PROT(double, d, packm_8xk_nn_zen)
PACKM_KER_PROT(double, d, packm_6xk_nn_zen)
// -- level-1v --
// amaxv (intrinsics)
AMAXV_KER_PROT( float, s, amaxv_zen_int )
AMAXV_KER_PROT( double, d, amaxv_zen_int )
// axpyv (intrinsics)
AXPYV_KER_PROT( float, s, axpyv_zen_int )
AXPYV_KER_PROT( double, d, axpyv_zen_int )
// axpyv (intrinsics unrolled x10)
AXPYV_KER_PROT( float, s, axpyv_zen_int10 )
AXPYV_KER_PROT( double, d, axpyv_zen_int10 )
// dotv (intrinsics)
DOTV_KER_PROT( float, s, dotv_zen_int )
DOTV_KER_PROT( double, d, dotv_zen_int )
// dotv (intrinsics, unrolled x10)
DOTV_KER_PROT( float, s, dotv_zen_int10 )
DOTV_KER_PROT( double, d, dotv_zen_int10 )
// dotxv (intrinsics)
DOTXV_KER_PROT( float, s, dotxv_zen_int )
DOTXV_KER_PROT( double, d, dotxv_zen_int )
// scalv (intrinsics)
SCALV_KER_PROT( float, s, scalv_zen_int )
SCALV_KER_PROT( double, d, scalv_zen_int )
// scalv (intrinsics unrolled x10)
SCALV_KER_PROT( float, s, scalv_zen_int10 )
SCALV_KER_PROT( double, d, scalv_zen_int10 )
// swapv (intrinsics)
SWAPV_KER_PROT(float, s, swapv_zen_int8 )
SWAPV_KER_PROT(double, d, swapv_zen_int8 )
// copyv (intrinsics)
COPYV_KER_PROT( float, s, copyv_zen_int )
COPYV_KER_PROT( double, d, copyv_zen_int )
//
SETV_KER_PROT(float, s, setv_zen_int)
SETV_KER_PROT(double, d, setv_zen_int)
// -- level-1f --
// axpyf (intrinsics)
AXPYF_KER_PROT( float, s, axpyf_zen_int_8 )
AXPYF_KER_PROT( double, d, axpyf_zen_int_8 )
// dotxf (intrinsics)
DOTXF_KER_PROT( float, s, dotxf_zen_int_8 )
DOTXF_KER_PROT( double, d, dotxf_zen_int_8 )
// -- level-3 sup --------------------------------------------------------------
// semmsup_rv
//GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_5x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_4x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_3x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_2x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_1x16 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_5x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_4x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_3x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_2x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_1x8 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_5x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_4x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_3x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_2x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_1x4 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_5x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_4x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_3x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_2x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_1x2 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_6x1 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_5x1 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_4x1 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_3x1 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_2x1 )
GEMMSUP_KER_PROT( float, s, gemmsup_r_zen_ref_1x1 )
// gemmsup_rv (mkernel in m dim)
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x16m )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x8m )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x4m )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x2m )
// gemmsup_rv (mkernel in n dim)
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_6x16n )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_5x16n )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_4x16n )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_3x16n )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_2x16n )
GEMMSUP_KER_PROT( float, s, gemmsup_rv_zen_asm_1x16n )
// gemmsup_rd
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_2x8)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_2x16)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_1x8)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_1x16)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x4)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_2x4)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_1x4)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x2)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_3x2)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_2x2)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_1x2)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x16m)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x8m)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x4m)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x2m)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_6x16n)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_3x16n)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_2x16n)
GEMMSUP_KER_PROT( float, s, gemmsup_rd_zen_asm_1x16n)
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x8m )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x4m )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x2m )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_2x8 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_1x8 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_2x4 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_1x4 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_2x2 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_1x2 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_3x4m )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_3x2m )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_2x4 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_1x4 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_2x2 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_1x2 )
// gemmsup_rv (mkernel in n dim)
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x8n )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_2x8n )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_1x8n )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x4 )
GEMMSUP_KER_PROT( scomplex, c, gemmsup_rv_zen_asm_3x2 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_3x4n )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_2x4n )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_1x4n )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_3x2 )
GEMMSUP_KER_PROT( dcomplex, z, gemmsup_rv_zen_asm_3x1 )
Reference in New Issue View Git Blame Copy Permalink