mirror of
https://github.com/amd/blis.git
synced 2026-04-20 15:48:50 +00:00
Implemented runtime kernel management.
Details: - Reworked the build system around a configuration registry file, named config_registry', that identifies valid configuration targets, their constituent sub-configurations, and the kernel sets that are needed by those sub-configurations. The build system now facilitates the building of a single library that can contains kernels and cache/register blocksizes for multiple configurations (microarchitectures). Reference kernels are also built on a per-configuration basis. - Updated the Makefile to use new variables set by configure via the config.mk.in template, such as CONFIG_LIST, KERNEL_LIST, and KCONFIG_MAP, in determining which sub-configurations (CONFIG_LIST) and kernel sets (KERNEL_LIST) are included in the library, and which make_defs.mk files' CFLAGS (KCONFIG_MAP) are used when compiling kernels. - Reorganized 'kernels' directory into a "flat" structure. Renamed kernel functions into a standard format that includes the kernel set name (e.g. 'haswell'). Created a "bli_kernels_<kernelset>.h" file in each kernels sub-directory. These files exist to provide prototypes for the kernels present in those directories. - Reorganized reference kernels into a top-level 'ref_kernels' directory. This directory includes a new source file, bli_cntx_ref.c (compiled on a per-configuration basis), that defines the code needed to initialize a reference context and a context for induced methods for the microarchitecture in question. - Rewrote make_defs.mk files in each configuration so that the compiler variables (e.g. CFLAGS) are "stored" (renamed) on a per-configuration basis. - Modified bli_config.h.in template so that bli_config.h is generated with #defines for the config (family) name, the sub-configurations that are associated with the family, and the kernel sets needed by those sub-configurations. - Deprecated all kernel-related information in bli_kernel.h and transferred what remains to new header files named "bli_arch_<configname>.h", which are conditionally #included from a new header bli_arch.h. These files are still needed to set library-wide parameters such as custom malloc()/free() functions or SIMD alignment values. - Added bli_cntx_init_<configname>.c files to each configuration directory. The files contain a function, named the same as the file, that initializes a "native" context for a particular configuration (microarchitecture). The idea is that optimized kernels, if available, will be initialized into these contexts. Other fields will retain pointers to reference functions, which will be compiled on a per-configuration basis. These bli_cntx_init_*() functions will be called during the initialization of the global kernel structure. They are thought of as initializing for "native" execution, but they also form the basis for contexts that use induced methods. These functions are prototyped, along with their _ref() and _ind() brethren, by prototype-generating macros in bli_arch.h. - Added a new typedef enum in bli_type_defs.h to define an arch_t, which identifies the various sub-configurations. - Redesigned the global kernel structure (gks) around a 2D array of cntx_t structures (pointers to cntx_t, actually). The first dimension is indexed over arch_t and the inner dimension is the ind_t (induced method) for each microarchitecture. When a microarchitecture (configuration) is "registered" at init-time, the inner array for that configuration in the 2D array is initialized (and allocated, if it hasn't been already). The cntx_t slot for BLIS_NAT is initialized immediately and those for other induced method types are initialized and cached on-demand, as needed. At cntx_t registration, we also store function pointers to cntx_init functions that will initialize (a) "reference" contexts and (b) contexts for use with induced methods. We don't cache the full contexts for reference contexts since they are rarely needed. The functions that initialize these two kinds of contexts are generated automatically for each targeted sub-configuration from cpp-templatized code at compile-time. Induced method contexts that need "stage" adjustments can still obtain them via functions in bli_cntx_ind_stage.c. - Added new functions and functionality to bli_cntx.c, such as for setting the level-1f, level-1v, and packm kernels, and for converting a native context into one for executing an induced method. - Moved the checking of register/cache blocksize consistency from being cpp macros in bli_kernel_macro_defs.h to being runtime checks defined in bli_check.c and called from bli_gks_register_cntx() at the time that the global kernel structure's internal context is initialized for a given microarchitecture/configuration. - Deprecated all of the old per-operation bli_*_cntx.c files and removed the previous operation-level cntx_t_init()/_finalize() invocations. Instead, we now query the gks for a suitable context, usually via bli_gks_query_cntx(). - Deprecated support for the 3m2 and 3m3 induced methods. (They required hackery that I was no longer willing to support.) - Consolidated the 1e and 1r packm kernels for any given register blocksize into a single kernel that will branch on the schema and support packing to both formats. - Added the cntx_t* argument to all packm kernel signatures. - Deprecated the local function pointer array in all bli_packm_cxk*.c files and instead obtain the packm kernel from the cntx_t. - Added bli_calloc_intl(), which serves as the calloc-equivalent to to bli_malloc_intl(). Useful when we wish to allocate and initialize to zero/NULL. - Converted existing cpp macro functions defined in bli_blksz.h, bli_func.h, bli_cntx.h into static functions.
This commit is contained in:
Field G. Van Zee
136
ref_kernels/3/bli_gemm_ref.c
Normal file
136
ref_kernels/3/bli_gemm_ref.c
Normal file
@@ -0,0 +1,136 @@
|
||||
/*
|
||||
|
||||
BLIS
|
||||
An object-based framework for developing high-performance BLAS-like
|
||||
libraries.
|
||||
|
||||
Copyright (C) 2014, The University of Texas at Austin
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
- Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
- Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in the
|
||||
documentation and/or other materials provided with the distribution.
|
||||
- Neither the name of The University of Texas at Austin nor the names
|
||||
of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
*/
|
||||
|
||||
#include "blis.h"
|
||||
|
||||
#undef GENTFUNC
|
||||
#define GENTFUNC( ctype, ch, opname, arch, suf ) \
|
||||
\
|
||||
void PASTEMAC3(ch,opname,arch,suf) \
|
||||
( \
|
||||
dim_t k, \
|
||||
ctype* restrict alpha, \
|
||||
ctype* restrict a, \
|
||||
ctype* restrict b, \
|
||||
ctype* restrict beta, \
|
||||
ctype* restrict c, inc_t rs_c, inc_t cs_c, \
|
||||
auxinfo_t* restrict data, \
|
||||
cntx_t* restrict cntx \
|
||||
) \
|
||||
{ \
|
||||
const num_t dt = PASTEMAC(ch,type); \
|
||||
\
|
||||
const dim_t mr = bli_cntx_get_blksz_def_dt( dt, BLIS_MR, cntx ); \
|
||||
const dim_t nr = bli_cntx_get_blksz_def_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const inc_t packmr = bli_cntx_get_blksz_max_dt( dt, BLIS_MR, cntx ); \
|
||||
const inc_t packnr = bli_cntx_get_blksz_max_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const dim_t m = mr; \
|
||||
const dim_t n = nr; \
|
||||
\
|
||||
const inc_t cs_a = packmr; \
|
||||
\
|
||||
const inc_t rs_b = packnr; \
|
||||
\
|
||||
ctype ab[ BLIS_STACK_BUF_MAX_SIZE \
|
||||
/ sizeof( ctype ) ] \
|
||||
__attribute__((aligned(BLIS_STACK_BUF_ALIGN_SIZE))); \
|
||||
const inc_t rs_ab = 1; \
|
||||
const inc_t cs_ab = mr; \
|
||||
\
|
||||
dim_t l, j, i; \
|
||||
\
|
||||
ctype ai; \
|
||||
ctype bj; \
|
||||
\
|
||||
\
|
||||
/* Initialize the accumulator elements in ab to zero. */ \
|
||||
for ( i = 0; i < m * n; ++i ) \
|
||||
{ \
|
||||
PASTEMAC(ch,set0s)( *(ab + i) ); \
|
||||
} \
|
||||
\
|
||||
/* Perform a series of k rank-1 updates into ab. */ \
|
||||
for ( l = 0; l < k; ++l ) \
|
||||
{ \
|
||||
ctype* restrict abij = ab; \
|
||||
\
|
||||
/* In an optimized implementation, these two loops over MR and NR
|
||||
are typically fully unrolled. */ \
|
||||
for ( j = 0; j < n; ++j ) \
|
||||
{ \
|
||||
bj = *(b + j); \
|
||||
\
|
||||
for ( i = 0; i < m; ++i ) \
|
||||
{ \
|
||||
ai = *(a + i); \
|
||||
\
|
||||
PASTEMAC(ch,dots)( ai, bj, *abij ); \
|
||||
\
|
||||
abij += rs_ab; \
|
||||
} \
|
||||
} \
|
||||
\
|
||||
a += cs_a; \
|
||||
b += rs_b; \
|
||||
} \
|
||||
\
|
||||
/* Scale the result in ab by alpha. */ \
|
||||
for ( i = 0; i < m * n; ++i ) \
|
||||
{ \
|
||||
PASTEMAC(ch,scals)( *alpha, *(ab + i) ); \
|
||||
} \
|
||||
\
|
||||
/* If beta is zero, overwrite c with the scaled result in ab. Otherwise,
|
||||
scale by beta and then add the scaled redult in ab. */ \
|
||||
if ( PASTEMAC(ch,eq0)( *beta ) ) \
|
||||
{ \
|
||||
PASTEMAC(ch,copys_mxn)( m, \
|
||||
n, \
|
||||
ab, rs_ab, cs_ab, \
|
||||
c, rs_c, cs_c ); \
|
||||
} \
|
||||
else \
|
||||
{ \
|
||||
PASTEMAC(ch,xpbys_mxn)( m, \
|
||||
n, \
|
||||
ab, rs_ab, cs_ab, \
|
||||
beta, \
|
||||
c, rs_c, cs_c ); \
|
||||
} \
|
||||
}
|
||||
|
||||
INSERT_GENTFUNC_BASIC2( gemm, BLIS_CNAME_INFIX, BLIS_REF_SUFFIX )
|
||||
|
||||
102
ref_kernels/3/bli_gemmtrsm_ref.c
Normal file
102
ref_kernels/3/bli_gemmtrsm_ref.c
Normal file
@@ -0,0 +1,102 @@
|
||||
/*
|
||||
|
||||
BLIS
|
||||
An object-based framework for developing high-performance BLAS-like
|
||||
libraries.
|
||||
|
||||
Copyright (C) 2014, The University of Texas at Austin
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
- Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
- Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in the
|
||||
documentation and/or other materials provided with the distribution.
|
||||
- Neither the name of The University of Texas at Austin nor the names
|
||||
of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
*/
|
||||
|
||||
#include "blis.h"
|
||||
|
||||
#undef GENTFUNC
|
||||
#define GENTFUNC( ctype, ch, opname, arch, suf, trsmkerid ) \
|
||||
\
|
||||
void PASTEMAC3(ch,opname,arch,suf) \
|
||||
( \
|
||||
dim_t k, \
|
||||
ctype* restrict alpha, \
|
||||
ctype* restrict a1x, \
|
||||
ctype* restrict a11, \
|
||||
ctype* restrict bx1, \
|
||||
ctype* restrict b11, \
|
||||
ctype* restrict c11, inc_t rs_c, inc_t cs_c, \
|
||||
auxinfo_t* restrict data, \
|
||||
cntx_t* restrict cntx \
|
||||
) \
|
||||
{ \
|
||||
const num_t dt = PASTEMAC(ch,type); \
|
||||
\
|
||||
const inc_t packnr = bli_cntx_get_blksz_max_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const inc_t rs_b = packnr; \
|
||||
const inc_t cs_b = 1; \
|
||||
\
|
||||
ctype* minus_one = PASTEMAC(ch,m1); \
|
||||
\
|
||||
PASTECH(ch,gemm_ukr_ft) \
|
||||
gemm_ukr = bli_cntx_get_l3_ukr_dt( dt, BLIS_GEMM_UKR, cntx ); \
|
||||
PASTECH(ch,trsm_ukr_ft) \
|
||||
trsm_ukr = bli_cntx_get_l3_ukr_dt( dt, trsmkerid, cntx ); \
|
||||
\
|
||||
/* lower: b11 = alpha * b11 - a10 * b01; */ \
|
||||
/* upper: b11 = alpha * b11 - a12 * b21; */ \
|
||||
gemm_ukr \
|
||||
( \
|
||||
k, \
|
||||
minus_one, \
|
||||
a1x, \
|
||||
bx1, \
|
||||
alpha, \
|
||||
b11, rs_b, cs_b, \
|
||||
data, \
|
||||
cntx \
|
||||
); \
|
||||
\
|
||||
/* b11 = inv(a11) * b11;
|
||||
c11 = b11; */ \
|
||||
trsm_ukr \
|
||||
( \
|
||||
a11, \
|
||||
b11, \
|
||||
c11, rs_c, cs_c, \
|
||||
data, \
|
||||
cntx \
|
||||
); \
|
||||
\
|
||||
/*
|
||||
PASTEMAC(d,fprintm)( stdout, "gemmtrsm_ukr: b0111p_r after", k+3, 8, \
|
||||
( double* )b01, 2*PASTEMAC(ch,packnr), 2, "%4.1f", "" ); \
|
||||
PASTEMAC(d,fprintm)( stdout, "gemmtrsm_ukr: b0111p_i after", k+3, 8, \
|
||||
( double* )b01 + 1, 2*PASTEMAC(ch,packnr), 2, "%4.1f", "" ); \
|
||||
*/ \
|
||||
}
|
||||
|
||||
INSERT_GENTFUNC_BASIC3( gemmtrsm_l, BLIS_CNAME_INFIX, BLIS_REF_SUFFIX, BLIS_TRSM_L_UKR )
|
||||
INSERT_GENTFUNC_BASIC3( gemmtrsm_u, BLIS_CNAME_INFIX, BLIS_REF_SUFFIX, BLIS_TRSM_U_UKR )
|
||||
|
||||
199
ref_kernels/3/bli_trsm_ref.c
Normal file
199
ref_kernels/3/bli_trsm_ref.c
Normal file
@@ -0,0 +1,199 @@
|
||||
/*
|
||||
|
||||
BLIS
|
||||
An object-based framework for developing high-performance BLAS-like
|
||||
libraries.
|
||||
|
||||
Copyright (C) 2014, The University of Texas at Austin
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
- Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
- Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in the
|
||||
documentation and/or other materials provided with the distribution.
|
||||
- Neither the name of The University of Texas at Austin nor the names
|
||||
of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
*/
|
||||
|
||||
#include "blis.h"
|
||||
|
||||
#undef GENTFUNC
|
||||
#define GENTFUNC( ctype, ch, opname, arch, suf ) \
|
||||
\
|
||||
void PASTEMAC3(ch,opname,arch,suf) \
|
||||
( \
|
||||
ctype* restrict a, \
|
||||
ctype* restrict b, \
|
||||
ctype* restrict c, inc_t rs_c, inc_t cs_c, \
|
||||
auxinfo_t* restrict data, \
|
||||
cntx_t* restrict cntx \
|
||||
) \
|
||||
{ \
|
||||
const num_t dt = PASTEMAC(ch,type); \
|
||||
\
|
||||
const dim_t mr = bli_cntx_get_blksz_def_dt( dt, BLIS_MR, cntx ); \
|
||||
const dim_t nr = bli_cntx_get_blksz_def_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const inc_t packmr = bli_cntx_get_blksz_max_dt( dt, BLIS_MR, cntx ); \
|
||||
const inc_t packnr = bli_cntx_get_blksz_max_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const dim_t m = mr; \
|
||||
const dim_t n = nr; \
|
||||
\
|
||||
const inc_t rs_a = 1; \
|
||||
const inc_t cs_a = packmr; \
|
||||
\
|
||||
const inc_t rs_b = packnr; \
|
||||
const inc_t cs_b = 1; \
|
||||
\
|
||||
dim_t iter, i, j, l; \
|
||||
dim_t n_behind; \
|
||||
\
|
||||
for ( iter = 0; iter < m; ++iter ) \
|
||||
{ \
|
||||
i = iter; \
|
||||
n_behind = i; \
|
||||
\
|
||||
ctype* restrict alpha11 = a + (i )*rs_a + (i )*cs_a; \
|
||||
ctype* restrict a10t = a + (i )*rs_a + (0 )*cs_a; \
|
||||
ctype* restrict B0 = b + (0 )*rs_b + (0 )*cs_b; \
|
||||
ctype* restrict b1 = b + (i )*rs_b + (0 )*cs_b; \
|
||||
\
|
||||
/* b1 = b1 - a10t * B0; */ \
|
||||
/* b1 = b1 / alpha11; */ \
|
||||
for ( j = 0; j < n; ++j ) \
|
||||
{ \
|
||||
ctype* restrict b01 = B0 + (0 )*rs_b + (j )*cs_b; \
|
||||
ctype* restrict beta11 = b1 + (0 )*rs_b + (j )*cs_b; \
|
||||
ctype* restrict gamma11 = c + (i )*rs_c + (j )*cs_c; \
|
||||
ctype beta11c = *beta11; \
|
||||
ctype rho11; \
|
||||
\
|
||||
/* beta11 = beta11 - a10t * b01; */ \
|
||||
PASTEMAC(ch,set0s)( rho11 ); \
|
||||
for ( l = 0; l < n_behind; ++l ) \
|
||||
{ \
|
||||
ctype* restrict alpha10 = a10t + (l )*cs_a; \
|
||||
ctype* restrict beta01 = b01 + (l )*rs_b; \
|
||||
\
|
||||
PASTEMAC(ch,axpys)( *alpha10, *beta01, rho11 ); \
|
||||
} \
|
||||
PASTEMAC(ch,subs)( rho11, beta11c ); \
|
||||
\
|
||||
/* beta11 = beta11 / alpha11; */ \
|
||||
/* NOTE: The INVERSE of alpha11 (1.0/alpha11) is stored instead
|
||||
of alpha11, so we can multiply rather than divide. We store
|
||||
the inverse of alpha11 intentionally to avoid expensive
|
||||
division instructions within the micro-kernel. */ \
|
||||
PASTEMAC(ch,scals)( *alpha11, beta11c ); \
|
||||
\
|
||||
/* Output final result to matrix c. */ \
|
||||
PASTEMAC(ch,copys)( beta11c, *gamma11 ); \
|
||||
\
|
||||
/* Store the local value back to b11. */ \
|
||||
PASTEMAC(ch,copys)( beta11c, *beta11 ); \
|
||||
} \
|
||||
} \
|
||||
}
|
||||
|
||||
INSERT_GENTFUNC_BASIC2( trsm_l, BLIS_CNAME_INFIX, BLIS_REF_SUFFIX )
|
||||
|
||||
|
||||
#undef GENTFUNC
|
||||
#define GENTFUNC( ctype, ch, opname, arch, suf ) \
|
||||
\
|
||||
void PASTEMAC3(ch,opname,arch,suf) \
|
||||
( \
|
||||
ctype* restrict a, \
|
||||
ctype* restrict b, \
|
||||
ctype* restrict c, inc_t rs_c, inc_t cs_c, \
|
||||
auxinfo_t* restrict data, \
|
||||
cntx_t* restrict cntx \
|
||||
) \
|
||||
{ \
|
||||
const num_t dt = PASTEMAC(ch,type); \
|
||||
\
|
||||
const dim_t mr = bli_cntx_get_blksz_def_dt( dt, BLIS_MR, cntx ); \
|
||||
const dim_t nr = bli_cntx_get_blksz_def_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const inc_t packmr = bli_cntx_get_blksz_max_dt( dt, BLIS_MR, cntx ); \
|
||||
const inc_t packnr = bli_cntx_get_blksz_max_dt( dt, BLIS_NR, cntx ); \
|
||||
\
|
||||
const dim_t m = mr; \
|
||||
const dim_t n = nr; \
|
||||
\
|
||||
const inc_t rs_a = 1; \
|
||||
const inc_t cs_a = packmr; \
|
||||
\
|
||||
const inc_t rs_b = packnr; \
|
||||
const inc_t cs_b = 1; \
|
||||
\
|
||||
dim_t iter, i, j, l; \
|
||||
dim_t n_behind; \
|
||||
\
|
||||
for ( iter = 0; iter < m; ++iter ) \
|
||||
{ \
|
||||
i = m - iter - 1; \
|
||||
n_behind = iter; \
|
||||
\
|
||||
ctype* restrict alpha11 = a + (i )*rs_a + (i )*cs_a; \
|
||||
ctype* restrict a12t = a + (i )*rs_a + (i+1)*cs_a; \
|
||||
ctype* restrict b1 = b + (i )*rs_b + (0 )*cs_b; \
|
||||
ctype* restrict B2 = b + (i+1)*rs_b + (0 )*cs_b; \
|
||||
\
|
||||
/* b1 = b1 - a12t * B2; */ \
|
||||
/* b1 = b1 / alpha11; */ \
|
||||
for ( j = 0; j < n; ++j ) \
|
||||
{ \
|
||||
ctype* restrict beta11 = b1 + (0 )*rs_b + (j )*cs_b; \
|
||||
ctype* restrict b21 = B2 + (0 )*rs_b + (j )*cs_b; \
|
||||
ctype* restrict gamma11 = c + (i )*rs_c + (j )*cs_c; \
|
||||
ctype beta11c = *beta11; \
|
||||
ctype rho11; \
|
||||
\
|
||||
/* beta11 = beta11 - a12t * b21; */ \
|
||||
PASTEMAC(ch,set0s)( rho11 ); \
|
||||
for ( l = 0; l < n_behind; ++l ) \
|
||||
{ \
|
||||
ctype* restrict alpha12 = a12t + (l )*cs_a; \
|
||||
ctype* restrict beta21 = b21 + (l )*rs_b; \
|
||||
\
|
||||
PASTEMAC(ch,axpys)( *alpha12, *beta21, rho11 ); \
|
||||
} \
|
||||
PASTEMAC(ch,subs)( rho11, beta11c ); \
|
||||
\
|
||||
/* beta11 = beta11 / alpha11; */ \
|
||||
/* NOTE: The INVERSE of alpha11 (1.0/alpha11) is stored instead
|
||||
of alpha11, so we can multiply rather than divide. We store
|
||||
the inverse of alpha11 intentionally to avoid expensive
|
||||
division instructions within the micro-kernel. */ \
|
||||
PASTEMAC(ch,scals)( *alpha11, beta11c ); \
|
||||
\
|
||||
/* Output final result to matrix c. */ \
|
||||
PASTEMAC(ch,copys)( beta11c, *gamma11 ); \
|
||||
\
|
||||
/* Store the local value back to b11. */ \
|
||||
PASTEMAC(ch,copys)( beta11c, *beta11 ); \
|
||||
} \
|
||||
} \
|
||||
}
|
||||
|
||||
INSERT_GENTFUNC_BASIC2( trsm_u, BLIS_CNAME_INFIX, BLIS_REF_SUFFIX )
|
||||
|
||||
53
ref_kernels/3/old/bli_l3_ref.h
Normal file
53
ref_kernels/3/old/bli_l3_ref.h
Normal file
@@ -0,0 +1,53 @@
|
||||
/*
|
||||
|
||||
BLIS
|
||||
An object-based framework for developing high-performance BLAS-like
|
||||
libraries.
|
||||
|
||||
Copyright (C) 2014, The University of Texas at Austin
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
- Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
- Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in the
|
||||
documentation and/or other materials provided with the distribution.
|
||||
- Neither the name of The University of Texas at Austin nor the names
|
||||
of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
*/
|
||||
|
||||
// Redefine level-3 micro-kernel API names to induce prototypes.
|
||||
|
||||
#undef gemm_ukr_name
|
||||
#define gemm_ukr_name gemm_ref
|
||||
|
||||
#undef gemmtrsm_l_ukr_name
|
||||
#define gemmtrsm_l_ukr_name gemmtrsm_l_ref
|
||||
#undef gemmtrsm_u_ukr_name
|
||||
#define gemmtrsm_u_ukr_name gemmtrsm_u_ref
|
||||
|
||||
#undef trsm_l_ukr_name
|
||||
#define trsm_l_ukr_name trsm_l_ref
|
||||
#undef trsm_u_ukr_name
|
||||
#define trsm_u_ukr_name trsm_u_ref
|
||||
|
||||
// Include the micro-kernel API template.
|
||||
|
||||
#include "bli_l3_ukr.h"
|
||||
|
||||
Reference in New Issue
Block a user