121 Commits

Author	SHA1	Message	Date
PoYen, Chen	5805f5aa73	Remove group mode from appendkv kernel	2024-08-16 10:04:48 +00:00
PoYen, Chen	9de0f35ebc	Remove unused template paremeter	2024-08-13 09:29:20 +00:00
PoYen, Chen	370babc996	Make tile window directly via PageBlockNavigator	2024-08-13 09:18:24 +00:00
PoYen, Chen	3dd6ef61ef	Re-order pipeline paremeters	2024-08-13 07:38:41 +00:00
PoYen, Chen	19c19d8bd3	Only expose necessary methods (not attributes)	2024-08-13 07:26:26 +00:00
PoYen, Chen	c54de6416a	Rename TileWindowNavigator to PageBlockNavigator	2024-08-13 07:23:40 +00:00
PoYen, Chen	d2f5d0910a	Remove no-longer used pipeline files	2024-08-08 17:40:05 +00:00
PoYen, Chen	2f42e4460f	Allow problem types without define kHasDropout attr	2024-08-08 10:53:42 +00:00
PoYen, Chen	a0d2163045	Remove dropout code in splitkv kernel	2024-08-08 10:21:34 +00:00
PoYen, Chen	cef9da0a76	Remove debug macro usages	2024-08-07 15:26:43 +00:00
PoYen, Chen	1b96dc2592	Donot perform write again if already in last page-block	2024-08-07 12:00:15 +00:00
PoYen, Chen	f265742b63	Handle cross-page-block write	2024-08-07 09:33:41 +00:00
PoYen, Chen	40f0d01e29	Allow transit tile_window to another page-block	2024-08-07 09:29:55 +00:00
PoYen, Chen	26ed468ac6	Pass re-created tile window to pipeline	2024-08-07 06:00:17 +00:00
PoYen, Chen	78209c7326	Fix wrong tensor descriptor lengths	2024-08-07 05:59:26 +00:00
PoYen, Chen	7789b53e15	Add tile navigators to the appendkv kernel	2024-08-07 04:51:21 +00:00
PoYen, Chen	443a528adc	Add block_table kernel args for appendkv kernel	2024-08-07 04:27:15 +00:00
PoYen, Chen	15d0034a64	Add paged-kv codegen logic for appendkv kernels	2024-08-07 04:19:45 +00:00
PoYen, Chen	db31475e07	Unify origin	2024-08-06 08:37:29 +00:00
PoYen, Chen	bd0d2f3975	Add batch_stride_k/batch_stride_v in group mode	2024-08-06 08:02:43 +00:00
PoYen, Chen	faf6b0e8ab	Fix wrong origin for bias	2024-08-06 08:02:08 +00:00
PoYen, Chen	f9e2bafd10	Make sure we always start reading complete tile	2024-08-06 03:13:57 +00:00
PoYen, Chen	4fed268723	Move code after decide seqlen_q/seqlen_k	2024-08-06 01:39:49 +00:00
PoYen, Chen	77dac7775c	Move V tile through TileWindowNavigator	2024-08-05 22:36:52 +00:00
PoYen, Chen	ab086bdb76	Simplify more make_tile_window() overloads	2024-08-05 22:16:24 +00:00
PoYen, Chen	bb78353264	Remove ununnecessary data members	2024-08-05 21:52:59 +00:00
PoYen, Chen	8fea4139df	Fix tile window navigation bugs	2024-08-05 21:34:15 +00:00
PoYen, Chen	ecaaa6f136	Simplify TileWindowNavigator interfaces	2024-08-05 16:31:31 +00:00
PoYen, Chen	1c9d77b606	Introduce 'TileWindowNavigator' types	2024-08-05 15:58:41 +00:00
PoYen, Chen	55b77cf962	Add another make_tile_window()	2024-08-05 15:57:03 +00:00
PoYen, Chen	24cb604373	Add copy_const<> type trait	2024-08-05 15:56:15 +00:00
PoYen, Chen	381f7e90e0	Merge branch 'develop' into feature/fmha-fwd-appendkv	2024-08-04 02:12:20 +00:00
PoYen, Chen	baf4a612f0	Fix wrong kernel name	2024-08-02 10:26:47 +00:00
PoYen, Chen	e7969b9fd2	Add template argument 'kIsPagedKV' for splitkv kernels	2024-08-02 10:14:51 +00:00
carlushuang	b3f86e79dd	workaround rocm-6.2 compiler issue (#1421)	2024-07-31 16:03:59 +08:00
PoYen, Chen	c1c50ee498	Enlarge KPerThread for rotary_interleaved=false	2024-07-26 07:09:53 +00:00
PoYen, Chen	bd28e96425	Remove no-longer used method in pipeline	2024-07-24 06:59:45 +00:00
PoYen, Chen	5c733dc568	Remove debug statements	2024-07-24 06:47:52 +00:00
PoYen, Chen	d84c915549	Disable host verification if API not exist	2024-07-24 06:02:41 +00:00
PoYen, Chen	59e1d9b84f	Shift rotary_cos/rotary_sin by cache_seqlen_k	2024-07-24 05:06:47 +00:00
PoYen, Chen	a4da1e7f22	Remove RoPEComputeDataType type alias	2024-07-24 04:45:28 +00:00
PoYen, Chen	251f8cfea9	Merge branch 'develop' into feature/fmha-fwd-appendkv	2024-07-24 04:16:35 +00:00
PoYen, Chen	3348131699	Fix wrong data type for Q rotary_cos/rotary_sin	2024-07-24 04:10:43 +00:00
PoYen, Chen	5ea60715ea	Update host/device specifiers	2024-07-24 03:45:19 +00:00
PoYen, Chen	6f95239229	Use different rotary_cos/rotary_sin distr for Q/Knew	2024-07-24 03:40:29 +00:00
PoYen, Chen	47a74f282d	Extract Q/Knew vector size to helper methods	2024-07-24 03:23:18 +00:00
PoYen, Chen	eb4ea3ac2a	Fix wrong rotary_cos/rotary_sin memory size for Q	2024-07-23 16:22:25 +00:00
PoYen, Chen	b11f92dc4c	Fix wrong shape of knew_host/vnew_host	2024-07-23 14:52:42 +00:00
PoYen, Chen	ca4b208b60	Fix wrong grid size	2024-07-23 14:20:52 +00:00
PoYen, Chen	52b47810bb	Rename more tile size constants	2024-07-23 09:30:05 +00:00