1191 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
turboderp	1d44bd6279	gpt-oss: Add recurrent SWA mode (default)	2026-07-11 23:55:35 +02:00
turboderp	6312448a8f	SlidingAttention: Add sink support	2026-07-11 23:55:22 +02:00
turboderp	1efa149bca	Gemma4/TP: Add missing TP exports needed by Gemma4, enable TP support	2026-07-11 23:01:32 +02:00
turboderp	b86ae91123	TP: Add TP boilerplate for SlidingAttention, add global checkpoint handle (to avoid possible collisions)	2026-07-11 23:00:47 +02:00
turboderp	7c8eb04cab	gpt-oss: TODO item	2026-07-11 22:57:21 +02:00
turboderp	b80d461993	BlockSparseMLP/TP: Allocate stable broadcast buffers for bsz1 graph	2026-07-11 21:22:30 +02:00
turboderp	2aa9b1603e	TP: Use round-to-nearest for BF16 downcast during native all-reduce	2026-07-11 21:21:05 +02:00
turboderp	635bae660b	BlockSparseMLP: Fix moe_bias_add_kernel expert indexing	2026-07-11 21:20:01 +02:00
turboderp	47c2d47df4	BlockSparseMLP: Add graph patching for run_bsz1	2026-07-11 21:19:32 +02:00
turboderp	135f140bbc	chat.py: Change /think for gpt-oss to mean "high" reasoning	2026-07-11 21:18:35 +02:00
turboderp	76cd9990aa	chat.py: Formatting	2026-07-11 21:18:11 +02:00
turboderp	f1a467bf6b	gpt-oss: Add TP plumbing, enable TP	2026-07-11 18:43:42 +02:00
turboderp	010fa7f88f	GatedRMSNorm: Initialize BC after module import	2026-07-11 16:52:44 +02:00
turboderp	8436248e01	TP: Fix NullConfig regression	2026-07-11 16:08:03 +02:00
turboderp	e9d2c757ca	Attention: Make sure quant-direct calls select from qc-aware backends preventing cache from dropping on the second chunk of a long prefill	2026-07-11 16:07:01 +02:00
turboderp	0aa4e48ea8	gpt-oss: Flag TP as unsupported (pending review)	2026-07-11 16:04:27 +02:00
turboderp	afb59b826b	chat.py: Increase default cache size and response length	2026-07-11 16:04:02 +02:00
turboderp	62598fc6a2	AsyncGenerator: Fix cancellation logic.	2026-07-11 12:12:45 +02:00
turboderp	72ed8485ca	chat.py: Add /python command and basic bwrap sandbox	2026-07-11 02:14:34 +02:00
turboderp	9ca4e92368	chat.py: Add harmony console mode (gpt-oss)	2026-07-11 00:46:44 +02:00
turboderp	386c467647	convert.py: Make mul1 the default codebook for new models	2026-07-11 00:26:03 +02:00
turboderp	5abaac7779	DFlash: Support direct quantized cache updates without FP16 staging	2026-07-11 00:25:32 +02:00
turboderp	95d3bdee43	Attention/Cache: Allow non-causal-spans with quantized cache without FP16 staging, remove redundant placeholder while loading, fix kernel shmem overflow when K and V bitrate exceed 13 bits total	2026-07-11 00:24:51 +02:00
turboderp	74b3cec2bf	requirements: Unpin pydantic, update setup.py install_requires	2026-07-10 23:17:09 +02:00
turboderp	d8cd7fe84b	Formatron: Monkey-patch the pydantic.typing namespace to restore missing aliases causing formatron to fail to import on recent pydantic versions	2026-07-10 22:31:11 +02:00
turboderp	1afffa690b	Merge remote-tracking branch 'origin/dev' into dev	2026-07-10 22:05:57 +02:00
turboderp	65d3eb02c7	Merge pull request #240 from caipiralink/fix/gemma4-serving ... Fix non-contiguous attn output crash and formatron import guard	2026-07-10 22:05:47 +02:00
turboderp	449ea5bd8d	Gemma4: Correctly handle atomic prefill when multimodal token span is longer than the sliding attn window	2026-07-10 22:02:46 +02:00
turboderp	bba588ceb4	Merge pull request #241 from caipiralink/fix/atomic-mm-recurrent-desync ... Fix recurrent state desync after atomic MM prefill	2026-07-10 21:24:29 +02:00
turboderp	4857d9b57a	Support gpt-oss, includes: expert bias, swiglu_oai activation fn, trimmed output in Attention and BlockSparseMLP (for padded hidden_size), attention sinks boilerplate	2026-07-10 19:53:11 +02:00
turboderp	721b164d2b	Linear: Add MXFP4 dequant-on-load and padding (to support models with hidden_size not divisible by 128)	2026-07-10 19:41:49 +02:00
turboderp	6af187349e	ppl.py: Set dequant mode for MXFP4 tensors in HF path, explicitly move input IDs to model device	2026-07-10 15:01:49 +02:00
turboderp	e0bcd47704	RMSNorm: Add contiguous() guard before fused residual	2026-07-10 15:00:49 +02:00
turboderp	978b15ba79	safetensors: Recognize U8 dtype	2026-07-10 14:59:27 +02:00
turboderp	a73786c930	RoPE: Support YaRN truncate argument	2026-07-10 14:58:56 +02:00
turboderp	4b7499616d	Attn: Support learned attention sinks	2026-07-10 02:02:38 +02:00
turboderp	b737db9322	DFlash: Remove dedicated DFlashAttention module (last place FA2 was required), specialize Attention instead, make FA2 an optional dependency	2026-07-10 00:42:57 +02:00
turboderp	7f0215d286	SlidingAttention: Add C++/graph path	2026-07-09 23:39:34 +02:00
turboderp	588e7c66ac	SlidingAttention: Replace FA2 code path, update Triton attn kernel with SWA dual-KV-input mode	2026-07-09 22:56:36 +02:00
turboderp	a82cafa740	Remove xformers requirement, respect ImportError as well as ModuleNotFoundError	2026-07-09 21:31:35 +02:00
turboderp	946c704c47	Add env_vars.md	2026-07-09 21:16:52 +02:00
turboderp	f2240dcf6c	Enable int8 mode by default (mul1 only), enable attn BC/graph by default	2026-07-09 21:14:19 +02:00
turboderp	52b47f0365	Attn: Fix excessive Triton recompiles over sequence length	2026-07-09 20:58:43 +02:00
turboderp	374ac60df7	Attn: Add C++/graph path for bsz <= 4, q_len <= 16	2026-07-09 20:22:43 +02:00
turboderp	91404f4e5f	MLP: Enable BC/graph path without fused up/gate proj (int8 mode)	2026-07-09 20:19:01 +02:00
turboderp	68d1eafe6a	GEMV: Don't disable mgemm when int8 mode enabled and tensors use mcg codebook	2026-07-09 19:50:54 +02:00
turboderp	c20c1fdb8c	Graph: Add quant_cache_paged graph boilerplate	2026-07-09 19:16:38 +02:00
turboderp	b15d0bb511	Graph: Add rope graph boilerplate	2026-07-09 19:15:47 +02:00
turboderp	0f3d34d12f	Graph: Add CUDA driver API helpers and wrapper class for ahead-of-time compiled Triton kernels (cubins), extend graph wrapper with ability to patch Triton nodes via driver	2026-07-09 19:14:07 +02:00
turboderp	178984a59d	Fix missing includes	2026-07-09 19:07:46 +02:00