[PR #213] [MERGED] Talkinghead next2 #233

New issue

Closed

opened 2026-10-15 18:05:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:05:48 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/SillyTavern/SillyTavern-Extras/pull/213
Author: @Technologicat
Created: 1/2/2024
Status: ✅ Merged
Merged: 1/3/2024
Merged by: @Cohee1207

Base: main ← Head: talkinghead-next2

📝 Commits (10+)

6558edb Squashed commit of the following:
a485584 update TODO
51077b5 comment accuracy
956e833 add some more postprocs for an even lofier look
0710a6d improve postproc: add a bad VHS effect
35f3555 update TODO
b598bd6 add low-res analog video filter, update postproc TODO
9201782 adjust badvhs settings
f73fe35 fix badvhs noise
bb6b424 go completely overboard with postproc filters, vol. 1

📊 Changes

9 files changed (+1163 additions, -150 deletions)

View changed files

📝 README.md (+5 -1)
📝 server.py (+20 -2)
➕ talkinghead/README.md (+163 -0)
➕ talkinghead/TODO.md (+81 -0)
📝 talkinghead/emotions/_defaults.json (+2 -2)
📝 talkinghead/tha3/app/app.py (+262 -111)
📝 talkinghead/tha3/app/manual_poser.py (+69 -11)
➕ talkinghead/tha3/app/postprocessor.py (+537 -0)
📝 talkinghead/tha3/app/util.py (+24 -23)

📄 Description

Here's an update to talkinghead.

/emote support via new API endpoint /api/talkinghead/set_emotion (needs https://github.com/SillyTavern/SillyTavern/pull/1622 for frontend support)
Additional performance improvements:
- Run render/encode in parallel to get each new frame ready as soon as possible
- Use PNG compression level 1 for much faster encoding (down to 20ms per frame, from 40ms at default 6); per-frame network send on localhost still under 0.15 ms
- Hit the target output frame rate almost exactly
Prepare for upcoming configurable pixel-space postprocessor (bloom/fake-HDR, scifi hologram, VHS tape, cheap video camera, ...)
Cleanliness: add missing @require_module to talkinghead API endpoints

The postprocessing filters (essentially fragment shaders written in Torch) exist and work, but the postprocessor is temporarily disabled, because we're still missing a way to send it a configuration from the client side. If someone wants to experiment with it in its current state, look at talkinghead/tha3/app/postprocessor.py, and see default_chain, which is a simple data structure containing the (currently hardcoded) settings. Adapt the provided example as desired, and restart your SillyTavern-extras.

I think /emote xxx and this second round of performance improvements could be worth a merge already.

Live mode is still practically usable only on a GPU, due to the posing engine THA3 being a deep-learning model.

Comments?

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/SillyTavern/SillyTavern-Extras/pull/213 **Author:** [@Technologicat](https://github.com/Technologicat) **Created:** 1/2/2024 **Status:** ✅ Merged **Merged:** 1/3/2024 **Merged by:** [@Cohee1207](https://github.com/Cohee1207) **Base:** `main` ← **Head:** `talkinghead-next2` --- ### 📝 Commits (10+) - [`6558edb`](https://github.com/SillyTavern/SillyTavern-Extras/commit/6558edb97f54bbe03870a54dc4bfffcaaf9d8c85) Squashed commit of the following: - [`a485584`](https://github.com/SillyTavern/SillyTavern-Extras/commit/a4855846062bebae316e7c284f31075902e92074) update TODO - [`51077b5`](https://github.com/SillyTavern/SillyTavern-Extras/commit/51077b51d3917e74b6f70652f6c3b8553e39109c) comment accuracy - [`956e833`](https://github.com/SillyTavern/SillyTavern-Extras/commit/956e833e301582ca9ed9767605ec6c42edf8a075) add some more postprocs for an even lofier look - [`0710a6d`](https://github.com/SillyTavern/SillyTavern-Extras/commit/0710a6d669beade1af1ec60658a05139b6627a15) improve postproc: add a bad VHS effect - [`35f3555`](https://github.com/SillyTavern/SillyTavern-Extras/commit/35f3555bd95ca524c8ae20c8b473e34818508cba) update TODO - [`b598bd6`](https://github.com/SillyTavern/SillyTavern-Extras/commit/b598bd668e5741901c4f39e9d2ce136b1f6e6b3c) add low-res analog video filter, update postproc TODO - [`9201782`](https://github.com/SillyTavern/SillyTavern-Extras/commit/9201782f8a57e7a1ba9737357092ca22de357b83) adjust badvhs settings - [`f73fe35`](https://github.com/SillyTavern/SillyTavern-Extras/commit/f73fe3586f922dd2ee6c40bdf25f48f260b80c6c) fix badvhs noise - [`bb6b424`](https://github.com/SillyTavern/SillyTavern-Extras/commit/bb6b424b131d8017e237a977de454ea6982c9c7f) go completely overboard with postproc filters, vol. 1 ### 📊 Changes **9 files changed** (+1163 additions, -150 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+5 -1) 📝 `server.py` (+20 -2) ➕ `talkinghead/README.md` (+163 -0) ➕ `talkinghead/TODO.md` (+81 -0) 📝 `talkinghead/emotions/_defaults.json` (+2 -2) 📝 `talkinghead/tha3/app/app.py` (+262 -111) 📝 `talkinghead/tha3/app/manual_poser.py` (+69 -11) ➕ `talkinghead/tha3/app/postprocessor.py` (+537 -0) 📝 `talkinghead/tha3/app/util.py` (+24 -23) </details> ### 📄 Description Here's an update to `talkinghead`. - `/emote` support via new API endpoint `/api/talkinghead/set_emotion` (needs https://github.com/SillyTavern/SillyTavern/pull/1622 for frontend support) - Additional performance improvements: - Run render/encode in parallel to get each new frame ready as soon as possible - Use PNG compression level `1` for much faster encoding (down to 20ms per frame, from 40ms at default `6`); per-frame network send on localhost still under 0.15 ms - Hit the target output frame rate almost exactly - Prepare for upcoming configurable pixel-space postprocessor (bloom/fake-HDR, scifi hologram, VHS tape, cheap video camera, ...) - Cleanliness: add missing `@require_module` to `talkinghead` API endpoints The postprocessing filters (essentially fragment shaders written in Torch) exist and work, but the postprocessor is temporarily disabled, because we're still missing a way to send it a configuration from the client side. If someone wants to experiment with it in its current state, look at `talkinghead/tha3/app/postprocessor.py`, and see `default_chain`, which is a simple data structure containing the (currently hardcoded) settings. Adapt the provided example as desired, and restart your *SillyTavern-extras*. I think `/emote xxx` and this second round of performance improvements could be worth a merge already. Live mode is still practically usable only on a GPU, due to the posing engine THA3 being a deep-learning model. Comments? --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-15 18:05:48 +00:00