[PR #216] [MERGED] Talkinghead next4: configurability! #236

New issue

Closed

opened 2026-10-15 18:05:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:05:49 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/SillyTavern/SillyTavern-Extras/pull/216
Author: @Technologicat
Created: 1/12/2024
Status: ✅ Merged
Merged: 1/14/2024
Merged by: @Cohee1207

Base: main ← Head: talkinghead-next4

📝 Commits (10+)

54fad16 remove commented-out old blink probability check
51eac8a target_fps will be modifiable, so rename it as lowercase
1f4f2a1 talkinghead now manages its threads internally
f8e1efb startup message: highlight talkinghead device and model
990c5b1 add info message at launch
c529f43 refactor backend settings to allow making them customizable
964ebc8 fix heading level
bac0d48 update description of talkinghead API endpoints in main README
a1f36f3 add new talkinghead options to README
ea835ed newline

📊 Changes

8 files changed (+1541 additions, -985 deletions)

View changed files

📝 README.md (+71 -15)
📝 server.py (+956 -889)
📝 talkinghead/Character_Card_Guide.png (+0 -0)
📝 talkinghead/README.md (+266 -23)
📝 talkinghead/TODO.md (+27 -25)
📝 talkinghead/tha3/app/app.py (+191 -32)
📝 talkinghead/tha3/app/manual_poser.py (+29 -0)
📝 talkinghead/tha3/app/postprocessor.py (+1 -1)

📄 Description

Configurability!
- Add new API endpoints load_emotion_templates, load_animator_settings.
- Needs matching PR for the frontend to actually send the configs, https://github.com/SillyTavern/SillyTavern/pull/1683.
- No GUI yet, just JSON files in the character's folder on the client end.
Docs, docs, docs!
- Document in the talkinghead README how to use the new configurability.
- Minor updates to main README.
Small fixes:
- Talkinghead now manages its threads internally, so there's no need to run launch in a separate thread. Better for predictable ordering of startup messages for the ST-extras server.
- In the startup message of the talkinghead module, highlight the name talkinghead, the CUDA device used, and the model variant. Makes it more explicit that it may use a CUDA device different from that of most other extras modules.
Reorganize server.py to format it more like a standard Python script.
- Standard ordering for imports, then functions first (grouped by topic), and the main program at the end.
- I kept the scoping as-is. I was tempted to move the main program into a main function and call it if __name__ == "__main__", as the pythonic equivalent to the top-level JS IIFE goes, but OTOH, flask makes that difficult (since we need to instantiate app to make @app.route available for the API endpoint definitions), and server.py is not intended to be imported as a module into other scripts anyway. So there would be no practical benefit, other than IDEs being able to catch unused variables.
- @Cohee1207: This is commit 332b4ed. If you don't like this change, I can revert it. My own opinion is that many pythonistas will find the code more quickly readable this way.
Update TODO.
- I noticed a few more per-frame timings that need to be fixed later, particularly the microsway, and dynamic postprocessor effects.

THIS PR IS NOT YET FINAL: EDIT: Now it is. :)

I will add one more feature - namely, the manual poser doesn't yet automatically generate a suitably formatted _emotions.json. This will only take a few lines of code, but I'm out of time right now - I'll finish this either tonight or over the weekend. I'll update the PR once done.

~~Meanwhile, the rest can be reviewed.~~

Tested with the following SillyTavern/public/characters/characternamehere/_animator.json (note: per-character config, at the client end!): EDIT: This configuration is now provided as an example in the talkinghead README.

{"target_fps": 25,
 "pose_interpolator_step": 0.1,
 "blink_interval_min": 2.0,
 "blink_interval_max": 5.0,
 "blink_probability": 0.03,
 "blink_confusion_duration": 10.0,
 "talking_fps": 12,
 "talking_morph": "mouth_aaa_index",
 "sway_morphs": ["head_x_index", "head_y_index", "neck_z_index", "body_y_index", "body_z_index"],
 "sway_interval_min": 5.0,
 "sway_interval_max": 10.0,
 "sway_macro_strength": 0.6,
 "sway_micro_strength": 0.02,
 "breathing_cycle_duration": 4.0,
 "postprocessor_chain": [["bloom", {}],
                         ["translucency", {"alpha": 0.9}],
                         ["alphanoise", {"magnitude": 0.1, "sigma": 0.0}],
                         ["banding", {}],
                         ["scanlines", {"dynamic": true}]
                        ]
}

The rest are defaults, but the "postprocessor_chain" makes the character into a scifi hologram.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/SillyTavern/SillyTavern-Extras/pull/216 **Author:** [@Technologicat](https://github.com/Technologicat) **Created:** 1/12/2024 **Status:** ✅ Merged **Merged:** 1/14/2024 **Merged by:** [@Cohee1207](https://github.com/Cohee1207) **Base:** `main` ← **Head:** `talkinghead-next4` --- ### 📝 Commits (10+) - [`54fad16`](https://github.com/SillyTavern/SillyTavern-Extras/commit/54fad16a161d2e06ae286bc71ef57dd8b3f261cb) remove commented-out old blink probability check - [`51eac8a`](https://github.com/SillyTavern/SillyTavern-Extras/commit/51eac8afe2764633db1809a657643a0ca7151054) target_fps will be modifiable, so rename it as lowercase - [`1f4f2a1`](https://github.com/SillyTavern/SillyTavern-Extras/commit/1f4f2a1ffa4dd1e6c69546338f7c7df7933324a6) talkinghead now manages its threads internally - [`f8e1efb`](https://github.com/SillyTavern/SillyTavern-Extras/commit/f8e1efb5b6d184637d81ffafc2855a8c37f5db46) startup message: highlight talkinghead device and model - [`990c5b1`](https://github.com/SillyTavern/SillyTavern-Extras/commit/990c5b12059d7df1aa773b22f36e25df3a318313) add info message at launch - [`c529f43`](https://github.com/SillyTavern/SillyTavern-Extras/commit/c529f439aaba55699ea784453b1d8f2973d155ba) refactor backend settings to allow making them customizable - [`964ebc8`](https://github.com/SillyTavern/SillyTavern-Extras/commit/964ebc8191e679b6f4b4f5c66652be2e4336714b) fix heading level - [`bac0d48`](https://github.com/SillyTavern/SillyTavern-Extras/commit/bac0d48f6cb0d21169165b034993e0f672065405) update description of talkinghead API endpoints in main README - [`a1f36f3`](https://github.com/SillyTavern/SillyTavern-Extras/commit/a1f36f35479fc6892cf24b738b5278f83e70ec2e) add new talkinghead options to README - [`ea835ed`](https://github.com/SillyTavern/SillyTavern-Extras/commit/ea835ed61b4a12845d43494fbc0adf1990ecaff1) newline ### 📊 Changes **8 files changed** (+1541 additions, -985 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+71 -15) 📝 `server.py` (+956 -889) 📝 `talkinghead/Character_Card_Guide.png` (+0 -0) 📝 `talkinghead/README.md` (+266 -23) 📝 `talkinghead/TODO.md` (+27 -25) 📝 `talkinghead/tha3/app/app.py` (+191 -32) 📝 `talkinghead/tha3/app/manual_poser.py` (+29 -0) 📝 `talkinghead/tha3/app/postprocessor.py` (+1 -1) </details> ### 📄 Description - Configurability! - Add new API endpoints `load_emotion_templates`, `load_animator_settings`. - Needs matching PR for the frontend to actually send the configs, https://github.com/SillyTavern/SillyTavern/pull/1683. - No GUI yet, just JSON files in the character's folder on the client end. - Docs, docs, docs! - Document in the `talkinghead` README how to use the new configurability. - Minor updates to main README. - Small fixes: - Talkinghead now manages its threads internally, so there's no need to run `launch` in a separate thread. Better for predictable ordering of startup messages for the ST-extras server. - In the startup message of the `talkinghead` module, highlight the name `talkinghead`, the CUDA device used, and the model variant. Makes it more explicit that it may use a CUDA device different from that of most other extras modules. - Reorganize `server.py` to format it more like a standard Python script. - Standard ordering for imports, then functions first (grouped by topic), and the main program at the end. - I kept the scoping as-is. I was tempted to move the main program into a `main` function and call it `if __name__ == "__main__"`, as the pythonic equivalent to the top-level JS IIFE goes, but OTOH, `flask` makes that difficult (since we need to instantiate `app` to make `@app.route` available for the API endpoint definitions), and `server.py` is not intended to be imported as a module into other scripts anyway. So there would be no practical benefit, other than IDEs being able to catch unused variables. - @Cohee1207: This is commit 332b4ed. If you don't like this change, I can revert it. My own opinion is that many pythonistas will find the code more quickly readable this way. - Update TODO. - I noticed a few more per-frame timings that need to be fixed later, particularly the microsway, and dynamic postprocessor effects. ~**THIS PR IS NOT YET FINAL**:~ **EDIT**: Now it is. :) ~I will add one more feature - namely, the manual poser doesn't yet automatically generate a suitably formatted `_emotions.json`. This will only take a few lines of code, but I'm out of time right now - I'll finish this either tonight or over the weekend. I'll update the PR once done.~ ~Meanwhile, the rest can be reviewed.~ Tested with the following `SillyTavern/public/characters/characternamehere/_animator.json` (note: per-character config, at the client end!): **EDIT**: *This configuration is now provided as an example in the `talkinghead` README.* ```json {"target_fps": 25, "pose_interpolator_step": 0.1, "blink_interval_min": 2.0, "blink_interval_max": 5.0, "blink_probability": 0.03, "blink_confusion_duration": 10.0, "talking_fps": 12, "talking_morph": "mouth_aaa_index", "sway_morphs": ["head_x_index", "head_y_index", "neck_z_index", "body_y_index", "body_z_index"], "sway_interval_min": 5.0, "sway_interval_max": 10.0, "sway_macro_strength": 0.6, "sway_micro_strength": 0.02, "breathing_cycle_duration": 4.0, "postprocessor_chain": [["bloom", {}], ["translucency", {"alpha": 0.9}], ["alphanoise", {"magnitude": 0.1, "sigma": 0.0}], ["banding", {}], ["scanlines", {"dynamic": true}] ] } ``` The rest are defaults, but the `"postprocessor_chain"` makes the character into a scifi hologram. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-15 18:05:49 +00:00