Commit Graph

513 Commits

Author SHA1 Message Date
Juha Jeronen
f9804dc30c add support for "/emote xxx" in talkinghead 2024-01-02 02:26:51 +02:00
Juha Jeronen
45c2ae5ea5 add missing @require_module for talkinghead API endpoints 2024-01-02 02:26:05 +02:00
Juha Jeronen
205cffa37d update TODO 2024-01-01 15:13:51 +02:00
Juha Jeronen
fe81d56f15 update section on talkinghead in main README 2023-12-31 04:03:12 +02:00
Juha Jeronen
c1c538f443 update TODO 2023-12-31 03:49:21 +02:00
Juha Jeronen
7876ecbe99 update TODO: frame timing good; PNG ok at compression level 1 2023-12-31 03:13:39 +02:00
Juha Jeronen
0b6c42a72c improve explanation 2023-12-31 03:08:22 +02:00
Juha Jeronen
e8dfc754ff target FPS: 25 2023-12-31 03:08:14 +02:00
Juha Jeronen
1b0746f327 first working frame sync that saves compute at low target FPS 2023-12-31 02:20:23 +02:00
Juha Jeronen
f134b5e448 wording 2023-12-31 01:49:58 +02:00
Juha Jeronen
5d5ccf14ef wording once more 2023-12-31 01:45:55 +02:00
Juha Jeronen
82115f60b5 use smallest possible critical section; wording 2023-12-31 01:41:35 +02:00
Juha Jeronen
4fc42fc899 improve per-frame time reporting; outline new strategy 2023-12-31 01:19:24 +02:00
Juha Jeronen
27e7133e18 much better frame timing
Render, encode, send in separate threads.
2023-12-30 03:13:17 +02:00
Juha Jeronen
f845ce42fc better type signature, correctly 2023-12-30 01:36:11 +02:00
Juha Jeronen
d8ad41132e better type signature 2023-12-30 01:32:28 +02:00
Juha Jeronen
2558bce64e update TODO 2023-12-30 01:20:08 +02:00
Juha Jeronen
242b212405 move TODO item to TODO list 2023-12-30 01:19:22 +02:00
Juha Jeronen
6ac26399ff fix mistake in comment 2023-12-29 01:27:04 +02:00
Juha Jeronen
3e0ac73194 fix some thinkos in commented-out code 2023-12-29 01:19:00 +02:00
Juha Jeronen
371a2cdc7f compute the pixel position meshgrid just one, at startup 2023-12-29 01:15:58 +02:00
Juha Jeronen
b41419d9de simplify 2023-12-29 00:56:20 +02:00
Juha Jeronen
9a833b51da postprocessor: configurability infra 2023-12-29 00:55:43 +02:00
Juha Jeronen
f8ae43934e clarify 2023-12-29 00:17:22 +02:00
Juha Jeronen
596837b608 clarify 2023-12-29 00:17:15 +02:00
Juha Jeronen
4df45c35ce fixxor bugzzor 2023-12-29 00:17:08 +02:00
Juha Jeronen
5b852f785e refactor postprocessor into new module 2023-12-29 00:13:05 +02:00
Juha Jeronen
162d27ede6 overboard with postproc filters, vol. 2
Maybe that's all for now - next up, refactoring the postprocessor.
2023-12-28 04:13:23 +02:00
Juha Jeronen
bb6b424b13 go completely overboard with postproc filters, vol. 1 2023-12-27 03:33:08 +02:00
Juha Jeronen
f73fe3586f fix badvhs noise
Everything else in Torch takes (height, width); GaussianBlur takes (x, y).
2023-12-27 01:19:49 +02:00
Juha Jeronen
9201782f8a adjust badvhs settings 2023-12-27 01:19:40 +02:00
Juha Jeronen
b598bd668e add low-res analog video filter, update postproc TODO 2023-12-26 13:20:08 +02:00
Juha Jeronen
35f3555bd9 update TODO 2023-12-26 01:57:17 +02:00
Juha Jeronen
0710a6d669 improve postproc: add a bad VHS effect 2023-12-26 01:55:30 +02:00
Juha Jeronen
956e833e30 add some more postprocs for an even lofier look 2023-12-26 01:16:26 +02:00
Juha Jeronen
51077b51d3 comment accuracy 2023-12-26 01:16:14 +02:00
Juha Jeronen
a485584606 update TODO 2023-12-25 03:06:57 +02:00
Juha Jeronen
6558edb97f Squashed commit of the following:
- Add experimental visual postproc chain (lo-fi scifi hologram)
  - TODO: not configurable yet, it's always on until I fix this
- Improve emotion preset loading logic
  - Even if an emotion preset JSON is missing, load the emotion from _defaults.json.
- Add blunder recovery (emotion preset factory reset) options to manual poser
- Fix factory-default preset name angry -> anger
- Manual poser: return nonzero exit code on init error
- Manual poser too now auto-installs THA3 models if needed
- Move TODO list into its own file, dump everything there
- Add a README for the new revised talkinghead
2023-12-25 02:08:10 +02:00
Cohee
08a18dc506 Update docker-compose.yml 2023-12-24 22:01:59 +02:00
Cohee
ac34c02c7b Update Dockerfile 2023-12-24 22:01:14 +02:00
Cohee
499d0ee93e Merge pull request #209 from Technologicat/talkinghead-next-pr
talkinghead new features
2023-12-24 21:56:36 +02:00
Juha Jeronen
ba28f6e5ee update docstring 2023-12-22 11:47:37 +02:00
Juha Jeronen
b1489ae668 merge TODO lists into the one in app.py, remove done TODO item. 2023-12-22 11:45:51 +02:00
Juha Jeronen
ebbdc6cc88 talkinghead-next new features
- Rewrite sway animation (much more natural now)
 - History-free, rate-based formulation
 - Use all non-morph axes: head, neck, body
 - Every few seconds, randomize a new deviation from the target pose for all sway axes.
 - Assign this as the target pose, and let the pose interpolator perform the actual animation.
 - Micro-sway: add small dynamic noise (re-generated every frame) on top of sway target pose.
   - This makes the motion look more natural, especially once we are near the target pose.
- Add breathing animation
- When entering "confusion" emotion, allow blinking quickly in succession
- Add framerate limiter, currently hardcoded to send ~24 FPS
  - Reduces average GPU usage if the GPU renders faster than this.
2023-12-22 10:59:04 +02:00
Juha Jeronen
4c6f843ff9 Talkinghead performance improvements and refactoring (#207)
* talkinghead: fix and improve THA3 manual poser

* server.py: no, don't yet use fp16 for talkinghead

* talkinghead: remove wxPython dependency from live mode

* comment out unused functions

* add TODO list

* coding style

* remove unused import

* add TODO marker

* message wordings

* fix typos in variable names

* talkinghead updates

* talkinghead updates

* Empty commit

* presentation order, sectioning

* fix the inMotion flag update

* mark a TODO

* refactor

* remove done TODO items

* mark a TODO

* comment wording

* pause animation while loading a new image

* parser doesn't belong here, not a command-line app anymore

* message wording

* use finally

* remove superfluous "global" declarations

* lots of cleanup

* remove silly sys.path entry

* improve docstring

* oops

* app.py now only serves the live mode for the plugin

* talkinghead live mode: remove ifacialmocap stuff (unused)

* improve comment

* list walking is so 1990s

* use double quotes consistently

* remove now-unused ifacialmocap-related files from the repo

* remove done TODO item

* improve docstring

* update comment

* remove now-unused function

* update comment

* improve docstring

* add TODO marker

* oops, typo

* add --talkinghead-model command-line option to server.py

Default is 'auto': float16 on GPU, float32 on CPU.

* talkinghead: auto-install THA3 models if needed

* remove tha3/models from git repo (have autodownload now)

* Add hf-hub as explicit dependency

* Add THA models to gitignore

---------

Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>
2023-12-21 23:48:25 +02:00
Cohee
47a5489142 Use pipeline for text summarization 2023-12-20 01:57:38 +02:00
Cohee
423f51e3f8 Use pipeline for image captioning 2023-12-20 01:28:53 +02:00
Juha Jeronen
7ca92eaeac talkinghead: fix and improve THA3 manual poser (#204)
* talkinghead: fix and improve THA3 manual poser

* server.py: no, don't yet use fp16 for talkinghead
2023-12-20 01:21:24 +02:00
Cohee
c5d1773f6d talkinghead inital.png path broken on Linux #198 2023-12-14 00:44:09 +02:00
Cohee
98535e6fa3 Add links to search results 2023-12-11 23:52:25 +02:00