Farmadupe d3e86a5431 Free raw multimedia data from server_tokens after encoding, as it will never be read again (#2029) ...

Data server_tokens.map_idx_to_media.tokens_image.batch_f32 is read exactly once, 
by mtmd_encode, however it was retained as long as the input image was present
in the sequence. Add a manual free function to clear out this data after encoding.

Solves:
* Memory wasted in struct server_tokens
* The same wasted memory in the ram cache 
* Long copy durations cloning this data to/from ram cache
* Accounting failures in ram cache (`batch_f32` can be larger than a sequence's entire KV)
* The above accounting failures leading to terminal memory leaks in pathological cases
* Remove JSON serialization for `batch_32` which was unused, and had no foreseeable usecase

2026-06-25 10:18:32 +02:00

66 KiB

Raw Permalink Blame History

View Raw