[GH-ISSUE #2087] Unauthenticated pickle deserialization RCE on the balance_serve scheduler ZMQ socket (bound on all interfaces) #1269

Closed
opened 2026-10-29 14:50:44 +00:00 by bsakaguchi · 0 comments
Owner

Originally created by @geo-chen on GitHub (Jul 13, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/2087

Reminder

  • I have read the above rules and searched the existing issues.

System Info

version: Commit 89d30a3

reported on 12 June 2026: https://github.com/kvcache-ai/ktransformers/security/advisories/GHSA-83vp-v6wg-x93x

Reproduction

Summary

The ktransformers balance_serve scheduler (the default high-performance backend) binds a ZeroMQ ROUTER socket on all network interfaces with no authentication and calls pickle.loads on every received message. Any host that can reach the scheduler port can send a crafted pickle stream whose __reduce__ executes a shell command, achieving remote code execution as the server process. Confirmed end to end against the socket: an os.system gadget executed via the pickle.loads on the wildcard-bound socket.

Details

archive/ktransformers/server/balance_serve/sched_rpc.py:

self.frontend.bind(f"tcp://*:{main_args.sched_port}")   # ~line 43: tcp://* = ALL interfaces, no auth
...
def worker_routine(...):
    message = worker.recv()                             # ~line 69
    data = pickle.loads(message)                        # ~line 70: untrusted bytes -> pickle.loads
    ... data.get('method') ...

There is no authentication, CURVE/TLS, or allowlist before the pickle.loads. The scheduler is launched as a subprocess in the default backend (archive/ktransformers/server/backend/interfaces/balance_serve.py ~line 495, subprocess.Popen([... sched_rpc.py ...])). tcp://* binds 0.0.0.0, so the socket is reachable from the network. The sched_port is auto-assigned from a free local port, so an attacker must discover it (a port scan), but the bind itself is on all interfaces and unauthenticated.

PoC

The scheduler binds tcp://*:{sched_port} (archive/.../balance_serve/sched_rpc.py:43, ss shows 0.0.0.0:PORT) and worker_routine does pickle.loads(message) (:70) with no auth; it is launched as a subprocess by the default balance_serve backend (backend/interfaces/balance_serve.py:495). An unauthenticated client sends a pickle gadget:

import zmq, pickle, os
class Exploit:
    def __reduce__(self):
        return (os.system, ("touch /tmp/ktrans_pwned",))
s = zmq.Context().socket(zmq.REQ)
s.connect("tcp://<scheduler-host>:<sched_port>")
s.send(pickle.dumps(Exploit()))

Validated against the sched_rpc.py: the SchedulerServer.__init__ ran the self.frontend.bind("tcp://*:<port>") (ss confirmed 0.0.0.0:<port> listening), and the worker_routine's pickle.loads(message) executed the gadget (the marker file contained the output of id). Only import-time CUDA dependencies were stubbed so the module could load; the bind and pickle.loads lines are the genuine, unmodified code. The server's subsequent error ('int' object has no attribute 'get') confirms pickle.loads returned the gadget's result before any dispatch.

Impact

An attacker who can reach the scheduler port (which is bound on all interfaces by default) executes arbitrary code as the ktransformers server process, with no authentication. On a deployment exposed to a LAN or the internet this is remote code execution; the only barrier is discovering the auto-assigned port.

Remediation

Bind the scheduler socket to loopback (tcp://127.0.0.1:<port>) rather than tcp://*, and do not deserialize untrusted network input with pickle: replace pickle.loads with a safe, schema-validated format (JSON, protobuf, msgpack with a restricted type set) for the inter-process RPC, or add ZeroMQ CURVE authentication and an allowlist. Inter-process scheduler RPC should not be exposed on the network at all.

Others

No response

Originally created by @geo-chen on GitHub (Jul 13, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/2087 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info version: Commit 89d30a3 reported on 12 June 2026: https://github.com/kvcache-ai/ktransformers/security/advisories/GHSA-83vp-v6wg-x93x ### Reproduction ### Summary The ktransformers `balance_serve` scheduler (the default high-performance backend) binds a ZeroMQ ROUTER socket on all network interfaces with no authentication and calls `pickle.loads` on every received message. Any host that can reach the scheduler port can send a crafted pickle stream whose `__reduce__` executes a shell command, achieving remote code execution as the server process. Confirmed end to end against the socket: an `os.system` gadget executed via the `pickle.loads` on the wildcard-bound socket. ### Details `archive/ktransformers/server/balance_serve/sched_rpc.py`: ```python self.frontend.bind(f"tcp://*:{main_args.sched_port}") # ~line 43: tcp://* = ALL interfaces, no auth ... def worker_routine(...): message = worker.recv() # ~line 69 data = pickle.loads(message) # ~line 70: untrusted bytes -> pickle.loads ... data.get('method') ... ``` There is no authentication, CURVE/TLS, or allowlist before the `pickle.loads`. The scheduler is launched as a subprocess in the default backend (`archive/ktransformers/server/backend/interfaces/balance_serve.py` ~line 495, `subprocess.Popen([... sched_rpc.py ...])`). `tcp://*` binds `0.0.0.0`, so the socket is reachable from the network. The `sched_port` is auto-assigned from a free local port, so an attacker must discover it (a port scan), but the bind itself is on all interfaces and unauthenticated. ### PoC The scheduler binds `tcp://*:{sched_port}` (`archive/.../balance_serve/sched_rpc.py:43`, `ss` shows `0.0.0.0:PORT`) and `worker_routine` does `pickle.loads(message)` (`:70`) with no auth; it is launched as a subprocess by the default `balance_serve` backend (`backend/interfaces/balance_serve.py:495`). An unauthenticated client sends a pickle gadget: ```python import zmq, pickle, os class Exploit: def __reduce__(self): return (os.system, ("touch /tmp/ktrans_pwned",)) s = zmq.Context().socket(zmq.REQ) s.connect("tcp://<scheduler-host>:<sched_port>") s.send(pickle.dumps(Exploit())) ``` Validated against the `sched_rpc.py`: the `SchedulerServer.__init__` ran the `self.frontend.bind("tcp://*:<port>")` (`ss` confirmed `0.0.0.0:<port>` listening), and the `worker_routine`'s `pickle.loads(message)` executed the gadget (the marker file contained the output of `id`). Only import-time CUDA dependencies were stubbed so the module could load; the bind and `pickle.loads` lines are the genuine, unmodified code. The server's subsequent error (`'int' object has no attribute 'get'`) confirms `pickle.loads` returned the gadget's result before any dispatch. ### Impact An attacker who can reach the scheduler port (which is bound on all interfaces by default) executes arbitrary code as the ktransformers server process, with no authentication. On a deployment exposed to a LAN or the internet this is remote code execution; the only barrier is discovering the auto-assigned port. ### Remediation Bind the scheduler socket to loopback (`tcp://127.0.0.1:<port>`) rather than `tcp://*`, and do not deserialize untrusted network input with pickle: replace `pickle.loads` with a safe, schema-validated format (JSON, protobuf, msgpack with a restricted type set) for the inter-process RPC, or add ZeroMQ CURVE authentication and an allowlist. Inter-process scheduler RPC should not be exposed on the network at all. ### Others _No response_
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1269
No description provided.