[GH-ISSUE #3696] [Feature] Java serialization reverse engineering: Object Serialization streams, class descriptors, handles and object graph reconstruction #1409

New issue

Open

opened 2026-10-16 21:05:12 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:05:12 +00:00

Owner

Originally created by @matbrgz on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3696

Problem Statement & Context

Java Object Serialization (java.io.Serializable, java.io.Externalizable) is Java's foundational binary object serialization protocol. Serialized stream payloads appear across enterprise middleware, remote method invocation (RMI), messaging queues (JMS), cached HTTP sessions, encrypted database blobs, and desktop application state.

In security analysis, deserialization vulnerabilities (gadget chains leading to remote code execution) represent a critical attack vector. However, analyzing a serialized Java payload using Java's native ObjectInputStream.readObject() is unsafe and executes untrusted code. Reverse engineers require a purely static, non-executing parser that can dissect the binary serialization protocol:

Magic Header & Protocol Version: Magic 0xACED (STREAM_MAGIC) and version 0x0005 (STREAM_VERSION).
Type Codes & Descriptors: TC_OBJECT, TC_CLASSDESC, TC_REFERENCE (handles), TC_STRING, TC_ARRAY, TC_ENUM, TC_BLOCKDATA, TC_ENDBLOCKDATA.
Class Hierarchy & SUID: Serial Version Unique Identifiers (serialVersionUID), class flags (SC_SERIALIZABLE, SC_EXTERNALIZABLE, SC_WRITE_METHOD), and superclass chains.
Handle Table & Cyclic References: 0-based handle indices resolving cyclic references and deduplicated instances throughout the stream.
Custom Serialization Blocks: Identifying writeObject / readObject custom block data or Externalizable streams.

REA needs a safe, static Java Object Serialization engine to parse payloads, reconstruct the in-memory object graph, and audit gadget chains.

Proposed Architecture

flowchart TD
    A["Serialized Java Stream (0xACED 0005 / .ser file)"] --> B["Static Java Serialization Parser"]
    B --> C["Protocol Stream Deserializer"]
    B --> D["Handle Reference Resolver"]
    B --> E["Class Descriptor & Gadget Chain Auditor"]
    
    C --> C1["Magic Verification (0xACED, Version 5)"]
    C --> C2["Type Tag Tokenizer (TC_OBJECT, TC_STRING, TC_ARRAY)"]
    C --> C3["Primitive & Array Data Parsing"]
    
    D --> D1["Handle Table Tracking (0x7E0000...)"]
    D --> D2["Cyclic Graph Reconstruction"]
    
    E --> E1["Class Hierarchy & serialVersionUID Matching"]
    E --> E2["Custom Block Data (TC_BLOCKDATA) Extraction"]
    E --> E3["Known Deserialization Gadget Fingerprinting (e.g. Commons Collections)"]
    
    C & D & E --> F["Deserialized Object Graph & Security Audit Report"]

Technical Capabilities & Modules

Protocol Header & Stream Slicing:
- Verify STREAM_MAGIC (0xACED) and STREAM_VERSION (0x0005).
- Robustly slice payloads embedded within network captures, HTTP requests, or database records.
Recursive Type Tag Tokenizer:
- Parse all standard Java Serialization tokens:
  - TC_OBJECT (0x73): Object instances.
  - TC_CLASSDESC (0x72) / TC_PROXYCLASSDESC (0x7D): Class metadata, field names, field type signatures.
  - TC_STRING (0x74) / TC_LONGSTRING (0x7C): Modified UTF-8 string pools.
  - TC_ARRAY (0x75): Arrays of primitive types or objects.
  - TC_ENUM (0x7E): Enum constant values.
  - TC_BLOCKDATA (0x77) / TC_BLOCKDATALONG (0x7A): Opaque bytes written by custom writeObject routines.
Handle Table & Reference Tracking:
- Maintain a sequential handle index beginning at base handle 0x7E0000.
- Resolve TC_REFERENCE (0x71) tags back to previously registered class descriptors, strings, or object instances to reconstruct complex cyclic object graphs.
Class Descriptor & Hierarchy Mapping:
- Extract fully qualified class names, serialVersionUID values, and inheritance parent chains.
- Audit declared fields against streamed field values.
Security & Gadget Chain Detection:
- Statically identify classes frequently leveraged in deserialization exploit payloads (e.g., InvokerTransformer, TemplatesImpl, BadAttributeValueExpException, AnnotationInvocationHandler).
- Flag dynamic proxy descriptors (TC_PROXYCLASSDESC) and their implemented interfaces.

Acceptance Criteria

100% static, safe parsing of Java Object Serialization streams without invoking the JVM or ObjectInputStream.
Accurate reconstruction of the full object graph, including nested instances and arrays.
Robust resolution of handle references and cyclic object dependencies.
Extraction of class descriptors, serialVersionUIDs, and custom block data.
Export of the reconstructed object graph as human-readable JSON or interactive property tree.

Complements #3617 (Java Applets & JNLP reverse engineering)
Complements #3645 (Java Card reverse engineering)
Complements #3652 (PyTorch checkpoint reverse engineering: pickle opcode graph)

Originally created by @matbrgz on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3696 [![Type: Feature](https://img.shields.io/badge/type-feature-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Domain: Game & Engine RE](https://img.shields.io/badge/domain-game_&_engine_RE-green.svg?style=flat-square)](https://github.com/morluto/rea) [![Status: Proposed](https://img.shields.io/badge/status-proposed-blue.svg?style=flat-square)](https://github.com/morluto/rea) [![Ecosystem: REA](https://img.shields.io/badge/ecosystem-morluto%2Frea-8A2BE2.svg?style=flat-square)](https://github.com/morluto/rea) ### Problem Statement & Context Java Object Serialization (`java.io.Serializable`, `java.io.Externalizable`) is Java's foundational binary object serialization protocol. Serialized stream payloads appear across enterprise middleware, remote method invocation (RMI), messaging queues (JMS), cached HTTP sessions, encrypted database blobs, and desktop application state. In security analysis, deserialization vulnerabilities (gadget chains leading to remote code execution) represent a critical attack vector. However, analyzing a serialized Java payload using Java's native `ObjectInputStream.readObject()` is unsafe and executes untrusted code. Reverse engineers require a purely static, non-executing parser that can dissect the binary serialization protocol: 1. **Magic Header & Protocol Version**: Magic `0xACED` (`STREAM_MAGIC`) and version `0x0005` (`STREAM_VERSION`). 2. **Type Codes & Descriptors**: `TC_OBJECT`, `TC_CLASSDESC`, `TC_REFERENCE` (handles), `TC_STRING`, `TC_ARRAY`, `TC_ENUM`, `TC_BLOCKDATA`, `TC_ENDBLOCKDATA`. 3. **Class Hierarchy & SUID**: Serial Version Unique Identifiers (`serialVersionUID`), class flags (`SC_SERIALIZABLE`, `SC_EXTERNALIZABLE`, `SC_WRITE_METHOD`), and superclass chains. 4. **Handle Table & Cyclic References**: 0-based handle indices resolving cyclic references and deduplicated instances throughout the stream. 5. **Custom Serialization Blocks**: Identifying `writeObject` / `readObject` custom block data or `Externalizable` streams. REA needs a safe, static Java Object Serialization engine to parse payloads, reconstruct the in-memory object graph, and audit gadget chains. ### Proposed Architecture ```mermaid flowchart TD A["Serialized Java Stream (0xACED 0005 / .ser file)"] --> B["Static Java Serialization Parser"] B --> C["Protocol Stream Deserializer"] B --> D["Handle Reference Resolver"] B --> E["Class Descriptor & Gadget Chain Auditor"] C --> C1["Magic Verification (0xACED, Version 5)"] C --> C2["Type Tag Tokenizer (TC_OBJECT, TC_STRING, TC_ARRAY)"] C --> C3["Primitive & Array Data Parsing"] D --> D1["Handle Table Tracking (0x7E0000...)"] D --> D2["Cyclic Graph Reconstruction"] E --> E1["Class Hierarchy & serialVersionUID Matching"] E --> E2["Custom Block Data (TC_BLOCKDATA) Extraction"] E --> E3["Known Deserialization Gadget Fingerprinting (e.g. Commons Collections)"] C & D & E --> F["Deserialized Object Graph & Security Audit Report"] ``` ### Technical Capabilities & Modules 1. **Protocol Header & Stream Slicing:** - Verify `STREAM_MAGIC` (`0xACED`) and `STREAM_VERSION` (`0x0005`). - Robustly slice payloads embedded within network captures, HTTP requests, or database records. 2. **Recursive Type Tag Tokenizer:** - Parse all standard Java Serialization tokens: - `TC_OBJECT` (0x73): Object instances. - `TC_CLASSDESC` (0x72) / `TC_PROXYCLASSDESC` (0x7D): Class metadata, field names, field type signatures. - `TC_STRING` (0x74) / `TC_LONGSTRING` (0x7C): Modified UTF-8 string pools. - `TC_ARRAY` (0x75): Arrays of primitive types or objects. - `TC_ENUM` (0x7E): Enum constant values. - `TC_BLOCKDATA` (0x77) / `TC_BLOCKDATALONG` (0x7A): Opaque bytes written by custom `writeObject` routines. 3. **Handle Table & Reference Tracking:** - Maintain a sequential handle index beginning at base handle `0x7E0000`. - Resolve `TC_REFERENCE` (0x71) tags back to previously registered class descriptors, strings, or object instances to reconstruct complex cyclic object graphs. 4. **Class Descriptor & Hierarchy Mapping:** - Extract fully qualified class names, `serialVersionUID` values, and inheritance parent chains. - Audit declared fields against streamed field values. 5. **Security & Gadget Chain Detection:** - Statically identify classes frequently leveraged in deserialization exploit payloads (e.g., `InvokerTransformer`, `TemplatesImpl`, `BadAttributeValueExpException`, `AnnotationInvocationHandler`). - Flag dynamic proxy descriptors (`TC_PROXYCLASSDESC`) and their implemented interfaces. ### Acceptance Criteria - [ ] 100% static, safe parsing of Java Object Serialization streams without invoking the JVM or `ObjectInputStream`. - [ ] Accurate reconstruction of the full object graph, including nested instances and arrays. - [ ] Robust resolution of handle references and cyclic object dependencies. - [ ] Extraction of class descriptors, `serialVersionUID`s, and custom block data. - [ ] Export of the reconstructed object graph as human-readable JSON or interactive property tree. ### Related Issues & Dependencies - Complements #3617 (Java Applets & JNLP reverse engineering) - Complements #3645 (Java Card reverse engineering) - Complements #3652 (PyTorch checkpoint reverse engineering: pickle opcode graph)

bsakaguchi referenced this issue

2026-10-17 21:11:12 +00:00

[PR #1415] [MERGED] feat(clients): register Qwen Code during setup #2869

bsakaguchi referenced this issue

2026-10-31 00:55:29 +00:00

[PR #1804] [CLOSED] feat(setup): register Kimi Code client with compact schema profile #4826

bsakaguchi referenced this issue

2026-10-31 00:55:29 +00:00

[PR #1806] [CLOSED] feat(setup): support Kimi Code client integrations #4828