X Tutup
Skip to content

[Bug]: Severe Direct Memory Leak via Unbounded ThreadLocal Queue in ByteBufferProxy #284

@QiuYucheng2003

Description

@QiuYucheng2003

Bug Description

There is a critical off-heap memory leak in ByteBufferProxy.java. The AbstractByteBufferProxy class utilizes a ThreadLocal to cache ByteBuffer instances in an ArrayDeque. However, this queue is entirely unbounded and caches Direct ByteBuffers, leading to massive native memory accumulation and physical memory fragmentation.

Root Cause

In ByteBufferProxy.java, the cache is defined as:

private static final ThreadLocal<ArrayDeque<ByteBuffer>> BUFFERS =
    withInitial(() -> new ArrayDeque<>(16));

When allocate() is called and the queue is empty, it allocates a direct buffer via ByteBuffer.allocateDirect(0).
When deallocate(ByteBuffer buff) is invoked, it blindly returns the buffer to the queue:

queue.offer(buff);

There is no maximum capacity limit on the ArrayDeque, and BUFFERS.remove() is never invoked.

Impact
In environments utilizing thread pools (e.g., web servers, async task workers), threads are long-lived. Because the ArrayDeque is unbounded and never explicitly cleared, each thread retains all the Direct ByteBuffers it has ever processed at peak load.
This causes:

1. Unbounded Native Memory Growth: Direct buffers bypass the JVM heap. Their accumulation quickly exhausts the OS physical memory.

2. Memory Fragmentation & OOM Killer: The uncontrolled accumulation of direct byte blocks leads to severe physical memory fragmentation, eventually triggering the OS-level OOM Killer and crashing the JVM.

Proposed Fix
1. Bound the Queue: Enforce a maximum capacity (e.g., MAX_CACHED_BUFFERS) in the deallocate method. If the ArrayDeque reaches this limit, allow the buffer to be naturally garbage-collected instead of re-queueing it.

2. Lifecycle Management: Provide a mechanism to call BUFFERS.remove() to clean up the ThreadLocalMap when a thread's lifecycle ends, or replace the ThreadLocal approach with a centralized, globally bounded object pool (e.g., ConcurrentLinkedQueue with a strict size limit).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      X Tutup