zstd: add 100 MB decompression size limit#11854
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds a 100 MB decompression-size cap ( ChangesZSTD Decompression Size Limit
Sequence Diagram(s)sequenceDiagram
participant Caller
participant flb_zstd as flb_zstd.c
participant ZSTD
Caller->>flb_zstd: provide compressed data
flb_zstd->>flb_zstd: check FLB_ZSTD_DECOMPRESS_MAX (frame or next growth)
alt size within limit
flb_zstd->>ZSTD: perform decompression
ZSTD-->>flb_zstd: decompressed bytes
flb_zstd-->>Caller: return data
else exceeds limit
flb_zstd-->>Caller: log error and return failure
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 820a279780
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| out_size *= 2; | ||
| if (out_size > FLB_ZSTD_DECOMPRESS_MAX) { | ||
| flb_error("[zstd] maximum decompression size reached (~100 MB)"); |
There was a problem hiding this comment.
The new limit check rejects valid payloads when ZSTD_CONTENTSIZE_UNKNOWN is used and the decompressed size is between 64 MiB and 100 MiB. In this path, out_size doubles from 64 MiB to 128 MiB, then the > FLB_ZSTD_DECOMPRESS_MAX check fails before attempting to decompress further, so frames that should be accepted under the 100 MB cap now fail. This introduces a functional regression for legitimate unknown-size frames and should instead clamp growth to the max (or otherwise permit expansion up to exactly the limit).
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/flb_zstd.c`:
- Around line 109-114: The decompression path incorrectly aborts when the next
doubling would exceed FLB_ZSTD_DECOMPRESS_MAX, which prevents growing buffers up
to the declared max; update the resizing logic where out_size is doubled to
compute new_size = out_size * 2 and if new_size > FLB_ZSTD_DECOMPRESS_MAX set
new_size = FLB_ZSTD_DECOMPRESS_MAX, then realloc buf to new_size (instead of
immediately erroring and freeing buf/ZSTD_freeDCtx/return -1); adjust the check
that currently uses out_size > FLB_ZSTD_DECOMPRESS_MAX to use the computed
new_size so payloads between the last power-of-two and the max are accepted.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
|
@TristanInSec would you please sign off the commits ? (DCO error / git commit -s ...) also note the auto-reviews by the CI agent: "Verify each finding against current code. Fix only still-valid issues, skip the Inline comments:
|
The gzip decompressor limits output to 100 MB but the zstd decompressor has no such cap. A small zstd-compressed payload with a high compression ratio can expand to gigabytes, exhausting memory and causing the process to be OOM-killed. Add FLB_ZSTD_DECOMPRESS_MAX (100 MB) matching the gzip limit, checked in both the known-size path (ZSTD_getFrameContentSize) and the streaming path (zstd_uncompress_unknown_size). Signed-off-by: Tristan <tristan@talencesecurity.com>
TristanInSec
force-pushed
the
fix/zstd-decompression-limit
branch
from
820a279 to
c7ec32a
Compare
When the streaming decompression buffer doubles past the 100 MB limit, cap the allocation at FLB_ZSTD_DECOMPRESS_MAX instead of immediately aborting. This allows payloads between the last power-of-two (64 MB) and the 100 MB limit to decompress successfully. Only abort when the buffer is already at the maximum and still needs more space. Signed-off-by: Tristan <tristan@talencesecurity.com>
3 participants
The gzip decompressor limits output to 100 MB but the zstd
decompressor has no such cap. A small zstd-compressed payload with
a high compression ratio can expand to gigabytes, exhausting memory
and causing the process to be OOM-killed.
Add FLB_ZSTD_DECOMPRESS_MAX (100 MB) matching the gzip limit,
checked in both the known-size path (ZSTD_getFrameContentSize) and
the streaming path (zstd_uncompress_unknown_size).
Summary by CodeRabbit