Skip to content

Do not stop the zstream loop for an interrupt - #151

Open
bruno-costanzo wants to merge 1 commit into
ruby:masterfrom
bruno-costanzo:fix-deflate-finish-after-interrupt
Open

bruno-costanzo wants to merge 1 commit into
ruby:masterfrom
bruno-costanzo:fix-deflate-finish-after-interrupt

Conversation

@bruno-costanzo

Copy link
Copy Markdown

#121 fixed the Zlib::BufError from #57, but its retry condition lets Zlib::Deflate#finish return before the stream ends. This is the iteration @ioquatix mentioned in #74: the same idea as #74 (@ianks), taken one step further.

Problem

In zlib 3.2.4 (and in Ruby 3.4.9 and later, which ship #121 as zlib "3.2.3"), when a thread is interrupted during a deflate, the output can be truncated:

  • Zlib.deflate returns truncated output and raises nothing.
  • Zlib::Deflate#finish returns truncated output. A later close raises Zlib::DataError, because deflateEnd sees an unfinished stream.
  • Zlib::GzipWriter#close writes truncated output, and then usually raises Zlib::DataError.

An interrupt is Thread#wakeup, or a signal such as SIGCHLD from a backtick in another thread. This partly brings back Bug #10961 ("Zlib corrupts data when receive signal"), which #22 fixed with the retry.

Reproduction (ruby 4.0.6, arm64-darwin, libz 1.3.1):

require "zlib"
require "securerandom"

content = SecureRandom.random_bytes(256 * 1024)
outputs = []

thr = Thread.new do
  100.times do
    z = Zlib::Deflate.new
    outputs << (z.deflate(content) << z.finish)
    z.close
  end
end

while thr.alive?
  begin
    thr.wakeup
  rescue ThreadError
  end
  Thread.pass
end
thr.join

p outputs.count { |output| Zlib::Inflate.inflate(output) == content }
  • zlib 3.2.3 gem: Zlib::BufError from deflate (Thread#wakeup leads to Zlib::BufError #57), 20 of 20 runs.
  • zlib 3.2.4 and master: Zlib::DataError from close, 20 of 20 runs. Same on Ruby 3.4.11's bundled zlib.
  • This branch: prints 100, 20 of 20 runs.

Cause

Since #88, each deflate() and inflate() call runs in its own rb_nogvl, and rb_nogvl handles pending interrupts when the call returns. zstream_run_func still stops its loop when the unblock function sets args->interrupt, and zstream_run_try then guesses whether to resume:

Fix

The loop no longer stops for an interrupt, and the resume block is removed. Interrupts still reach Ruby after each call through rb_nogvl. The unblock function still sets args->interrupt, but nothing reads it now. I kept the field to make the diff small and can remove it if you prefer.

Checked on this branch:

  • Thread#raise, Thread#kill, Timeout.timeout and a raising trap still stop deflate, inflate, Zlib.gzip and GzipWriter within about 60 ms.
  • With a Thread#wakeup storm or about 1,700 SIGUSR1 with a non-raising trap, every API gives correct output, with no slowdown: Deflate with each flush mode, Zlib.deflate, Zlib.gzip, GzipWriter, Zlib.inflate, chunked Inflate, Zlib.gunzip, and GzipReader#read and #readpartial.
  • The SIGINT/fork reproduction from Thread#wakeup leads to Zlib::BufError #57 passes 30 of 30 runs.

Test

TestZlibDeflate#test_thread_wakeup_interrupt deflates 256 KiB of random data 100 times in a thread that is woken up in a loop, then inflates every output. It runs in less than 0.5 s and is pended on TruffleRuby, like the existing interrupt tests.

  • Without the fix it fails on every run: Zlib::DataError from close. I checked this on macOS and on Linux (ruby 3.4.11, aarch64).
  • With the fix, bundle exec rake compile test passes on Ruby 4.0.6, 3.4.8, 3.3.10 and 3.2.9 (99 tests, 639 assertions on macOS). The existing test_thread_wakeup_interrupt tests in TestZlib and TestZlibGzipReader still pass.

Ruby's ruby_3_4 branch has #121, so it needs this change too.

Each deflate() and inflate() call runs in its own rb_nogvl since ruby#88, and
rb_nogvl handles pending interrupts when the call returns. So the loop
does not need to stop for an interrupt and resume.

The resume check from ruby#121 skipped the resume when avail_in == 0. Under
Z_FINISH, deflate can still have pending output then, so
Zlib::Deflate#finish could return before Z_STREAM_END and drop the end
of the stream.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The C-level change affects Ruby VM interrupt handling across all deflate and inflate paths and warrants final human review.

0 open findings

What changed in this PR

Fixes truncated deflate output when non-raising interrupts occur during stream processing.

Changes:

  • Continues zstream processing after interrupts until normal termination.
  • Adds regression coverage for repeated Thread#wakeup calls.
File Description
ext/​zlib/​zlib.c Removes interrupt-based loop termination and retry logic.
test/​zlib/​test_zlib.rb Verifies interrupted deflate streams remain complete.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants