Bump apache-beam from 2.75.0 to 2.77.0

[dependabot]

Bumps apache-beam from 2.75.0 to 2.77.0.

Release notes

Sourced from apache-beam's releases.

Beam 2.77.0 release

We are happy to present the new 2.77.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.

For more information on changes in 2.77.0, check out the detailed release notes.

Highlights

I/Os

• Added schema_update_options to WriteToBigQuery file loads, allowing BigQuery load jobs to add nullable fields or relax required fields when appending data (Python) (#21141).
• BigQueryIO now supports reading BigQuery Lakehouse runtime catalog (BigLake metastore) Iceberg tables with the Storage Read API, using 4-part project.catalog.namespace.table identifiers (or a TableReference with a composite catalog.namespace dataset id). Previously such references were silently mis-parsed (Java) (#39597) .
• SolaceIO now supports reading and writing binary and text content data payload (Java) (#39875).
• ClickHouseIO: support writing Decimal(P, S) / Decimal32/64/128/256 columns (Java) (#39840).
• SolaceIO now supports reading and writing user properties (message metadata) (Java) (#40099).
• [IcebergIO] AddFiles (IcebergAddFiles in YAML) can evolve the table schema before registering files, with schema_evolution_options, required_columns, incompatible_schema_handling and unverifiable_file_handling (Java/YAML, batch only) (#40144).

New Features / Improvements

• (Java/Python) Watch can bound its deduplication state by event time, retiring an output key once the greatest emitted timestamp has moved more than the allowed lateness past it. Java adds Watch.growthOf(...).withTimestampCursor(). Python adds allowed_lateness for the existing timestamp_cursor option (#18459).
• (Java) Spark Structured Streaming runner: stateful ParDo with state, timers, @RequiresTimeSortedInput and tagged outputs is now supported in batch mode (#39779).
• (Python) Added support for Vertex AI Model Monitoring V2 in RunInference (#39738).
• [Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new sourceStaticSplitThresholdMb pipeline option. The default of 0 keeps the existing lazy pull-based assignment (#39873).
• Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution (#39866).
• (Python) Added Sample.Any, the Python equivalent of Java's Sample.any, which returns up to n arbitrary elements from a PCollection (#18552).

Breaking Changes

• Portable Java SDK now encodes SchemaCoders in a portable way (#34672).
  • Original custom Java coder encoding can still be obtained using StreamingOptions.setUpdateCompatibilityVersion("2.76") (#34672).
  • Fixes (#36496), (#30276), (#29245).
• (Python) TensorRTEngineHandlerNumPy now requires TensorRT 10 or later. TensorRT 8.x is no longer supported, since TensorRT 10 removed the engine binding API the handler was written against (#36306).
  • Engines serialized by TensorRT 8.x must be rebuilt, as an engine can only be deserialized by the major version that built it.
  • TensorRT 10 and later require a GPU with compute capability 7.5 or higher, which excludes NVIDIA Pascal and Volta GPUs.
  • If dropping TensorRT 8.x support is a hard blocker for you, please comment on (#36306).

Bugfixes

• (Java) Fixed the Spark runner firing processing-time timers in reverse timestamp order (#39824).
• (Java) Fixed the Spark runner dropping the stored watermark of a streaming source with no update in a batch (#39822).
• (Python) Fixed incorrect profiler options handling on portable runners (#39613).
• (Java) KafkaIO dynamic reads no longer require the obsolete beam_fn_api experiment (#29998).
• (Prism) Self-checkpointing splittable DoFns now resume after their requested delay instead of immediately, so polling SDFs no longer busy-spin (#39848).
• (Java) MongoDbIO read splitting now preserves non-ObjectId _id types (e.g. string ids) instead of failing to parse the generated range filters (#39900).
• (Go) Fixed GCS glob matching silently dropping objects when the glob pattern contains multi-byte characters (#39969).
• (Python) Fixed TensorRTEngineHandlerNumPy failing with CUDA_ERROR_INVALID_VALUE on models with a single-element input or output tensor (#36306).
• (Python) Fixed PickleCoder/_MemoizingPickleCoder.as_deterministic_coder() raising TypeError instead of returning a working deterministic coder (#28558).

... (truncated)

Changelog

Sourced from apache-beam's changelog.

[2.77.0] - 2026-10-07

I/Os

• Added schema_update_options to WriteToBigQuery file loads, allowing BigQuery load jobs to add nullable fields or relax required fields when appending data (Python) (#21141).
• BigQueryIO now supports reading BigQuery Lakehouse runtime catalog (BigLake metastore) Iceberg tables with the Storage Read API, using 4-part project.catalog.namespace.table identifiers (or a TableReference with a composite catalog.namespace dataset id). Previously such references were silently mis-parsed (Java) (#39597) .
• SolaceIO now supports reading and writing binary and text content data payload (Java) (#39875).
• ClickHouseIO: support writing Decimal(P, S) / Decimal32/64/128/256 columns (Java) (#39840).
• SolaceIO now supports reading and writing user properties (message metadata) (Java) (#40099).
• [IcebergIO] AddFiles (IcebergAddFiles in YAML) can evolve the table schema before registering files, with schema_evolution_options, required_columns, incompatible_schema_handling and unverifiable_file_handling (Java/YAML, batch only) (#40144).
• [IcebergIO] Added batch and streaming CDC writes that applies INSERT/UPDATE_BEFORE/UPDATE_AFTER/DELETE changes to Iceberg V2+ tables by primary key. Invoke with IcebergIO.writeCdcRows (Java) or by setting mode: merge-on-read on the Managed ICEBERG write (Java, Python, YAML) (#39979).
• [IcebergIO] Added an optional side-input table cache for writes to significantly reduce catalog and table requests for large pipelines. A single worker polls the table and broadcasts it to other workers in the pipeline. Enable with IcebergIO.writeRows(...).withSideInputTableCache() (Java) or by setting use_side_input_table_cache: true on the Managed ICEBERG write (Java, Python) (#39723).

New Features / Improvements

• (Java/Python) Watch can bound its deduplication state by event time, retiring an output key once the greatest emitted timestamp has moved more than the allowed lateness past it. Java adds Watch.growthOf(...).withTimestampCursor(). Python adds allowed_lateness for the existing timestamp_cursor option (#18459).
• (Java) Spark Structured Streaming runner: stateful ParDo with state, timers, @RequiresTimeSortedInput and tagged outputs is now supported in batch mode (#39779).
• (Python) Added support for Vertex AI Model Monitoring V2 in RunInference (#39738).
• [Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new sourceStaticSplitThresholdMb pipeline option. The default of 0 keeps the existing lazy pull-based assignment (#39873).
• Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution (#39866).
• (Python) Added Sample.Any, the Python equivalent of Java's Sample.any, which returns up to n arbitrary elements from a PCollection (#18552).

Breaking Changes

• Portable Java SDK now encodes SchemaCoders in a portable way (#34672).
  • Original custom Java coder encoding can still be obtained using StreamingOptions.setUpdateCompatibilityVersion("2.76") (#34672).
  • Fixes (#36496), (#30276), (#29245).
• (Python) TensorRTEngineHandlerNumPy now requires TensorRT 10 or later. TensorRT 8.x is no longer supported, since TensorRT 10 removed the engine binding API the handler was written against (#36306).
  • Engines serialized by TensorRT 8.x must be rebuilt, as an engine can only be deserialized by the major version that built it.
  • TensorRT 10 and later require a GPU with compute capability 7.5 or higher, which excludes NVIDIA Pascal and Volta GPUs.
  • If dropping TensorRT 8.x support is a hard blocker for you, please comment on (#36306).

Bugfixes

• (Java) Fixed the Spark runner firing processing-time timers in reverse timestamp order (#39824).
• (Java) Fixed the Spark runner dropping the stored watermark of a streaming source with no update in a batch (#39822).
• (Python) Fixed incorrect profiler options handling on portable runners (#39613).
• (Python) Fixed AsyncWrapper mixing results between keys or windows whose values or custom identifiers match, and cancelling work belonging to other windows (#39996).
• (Java) KafkaIO dynamic reads no longer require the obsolete beam_fn_api experiment (#29998).
• (Prism) Self-checkpointing splittable DoFns now resume after their requested delay instead of immediately, so polling SDFs no longer busy-spin (#39848).
• (Java) MongoDbIO read splitting now preserves non-ObjectId _id types (e.g. string ids) instead of failing to parse the generated range filters (#39900).
• (Go) Fixed GCS glob matching silently dropping objects when the glob pattern contains multi-byte characters (#39969).
• (Python) Fixed TensorRTEngineHandlerNumPy failing with CUDA_ERROR_INVALID_VALUE on models with a single-element input or output tensor (#36306).
• (Python) Fixed PickleCoder/_MemoizingPickleCoder.as_deterministic_coder() raising TypeError instead of returning a working deterministic coder (#28558).

[2.76.0] - 2026-08-31

Highlights

• Added a full Iceberg batch and streaming changelog source (CDC) (#38831)

... (truncated)

Commits

• 1a6fce4 Set version for 2.77.0 RC4
• 5cd677e Merge pull request #40337 from claudevdm/cp-40252
• be27783 Exposing evictWritersWhenFull in AvroIO (#40336) (#40338)
• 8c9d31e Merge pull request #40259 from reuvenlax/improve_gcp_presubmit_speed (#40342)
• c45e45c Addfiles partition bounds fixes (#40252)
• 34e08cf Merge pull request #40287 from damccorm/cp-40205-40251-release-2.77
• e1b0794 Merge pull request #40271 from ahmedabu98/cp40253
• bcb44cf Time out RestrictionTrackers.trySplit after 5 minutes on lock contention (#40...
• 84da653 Return last evaluated progress on lock timeout in RestrictionTrackers.getProg...
• 396ae8e Merge branch 'release-2.77' of https://github.com/apache/beam into cp40253
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)