Skip to content

Retry Pub/Sub topic and subscription creation in TestPubsub and TestPubsubSignal - #41064

Draft
damccorm wants to merge 2 commits into
masterfrom
fix/test-pubsub-admin-retry
Draft

damccorm wants to merge 2 commits into
masterfrom
fix/test-pubsub-admin-retry

Conversation

@damccorm

@damccorm damccorm commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes flaky DeadlineExceededException failures in PostCommit SQL (PubsubTableProviderIT and BeamSqlLineIT, e.g. https://github.com/apache/beam/actions/runs/42312418759, https://github.com/apache/beam/actions/runs/41878199992, https://github.com/apache/beam/actions/runs/41779718356).

In com.google.cloud.pubsub.v1, TopicAdminClient.createTopic and SubscriptionAdminClient.createSubscription have a 60s RPC timeout with no default retries configured on DEADLINE_EXCEEDED / UNAVAILABLE. When a single createSubscription or createTopic RPC in TestPubsub.initializePubsub or TestPubsubSignal.initializePubsub stalls for 60s against pubsub.googleapis.com, the test immediately fails with com.google.api.gax.rpc.DeadlineExceededException.

This PR adds a 3-attempt retry helper in TestPubsub (used by both TestPubsub and TestPubsubSignal, similar to PubsubResourceManager) that retries createTopic and createSubscription on DeadlineExceededException and UnavailableException (treating AlreadyExistsException on retry as success).

Verification

@damccorm

damccorm commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@github-actions github-actions Bot added java io gcp size/s S: 10-49 lines changed labels Nov 7, 2026
@damccorm

damccorm commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR introduces 3-attempt retry logic for Pub/Sub topic and subscription creation in TestPubsub, TestPubsubSignal, and PubsubWriteIT to mitigate flaky integration test failures caused by transient DEADLINE_EXCEEDED and UNAVAILABLE errors. The implementation cleanly handles AlreadyExistsException / ALREADY_EXISTS on retries and preserves teardown state invariants.

🔍 General Feedback

  • The retry wrappers are well-scoped, idiomatic, and align with existing resource management patterns in the codebase.
  • Treating AlreadyExistsException (and Status.Code.ALREADY_EXISTS) as success on retry safely handles cases where a previous attempt succeeded server-side before timing out.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gcp io java size/s S: 10-49 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant