Skip to content

Latest commit

 

History

History
390 lines (307 loc) · 30.9 KB

AGENTS.md

File metadata and controls

390 lines (307 loc) · 30.9 KB

Flink AI Agent Instructions

This file provides guidance for AI coding agents working with the Apache Flink codebase.

Prerequisites

  • Java 11, 17 (default), or 21. Java 11 syntax must be used in all modules. Java 17 syntax (records, sealed classes, pattern matching) is only permitted in the flink-tests-java17 module.
  • Maven 3.9.16 (Maven wrapper ./mvnw included; prefer it)
  • Git
  • Unix-like environment (Linux, macOS, WSL, Cygwin)

Commands

Build

  • Fast dev build: ./mvnw clean install -DskipTests -Dfast -Pskip-webui-build -T1C
  • Full build (Java 17 default): ./mvnw clean package -DskipTests -Djdk17 -Pjava17-target
  • Java 11: ./mvnw clean package -DskipTests -Djdk11 -Pjava11-target
  • Java 21: ./mvnw clean package -DskipTests -Djdk21 -Pjava21-target
  • Full build with tests: ./mvnw clean verify
  • Single module: ./mvnw clean package -DskipTests -pl flink-core-api
  • Single module with tests: ./mvnw clean verify -pl flink-core-api

Testing

  • Single test class: ./mvnw -pl flink-core-api -Dtest=MemorySizeTest test
  • Single test method: ./mvnw -pl flink-core-api -Dtest=MemorySizeTest#testParseBytes test

Code Quality

  • Format code (Java + Scala): ./mvnw spotless:apply
  • Check formatting: ./mvnw spotless:check
  • Checkstyle: ./mvnw checkstyle:check -T1C
  • Checkstyle config: tools/maven/checkstyle.xml

Repository Structure

Every module from the root pom.xml, organized by function. Flink provides three main user-facing APIs (recommended in this order: SQL, Table API, DataStream API) plus a newer DataStream v2 API.

Core Infrastructure

  • flink-annotations — Stability annotations (@Public, @PublicEvolving, @Internal, @Experimental) and @VisibleForTesting
  • flink-core-api — Core API interfaces (functions, state, types) shared by all APIs
  • flink-core — Core implementation (type system, serialization, memory management, configuration)
  • flink-runtime — Distributed runtime (JobManager, TaskManager, scheduling, network, state)
  • flink-clients — CLI and client-side job submission
  • flink-rpc/ — RPC framework
    • flink-rpc-core — RPC interfaces
    • flink-rpc-akka, flink-rpc-akka-loader — Pekko-based RPC implementation

SQL / Table API (recommended API for most users)

  • flink-table/
    • flink-sql-parser — SQL parser (extends Calcite SQL parser)
    • flink-table-common — Shared types, descriptors, catalog interfaces
    • flink-table-api-java — Table API for Java
    • flink-table-api-scala — Table API for Scala
    • flink-table-api-bridge-base, flink-table-api-java-bridge, flink-table-api-scala-bridge — Bridges between Table and DataStream APIs
    • flink-table-api-java-uber — Uber JAR for Table API
    • flink-table-planner — SQL/Table query planning and optimization (Calcite-based)
    • flink-table-planner-loader, flink-table-planner-loader-bundle — Classloader isolation for planner
    • flink-table-runtime — Runtime operators for Table/SQL queries
    • flink-table-calcite-bridge — Bridge to Apache Calcite
    • flink-sql-gateway-api, flink-sql-gateway — SQL Gateway for remote SQL execution
    • flink-sql-client — Interactive SQL CLI
    • flink-sql-jdbc-driver, flink-sql-jdbc-driver-bundle — JDBC driver for SQL Gateway
    • flink-table-code-splitter — Code generation utilities
    • flink-table-test-utils — Test utilities for Table/SQL

DataStream API (original streaming API)

  • flink-streaming-java — DataStream API and stream processing operator implementations

DataStream API v2 (newer event-driven API)

  • flink-datastream-api — DataStream v2 API definitions
  • flink-datastream — DataStream v2 API implementation

Connectors (in-tree)

  • flink-connectors/
    • flink-connector-base — Base classes for source/sink connectors
    • flink-connector-files — Unified file system source and sink
    • flink-connector-datagen — DataGen source for testing
    • flink-connector-datagen-test — Tests for DataGen connector
    • flink-hadoop-compatibility — Hadoop InputFormat/OutputFormat compatibility
    • flink-file-sink-common — Common file sink utilities
  • Most connectors (Kafka, JDBC, Elasticsearch, etc.) live in separate repos under github.com/apache; see README.md for the full list

Formats

  • flink-formats/
    • flink-json, flink-csv, flink-avro, flink-parquet, flink-orc, flink-protobuf — Serialization formats
    • flink-avro-confluent-registry — Avro with Confluent Schema Registry
    • flink-sequence-file, flink-compress, flink-hadoop-bulk, flink-orc-nohive — Hadoop-related formats
    • flink-format-common — Shared format utilities
    • flink-sql-json, flink-sql-csv, flink-sql-avro, flink-sql-parquet, flink-sql-orc, flink-sql-protobuf — SQL-layer format integrations
    • flink-sql-avro-confluent-registry — SQL-layer Avro with Confluent Schema Registry

State Backends

  • flink-state-backends/
    • flink-statebackend-rocksdb — RocksDB state backend
    • flink-statebackend-forst — ForSt state backend (experimental; a fork of RocksDB)
    • flink-statebackend-heap-spillable — Heap-based spillable state backend
    • flink-statebackend-changelog — Changelog state backend
    • flink-statebackend-common — Shared state backend utilities
  • flink-dstl/flink-dstl-dfs — State changelog storage (DFS-based persistent changelog for incremental checkpointing)

File Systems

  • flink-filesystems/
    • flink-hadoop-fs — Hadoop FileSystem abstraction
    • flink-s3-fs-native, flink-s3-fs-hadoop, flink-s3-fs-presto, flink-s3-fs-base — S3 file systems
    • flink-oss-fs-hadoop — Alibaba OSS
    • flink-azure-fs-hadoop — Azure Blob Storage
    • flink-gs-fs-hadoop — Google Cloud Storage
    • flink-fs-hadoop-shaded — Shaded Hadoop dependencies

Queryable State

  • flink-queryable-state/
    • flink-queryable-state-runtime — Server-side queryable state service
    • flink-queryable-state-client-java — Client for querying operator state from running jobs

Deployment

  • flink-kubernetes — Kubernetes integration
  • flink-yarn — YARN integration
  • flink-dist, flink-dist-scala — Distribution packaging
  • flink-container — Container entry-point and utilities for containerized deployments

Metrics

  • flink-metrics/
    • flink-metrics-core — Metrics API and core implementation
    • Reporter implementations: flink-metrics-jmx, flink-metrics-prometheus, flink-metrics-datadog, flink-metrics-statsd, flink-metrics-graphite, flink-metrics-influxdb, flink-metrics-slf4j, flink-metrics-dropwizard, flink-metrics-otel

Libraries

  • flink-libraries/
    • flink-cep — Complex Event Processing
    • flink-state-processing-api — Offline state access (savepoint reading/writing)

Other

  • flink-models — AI model integration (sub-modules: flink-model-openai, flink-model-triton)
  • flink-python — PyFlink (Python API)
  • flink-runtime-web — Web UI for JobManager dashboard
  • flink-external-resources — External resource management (e.g., GPU)
  • docs/ — Documentation content (Hugo site). This is where user-facing docs are written.
  • flink-docs — Documentation build module (auto-generated config reference docs)
  • flink-examples — Example programs
  • flink-quickstart — Maven archetype for new projects
  • flink-walkthroughs — Tutorial walkthrough projects

Testing

  • flink-tests — Integration tests
  • flink-end-to-end-tests — End-to-end tests
  • flink-test-utils-parent — Test utility classes
  • flink-yarn-tests — YARN-specific tests
  • flink-fs-tests — FileSystem tests
  • flink-architecture-tests — ArchUnit architectural boundary tests
  • tools/ci/flink-ci-tools — CI tooling

Architecture Boundaries

  1. Client submits jobs to the cluster. Submission paths include the CLI (bin/flink run via flink-clients), the SQL Client (bin/sql-client.sh via flink-sql-client), the SQL Gateway (flink-sql-gateway, also accessible via JDBC driver), the REST API (direct HTTP to JobManager), programmatic execution (StreamExecutionEnvironment.execute() or TableEnvironment.executeSql()), and PyFlink (flink-python, wraps the Java APIs).
  2. JobManager (flink-runtime) orchestrates execution: receives jobs, creates the execution graph, manages scheduling, coordinates checkpoints, and handles failover. Never runs user code directly.
  3. TaskManager (flink-runtime) executes the user's operators in task slots. Manages network buffers, state backends, and I/O.
  4. Table Planner (flink-table-planner) translates SQL/Table API programs into DataStream programs. The planner is loaded in a separate classloader (flink-table-planner-loader) to isolate Calcite dependencies.
  5. Connectors communicate with external systems. Source connectors implement the Source API (FLIP-27); sinks implement the Sink API (package sink2). Most connectors are externalized to separate repositories.
  6. State Backends persist keyed state and operator state. RocksDB is the primary backend for production use.
  7. Checkpointing provides exactly-once guarantees. The JobManager coordinates barriers through the data stream; TaskManagers snapshot local state to a distributed file system.

Key separations:

  • Planner vs Runtime: The table planner generates code and execution plans; the runtime executes them. Changes to planning logic live in flink-table-planner; changes to runtime operators live in flink-table-runtime or flink-streaming-java.
  • Codegen vs hand-written operators: Per-record expression logic (casts, projections, filters, function calls) is generated at planning time by cast rules in flink-table-planner/.../functions/casting/ and call generators in flink-table-planner/.../codegen/calls/, then compiled by Janino into the surrounding operator class. Operators with fixed structure (joins, aggregations, source/sink runtime) are hand-written Java in flink-table-runtime or flink-streaming-java. New scalar functions usually only need a BuiltInFunctionDefinitions entry plus a BuiltInScalarFunction subclass. New cast behaviour or a custom call shape needs a cast rule or call generator.
  • API vs Implementation: Public API surfaces (flink-core-api, flink-datastream-api, flink-table-api-java) are separate from implementation modules. API stability annotations control what users can depend on.
  • ArchUnit enforcement: flink-architecture-tests/ contains ArchUnit tests that enforce module boundaries. New violations should be avoided; if unavoidable, follow the freeze procedure in flink-architecture-tests/README.md.

Common Change Patterns

This section maps common types of Flink changes to the modules they touch and the verification they require.

Adding a new SQL built-in function

A built-in function touches six modules. Each step names an example of that step done right. Don't copy a whole older function: most of them use the Calcite path, older test utilities, or test the same case at several levels.

  1. Check what other engines do. Compare the function with Spark, Snowflake, BigQuery, DuckDB, PostgreSQL and similar engines: its name, its NULL handling and the result type for each input type. Follow the closest one, and state the reason for any difference in the PR.
  2. Define it in BuiltInFunctionDefinitions in flink-table-common: name, kind, input and output type strategies, and how it runs, see step 3.
    • Write the name in upper case with underscores, such as URL_DECODE, so that SQL and the Table API use the same name.
    • Reuse a strategy from InputTypeStrategies, TypeStrategies, SpecificInputTypeStrategies or SpecificTypeStrategies before writing a new one.
    • When a new strategy would repeat code that other strategies share, extract a shared abstraction instead of copying it. ValidationOnlyInputTypeStrategy is one: for a function that declares its arguments with staticArguments(...), it fixes the argument count and signature, so a subclass only implements inferInputTypes to validate. The strategies of FROM_CHANGELOG and TO_CHANGELOG use it.
    • Validate literal arguments in the input strategy, so that a malformed literal fails at planning time instead of on the first record.
    • Register the function only here. Don't also add a field for it to FlinkSqlOperatorTable: that class registers every field by reflection, and a second operator with the same name makes compiled plans that call the function fail to restore.
    • Every public class in flink-table-common and flink-table-api-java, nested classes included, needs a visibility annotation such as @Internal. ArchUnit checks this in TableApiRules.
  3. Choose how it runs. The builder call in BuiltInFunctionDefinitions decides:
    • runtimeClass("...") is the default for scalar, table and process table functions. The class extends BuiltInScalarFunction, BuiltInTableFunction or BuiltInProcessTableFunction in flink-table-runtime, and the generated code calls its eval method directly. Precomputing a literal argument, such as parsing a path or compiling a pattern, is no reason for codegen. Do it once in the constructor from SpecializedContext, as VariantGetFunction does.
    • runtimeProvided() on a scalar function means that the planner generates its code. Use it only when a runtime class can't do the job:
      • Performance, when calls can share work. Each call of a runtime class runs on its own, but generated code can reuse a result across the calls in one generated method. PARSE_JSON and TRY_PARSE_JSON parse an input once for all calls on it, which a runtime class can't do.
      • The function reads the operator context, as CURRENT_WATERMARK does.
      • The function evaluates an argument only when needed. A runtime class gets every argument already evaluated. COALESCE and IF_NULL have generated code for this, although they declare runtimeDeferred() and runtimeClass(...): a case in ExprCodeGenerator takes precedence over the builder call.
    • runtimeProvided() on an aggregate function means that AggFunctionFactory.scala picks the implementation, a BuiltInAggregateFunction in flink-table-runtime. Every built-in aggregate takes this path, because the factory can choose a class from planning facts, such as a retracting variant when the input has updates. See BITMAP_BUILD_AGG.
    • runtimeDeferred() maps the function to a Calcite SqlOperator in FlinkSqlOperatorTable. Don't use it for a new function. It remains for older functions and for syntax that the SQL parser must handle, such as JSON_VALUE(... RETURNING ...).
  4. Implement it. See flink-table/flink-table-runtime/AGENTS.md for a runtime class and flink-table/flink-table-planner/AGENTS.md for codegen.
    • A SQL NULL argument gives NULL, unless the function defines otherwise.
    • A TRY_ variant returns NULL where the function would throw. Share one implementation, as TryVariantGetFunction does by passing a flag to VariantGetFunction.
    • Throw a ValidationException for an error found during planning, such as in a type strategy, and a TableRuntimeException for an error on a record.
    • Keep the per-record path fast, in a runtime class and in generated code alike. eval and the generated call run once for every record, so a small cost there multiplies.
      • Do work that doesn't depend on the record once per query. A runtime class does it in its constructor, as VariantGetFunction does for a literal path. Generated code passes it in through ctx.addReusableObject or a reusable member, as StringCallGen does for a literal pattern.
      • Cache what depends on an argument that is not a literal but rarely changes. Compare the argument with a copy of the last internal value, such as a BinaryStringData, whose equals compares bytes. A cache hit then converts nothing.
      • Stay on internal data structures. Converting to a Java object per record, such as StringData#toString(), allocates and copies. Don't allocate per record what can be reused, and avoid boxing.
      • A TRY_ variant doesn't build the error it would have thrown. On a rejected row, TRY_PARSE_JSON skips the message that PARSE_JSON builds for its exception.
      • Don't log in the per-record path. A log line there can fill a TaskManager's disk.
      • When two designs differ on the per-record path, measure both and put the numbers in the PR.
  5. Add the Table API method in flink-table-api-java: on BaseExpressions for a call on a value, such as $("v").variantGet("$.a"), or on Expressions for a call without one, such as uuidV4(). ExpressionSerializationTest checks the SQL that a Table API call prints, so add the call there. Keep the Javadoc to a few sentences: what the call returns, and when it returns NULL or fails. Refer to the SQL function for longer rules, such as a path syntax.
  6. Add the PyFlink method. CI fails without it.
    • A BaseExpressions method goes on Expression in flink-python/pyflink/table/expression.py. An Expressions method goes into flink-python/pyflink/table/expressions.py, with @PublicEvolving() like the functions around it.
    • Name it in snake_case after the Java method, such as variantGet and variant_get. test_expression_completeness.py and test_expressions_completeness.py turn every Python name back into camelCase, and fail on a Java method without a match.
    • Call Java through the helpers in the same file, such as _binary_op("variantGet"). Convert a DataType argument with _to_java_data_type.
    • Keep the docstring as short as the Javadoc. Write SQL NULL as NULL, not None, and give an example.
    • In flink-python/pyflink/table/tests/test_expression.py, assert the SQL that the expression prints, such as "VARIANT_GET(a, '$.x')".
    • List the method in flink-python/docs/reference/pyflink.table/expressions.rst, under its category. No test checks this list.
  7. Test each behavior once, with the cheapest test that covers it. An ITCase runs every case as a query on a mini cluster, so it costs the most CI time.
    • Test the type strategies with a unit test that extends InputTypeStrategiesTestBase or TypeStrategiesTestBase in flink-table-common, such as RegexpExtractInputTypeStrategyTest. Cover every argument type the strategy accepts, including lengths and precisions such as CHAR(33) and TIMESTAMP_LTZ(9), the inferred result type, and each validation error with its exact message. The ITCase doesn't repeat these cases.
    • Test a helper class, such as a path parser, with a plain unit test in its module, as SimplePathTest does.
    • Add the ITCase cases to the existing ITCase of the function's category in flink-table-planner, such as VariantFunctionsITCase for a VARIANT function or CollectionFunctionsITCase for an array function. Don't create a file for one function.
    • In the ITCase, cover what needs a running query: results, NULL inputs and runtime errors. Add a case for each input that the implementation handles differently, not for each type that the strategy accepts. Use testResult to run the SQL and the Table API call against one expected value, and testSqlRuntimeError with the exact message. Call the Table API method, not call("NAME").
    • Don't add a case that another test already covers, directly or through a component the function reuses, such as a cast with its own tests.
  8. Document it in docs/data/sql_functions.yml: add a row with sql, table and description to the function's category. Add the same row to docs/data/sql_functions_zh.yml, in English if you don't write Chinese. Describe what a user can observe: the arguments, the result, the NULL cases and the errors. Leave out implementation details, such as class names, code generation or caching. Check every statement against the type strategies, the runtime class and the tests, and take every result in an example from running its SQL.

Adding a new configuration option

  1. Define ConfigOption<T> in the relevant config class (e.g., ExecutionConfigOptions.java in flink-table-api-java)
  2. Use ConfigOptions.key("table.exec....") builder with type, default value, and description
  3. Add @Documentation.TableOption annotation for auto-generated docs
  4. Document in docs/ if user-facing
  5. Verify: unit test for default value, ITCase for behavior change

Adding a new table operator (e.g., join type, aggregate)

  1. Involves flink-table-runtime (operator), flink-table-planner (ExecNode, physical/logical rules), and tests across both
  2. See flink-table/flink-table-planner/AGENTS.md and flink-table/flink-table-runtime/AGENTS.md for detailed development order and testing patterns

Adding a new connector (Source or Sink)

  1. Implement the Source API (flink-connector-base): SplitEnumerator, SourceReader, SourceSplit, serializers (SimpleVersionedSerializer)
  2. Or implement the Sink API (package sink2) for sinks
  3. Most new connectors go in separate repos under github.com/apache, not in the main Flink repo
  4. Verify: unit tests + ITCase with real or embedded external system

Modifying state serializers

  1. Changes to TypeSerializer require a corresponding TypeSerializerSnapshot for migration
  2. Bump version in getCurrentVersion(), handle old versions in readSnapshot()
  3. Snapshot must have no-arg constructor for reflection-based deserialization
  4. Implement resolveSchemaCompatibility() for upgrade paths
  5. Verify: serializer snapshot migration tests, checkpoint restore tests across versions

Introducing or changing user-facing APIs (@Public, @PublicEvolving, @Experimental)

  1. New user-facing API requires a voted FLIP (Flink Improvement Proposal); this applies to @Public, @PublicEvolving, and @Experimental since users build against all three
  2. Every user-facing API class and method must carry a stability annotation
  3. Changes to existing @Public or @PublicEvolving API must maintain backward compatibility
  4. @Internal APIs can be changed freely; users should not depend on them
  5. Update JavaDoc on the changed class/method
  6. Add to release notes
  7. Verify: ArchUnit tests pass, no new architecture violations

Coding Standards

  • Format Java files with Spotless immediately after editing: ./mvnw spotless:apply. Uses google-java-format with AOSP style.
  • Scala formatting: Spotless + scalafmt (config at .scalafmt.conf, maxColumn 100).
  • Checkstyle: tools/maven/checkstyle.xml (version defined in root pom.xml as checkstyle.version). Some modules (flink-core, flink-optimizer, flink-runtime) are not covered by checkstyle enforcement, but conventions should still be followed.
  • No new Scala code. All Flink Scala APIs are deprecated per FLIP-265. Write all new code in Java.
  • Apache License 2.0 header required on all new files (enforced by Apache Rat). Use an HTML comment for markdown files.
  • API stability annotations: Every user-facing API class and method must have a stability annotation. @Public (stable across minor releases), @PublicEvolving (may change in minor releases), @Experimental (may change at any time). These are all part of the public API surface that users build against. @Internal marks APIs with no stability guarantees that users should not depend on.
  • Logging: Use parameterized log statements (SLF4J {} placeholders), never string concatenation.
  • No Java serialization for new features (except internal RPC message transport).
  • Use final for variables and fields where applicable.
  • Comments: Do not add unnecessary comments that restate what the code does. Add comments that explain "the why" where relevant.
  • Reuse existing code. Before implementing new utilities or abstractions, search for existing ones in the codebase. Prioritize architecture consistency and code reusability.
  • Full code style guide: https://flink.apache.org/how-to-contribute/code-style-and-quality-preamble/

Testing Standards

  • Add tests for new behavior, covering success, failure, and edge cases.
  • Use JUnit 5 + AssertJ assertions. Do not use JUnit 4 or Hamcrest in new test code.
  • Prefer real test implementations over Mockito mocks where possible.
  • Integration tests: Name classes with ITCase suffix (e.g., MyFeatureITCase.java).
  • Red-green verification: For bug fixes, verify that new tests actually fail without the fix before confirming they pass with it.
  • Test location mirrors source structure within each module.
  • Follow the testing conventions at https://flink.apache.org/how-to-contribute/code-style-and-quality-common/#7-testing

Commits and PRs

Commit message format

  • [FLINK-XXXX][component] Description where FLINK-XXXX is the JIRA issue number
  • [hotfix][component] Description for typo fixes without JIRA
  • Each commit must have a meaningful message including the JIRA ID. If you don't know the ticket number, ask.
  • Separate cleanup/refactoring from functional changes into distinct commits
  • When AI tools were used: add Generated-by: <Tool Name and Version> trailer per ASF generative tooling guidance

Pull request conventions

  • Title format: [FLINK-XXXX][component] Title of the pull request
  • A corresponding JIRA issue is required (except hotfixes for typos)
  • Fill out the PR template completely but concisely: describe purpose, change log, testing approach, impact assessment
  • Each PR should address exactly one issue
  • Ensure ./mvnw clean verify passes before opening a PR
  • Always push to your fork, not directly to apache/flink
  • Rebase onto the latest target branch before submitting
  • For user-visible behaviour changes, breaking changes, new SQL features, or new config options: fill in the Release Notes field on the JIRA ticket. The release manager consolidates these when cutting a release. The next version's docs/content/release-notes/flink-X.Y.md will be generated based of the jira tickets, so make sure to fill them in properly.

AI-assisted contributions

  • Disclose AI usage by checking the AI disclosure checkbox and uncommenting the Generated-by line in the PR template
  • Add Generated-by: <Tool Name and Version> to commit messages
  • Never add Co-Authored-By with an AI agent as co-author; agents are assistants, not authors
  • You must be able to explain the design, code, and tests, debug them, and respond to review feedback substantively
  • Reviewer-ready quality bar: the author owns PR quality. PRs that look AI-generated without author refinement (walls of unreviewed prose, scaffolding without behaviour, tests that do not exercise the change, padded commit messages) will be closed without review

Code Review Guidelines

When reviewing a PR or diff against this repo:

  • Look for opportunities to simplify the code, scoped to the diff itself (not pre-existing code outside the change).
  • Flag comments that are obvious (restate what the code already says) or overly verbose.
  • In test code, look for potential flakiness — e.g. Thread.sleep used outside a retry/poll loop, or similar timing-dependent, non-deterministic patterns. Where applicable, suggest clock injection (e.g. a manually-advanced Clock/ManualClock) instead of relying on wall-clock time, or waiting for the actual condition in a loop with a timeout, for deterministic tests.
  • Check that each commit message conforms to Flink conventions: it must start with [FLINK-XXXX] or [hotfix], and must specify a subsystem/component (e.g. [FLINK-XXXX][runtime] Description).
  • If a change introduces a new feature controlled by a config option/flag, check that the resolved state (enabled/disabled, and the effective value) is logged at INFO level when the feature initializes/activates.
  • Consider whether a change should be hidden behind a feature flag, especially if it's non-trivial (touches core paths, changes default behavior, or is hard to reason about in isolation). This is mandatory if the change is risky (correctness, performance, backward-compatibility, or data-safety risk) and no flag/kill-switch already exists.
  • For changes to configuration options, check if the corresponding documentation has been regenerated (this should be covered by tests but flagging it earlier speeds up development)

Boundaries

Ask first

  • Adding or changing @Public, @PublicEvolving, or @Experimental annotations (these are user-facing API commitments requiring a FLIP)
  • Large cross-module refactors
  • New dependencies
  • Changes to serialization formats (affects state compatibility)
  • Changes to checkpoint/savepoint behavior
  • Changes that could impact performance on hot paths (per-record processing, serialization, state access)

Never

  • Commit secrets, credentials, or tokens
  • Push directly to apache/flink; always work from your fork
  • Mix unrelated changes into one PR
  • Use Java serialization for new features
  • Edit generated files by hand when a generation workflow exists
  • Use the legacy SourceFunction or SinkFunction interfaces for connectors; use the Source API (FLIP-27) and Sink API (package sink2) instead
  • Add Co-Authored-By with an AI agent as co-author in commit messages; AI agents are assistants, not authors. Use Generated-by: <Tool Name and Version> instead.
  • Suppress or bypass checkstyle rules (no CHECKSTYLE:ON/CHECKSTYLE:OFF comments, no adding entries to tools/maven/suppressions.xml, no @SuppressWarnings). Fix the code to satisfy checkstyle instead.
  • Add, change, or remove classes outside the org.apache.flink.* package (for example, classes copied from Calcite)
  • Modify Parser.jj (Calcite's generated parser grammar; expected to be removed in future Calcite upgrades)
  • Use destructive git operations unless explicitly requested

References