Repository navigation
CoreEx v4.3.0 release - #192
Conversation
Introduce non-packable CoreEx.IO contracts, structured diagnostics, processing summaries, unit handler hooks, and baseline coverage. Add and adopt CoreEx argument guards and document the phased migration design. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Resolve fresh registered contexts in async DI scopes while preserving the ambient parent across success, failures, cancellation, and disposal. Add focused regression coverage and document explicit trusted context propagation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Implement whole-file validation and scoped replay execution with streamed diagnostics, strict decoding, and lifecycle coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Share the flat and hierarchical lifecycle, add typed routing and ordered group assembly, and cover scoped replay and reconciliation. Record framework type-file and property-spacing conventions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Enforce input-byte and root-group budgets on validation and replay. Handle matching Unicode BOMs, reject invalid encoding, and document measured operating budgets with cross-framework coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Introduce batch file-definition generation, runtime templates, configuration schemas and documentation, and generated-consumer regression coverage. Integrate OnRamp 2.4.0 helpers and preserve RefData output behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Support unordered routed validation and scoped execution, typed generated dispatch, file-level instrumentation, strict dialect guards and source-column validation labels. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Add a generation-first Functions inventory example with SQL Server transactional receipts/outbox and Azure Blob reports. Verify real ingestion through Aspire/Azurite, ship File AI assets, enforce CodeGen parity in CI, and document current capabilities. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
There was a problem hiding this comment.
🔵 Needs a closer look
The new public CSV contracts and Blob-to-SQL receipt/outbox flow require architectural and runtime review of replay, retries, and transaction boundaries, with infrastructure-backed verification unavailable in this environment.
No open issues 🎉
What changed in this PR
Adds the host-neutral CoreEx.IO CSV runtime and File CodeGen, demonstrated through an Azure Functions inventory import with ETag-pinned Blob reads, durable reports, and transactional SQL receipts, mutations, and outbox writes.
Changes:
- Introduces flat, routed, and hierarchical CSV processing with immutable replay, whole-file validation, diagnostics, and bounded inputs.
- Adds File generation commands, templates, configuration references, and generated inventory artifacts.
- Integrates Functions/Aspire testing, generation-parity CI, package publishing, and developer guidance.
This pull request requires changes. Three moderate issues remain:
FileMapping_cs.hbs:23: Generated names can shadowSystem.DateTime, causing valid configurations to produce uncompilable mappings. Qualify emitted BCL types or reject conflicting names.FileMapping_cs.hbs:23: A conversion helper can share its enclosing mapping class’s name, producing CS0542. Separate helpers or validate this collision before generation.InventoryImportLocal.cs:10: Add tests throughRunFileAsynccovering immutable replay after source changes, snapshot-size limits, and cancellation before execution; existing stream-factory tests do not verify this adapter’s snapshot or revision-hash behavior.
| File | Description |
|---|---|
tools/CoreEx.Tooling.Console/Program.cs |
Generates File schemas and configuration documentation. |
tools/CoreEx.Tooling.Console/GlobalUsing.cs |
Centralizes tooling imports. |
tests/CoreEx.Test.Unit/ArgumentGuardExtensionsTests.cs |
Tests the whitespace argument guard. |
tests/CoreEx.IO.Test.Unit/GlobalUsing.cs |
Provides CSV runtime-test imports. |
tests/CoreEx.IO.Test.Unit/Fixtures/orders-valid.csv |
Supplies valid hierarchical input. |
tests/CoreEx.IO.Test.Unit/Fixtures/orders-invalid-trailer.csv |
Supplies trailer reconciliation failure input. |
tests/CoreEx.IO.Test.Unit/Fixtures/orders-invalid-structure.csv |
Supplies structural failure input. |
tests/CoreEx.IO.Test.Unit/Fixtures/flat-valid.csv |
Covers quoting and multiline input. |
tests/CoreEx.IO.Test.Unit/Fixtures/flat-invalid.csv |
Supplies late conversion failure input. |
tests/CoreEx.IO.Test.Unit/Fixtures/baseline.json |
Records expected fixture outcomes. |
tests/CoreEx.IO.Test.Unit/FileBaselineFixtureTests.cs |
Checks fixture integrity and diagnostic anchors. |
tests/CoreEx.IO.Test.Unit/CoreEx.IO.Test.Unit.csproj |
Adds runtime-test dependencies and fixtures. |
tests/CoreEx.CodeGen.Test.Unit/GlobalUsing.cs |
Provides generator-test imports. |
tests/CoreEx.CodeGen.Test.Unit/CoreEx.CodeGen.Test.Unit.csproj |
Defines the generator-test project. |
tests/CoreEx.Azure.Functions.TestApp/Program.cs |
Registers inventory-import configuration. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryProcessor.cs |
Adds trusted-context and reporting hooks. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryImportRunner.cs |
Coordinates import execution and reporting. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryImportRegistration.cs |
Registers import and persistence services. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryImportOptions.cs |
Defines deployment-owned settings. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryImportLocal.cs |
Provides bounded immutable local snapshots. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryImportIdentity.cs |
Derives receipt and report identities. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryBlobImport.cs |
Pins Blob reads to an ETag. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/InventoryBlobFunctions.cs |
Adds the native inventory Blob trigger. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/IInventoryImportService.cs |
Defines the transactional application boundary. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/Files/InventoryHandler.cs |
Delegates unit execution to the application service. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/Files/InventoryContext.g.cs |
Adds generated import processing context. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/Files/InventoryAdjustmentValidator.g.cs |
Adds generated adjustment validation. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/Files/InventoryAdjustmentMapping.g.cs |
Adds generated CSV field mapping. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/Files/InventoryAdjustment.g.cs |
Adds the generated nullable wire model. |
tests/CoreEx.Azure.Functions.TestApp/InventoryImports/example.csv |
Supplies example inventory adjustments. |
tests/CoreEx.Azure.Functions.TestApp/GlobalUsing.cs |
Adds CSV, Blob, and SQL imports. |
tests/CoreEx.Azure.Functions.TestApp/CoreEx.Azure.Functions.TestApp.csproj |
References IO and SQL Server libraries. |
tests/CoreEx.Azure.Functions.TestApp/AGENTS.md |
Documents capability ownership and verification boundaries. |
tests/CoreEx.Azure.Functions.TestApp.CodeGen/Program.cs |
Adds the inventory generation entry point. |
tests/CoreEx.Azure.Functions.TestApp.CodeGen/inventory-file.yaml |
Defines inventory mapping and validation. |
tests/CoreEx.Azure.Functions.TestApp.CodeGen/CoreEx.Azure.Functions.TestApp.CodeGen.csproj |
Adds the design-time generation project. |
tests/CoreEx.Azure.Functions.Test.Unit/GlobalUsing.cs |
Adds inventory component-test imports. |
tests/aspire/CoreEx.Azure.Functions.Test.Aspire/InventorySchema.sql |
Defines inventory, receipts, and outbox fixtures. |
tests/aspire/CoreEx.Azure.Functions.Test.Aspire/GlobalUsing.cs |
Adds inventory runtime-test imports. |
tests/aspire/CoreEx.Azure.Functions.Test.Aspire/FunctionsRuntimeTests.cs |
Includes the inventory SQL port check. |
tests/aspire/CoreEx.Azure.Functions.Test.Aspire/CoreEx.Azure.Functions.Test.Aspire.csproj |
Embeds the inventory schema fixture. |
tests/aspire/CoreEx.Azure.Functions.Test.Aspire/BlobRuntimeTests.cs |
Prepares inventory tests and expands poison-queue inspection. |
tests/aspire/CoreEx.Azure.Functions.Test.AppHost/AppHost.cs |
Wires inventory settings and SQL access. |
src/CoreEx/Extensions.cs |
Adds a fluent whitespace argument guard. |
src/CoreEx.IO/GlobalUsing.cs |
Provides CSV runtime imports. |
src/CoreEx.IO/FileProcessingOutcome.cs |
Defines returned processing outcomes. |
src/CoreEx.IO/FileProcessingMode.cs |
Defines validation and execution modes. |
src/CoreEx.IO/FileProcessingIdentity.cs |
Separates definition, revision, and attempt identity. |
src/CoreEx.IO/FileProcessingContext.cs |
Holds invocation policy and progress. |
src/CoreEx.IO/FileErrorMode.cs |
Defines content-error continuation policy. |
src/CoreEx.IO/FileDiagnosticCategory.cs |
Classifies content diagnostics. |
src/CoreEx.IO/Csv/CsvRoutedInput.cs |
Carries projected records and validator dispatch. |
src/CoreEx.IO/Csv/CsvRoutedDefinition.cs |
Validates immutable unordered route tables. |
src/CoreEx.IO/Csv/CsvRoutedContext.cs |
Tracks pass-local route occurrences. |
src/CoreEx.IO/Csv/CsvRecordRoute.cs |
Defines mapping and validation route contracts. |
src/CoreEx.IO/Csv/CsvRecordPosition.cs |
Defines file-boundary restrictions. |
src/CoreEx.IO/Csv/CsvRecordCardinality.cs |
Defines occurrence constraints. |
src/CoreEx.IO/Csv/CsvProcessingTelemetry.cs |
Adds bounded-outcome processing metrics. |
src/CoreEx.IO/Csv/CsvParsedRecord.cs |
Carries parser state and syntax diagnostics. |
src/CoreEx.IO/Csv/CsvInputRecord.cs |
Carries mapped records and diagnostics. |
src/CoreEx.IO/Csv/CsvHierarchyNode.cs |
Defines ordered hierarchy positions. |
src/CoreEx.IO/Csv/CsvFileOptions.cs |
Configures dialect, decoding, and limits. |
src/CoreEx.IO/Csv/CsvFieldConverter.cs |
Defines non-throwing field conversion. |
src/CoreEx.IO/CoreEx.IO.csproj |
Adds the publishable CSV runtime package. |
src/CoreEx.CodeGen/Scripts/file-script.yaml |
Configures File artifact generation. |
src/CoreEx.CodeGen/RefData/docs/Property.md |
Removes a trailing blank line. |
src/CoreEx.CodeGen/RefData/docs/Entity.md |
Removes a trailing blank line. |
src/CoreEx.CodeGen/RefData/docs/CodeGeneration.md |
Removes a trailing blank line. |
src/CoreEx.CodeGen/GlobalUsing.cs |
Adds generator infrastructure imports. |
src/CoreEx.CodeGen/Files/Templates/FileValidator_cs.hbs |
Generates validation and cached regexes. |
src/CoreEx.CodeGen/Files/Templates/FileRecordBase_cs.hbs |
Generates common routed record types. |
src/CoreEx.CodeGen/Files/Templates/FileRecord_cs.hbs |
Generates nullable wire records. |
src/CoreEx.CodeGen/Files/Templates/FileMapping_cs.hbs |
Generates mappings and exact date converters. |
src/CoreEx.CodeGen/Files/Templates/FileHandler_cs.hbs |
Generates create-only handler starters. |
src/CoreEx.CodeGen/Files/Templates/FileGroup_cs.hbs |
Generates typed hierarchy groups. |
src/CoreEx.CodeGen/Files/Templates/FileContext_cs.hbs |
Generates definition-specific contexts. |
src/CoreEx.CodeGen/Files/Generators/RoutedGenerator.cs |
Selects unordered definition artifacts. |
src/CoreEx.CodeGen/Files/Generators/RootGenerator.cs |
Selects definition-wide artifacts. |
src/CoreEx.CodeGen/Files/Generators/RecordGenerator.cs |
Selects per-record artifacts. |
src/CoreEx.CodeGen/Files/Generators/GroupGenerator.cs |
Selects assembled hierarchy artifacts. |
src/CoreEx.CodeGen/Files/docs/Structure.md |
Documents hierarchy configuration. |
src/CoreEx.CodeGen/Files/docs/ReferenceData.md |
Documents reference-code validation. |
src/CoreEx.CodeGen/Files/docs/Record.md |
Documents record routing configuration. |
src/CoreEx.CodeGen/Files/docs/Node.md |
Documents hierarchy-node configuration. |
src/CoreEx.CodeGen/Files/Config/ReferenceDataConfig.cs |
Validates reference-data configuration. |
src/CoreEx.CodeGen/Files/Config/ConfigValidation.cs |
Validates identifiers and namespaces. |
src/CoreEx.CodeGen/CoreEx.CodeGen.csproj |
Separates File and RefData documentation folders. |
src/CoreEx.CodeGen/CommandType.cs |
Adds the File generation command. |
src/CoreEx.Azure.Functions/README.md |
Documents CSV integration and verification limits. |
src/CoreEx.Azure.Functions/AGENTS.md |
Adds generation-first CSV guidance. |
schema/coreex-refdata.json |
Clarifies the reference-data schema title. |
nuget-publish.ps1 |
Includes CoreEx.IO in publishing. |
docs/capabilities.md |
Documents CSV runtime and generation capabilities. |
Directory.Packages.props |
Pins CsvHelper and updates OnRamp. |
CoreEx.Core.Test.Parallel.slnf |
Includes IO and generator tests. |
CoreEx.Core.slnf |
Includes runtime, generation, and test projects. |
consumer-instructions/.github/copilot-instructions.md |
Adds consumer-facing IO and File guidance. |
CHANGELOG.md |
Records the CSV processing enhancement. |
.github/workflows/CI.yml |
Checks generation parity and updates runtime coverage labeling. |
.github/skills/coreex-functions/references/workflow.md |
Adds transactional CSV Blob workflow guidance. |
.github/skills/coreex-functions/references/patterns.md |
Links generation-first CSV patterns. |
.github/skills/coreex-file/references/patterns.md |
Documents source, report, and transaction boundaries. |
.github/skills/coreex-docs-sync/README.md |
Updates documentation bundle contents. |
.github/prompts/coreex-file.prompt.md |
Adds the Copilot File workflow wrapper. |
.github/instructions/coreex-tooling.instructions.md |
Documents File commands and updated documentation paths. |
.github/instructions/coreex-tests.instructions.md |
Clarifies simulated Blob notification boundaries. |
.github/instructions/coreex-functions.instructions.md |
Adds CSV composition and testing invariants. |
.github/coreex-ai-workflows.md |
Catalogs File workflows and guidance. |
.github/agents/README.md |
Updates package-guide cache documentation. |
.claude/commands/coreex-file.md |
Adds the Claude File workflow wrapper. |
Review effort: Balanced
Add standalone solution/Test.Common and independent CodeGen templates, separate Functions package policy from shared business layers, and extend scaffold guidance and validation. Include File generator naming guards and updated Functions examples. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
…uration Persist cleaned property replacements, classify final Service Bus failures, and add per-call validation exception configuration with regression coverage and updated release guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
There was a problem hiding this comment.
🔵 Needs a closer look
The new CSV runtime, generated consumers/templates, and Blob-to-SQL/outbox retry/report boundaries require multi-framework and native-runtime CI confirmation that the restricted review environment cannot provide.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Run framework sample suites and Aspire E2E on isolated runners, merge coverage, and preserve the build status as an aggregate success gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>

CoreEx v4.3.0
Official release PR for CoreEx v4.3.0.
Version.propsis set to4.3.0, and the detailedCHANGELOG.mdrelease entry is finalized. This PR brings together the release-wide changes below while retaining the original File processing implementation details and verification history.Release highlights
CoreEx.IOCSV runtime and generation-first File workflows inCoreEx.CodeGen.UnitTestEx,UnitTestEx.Aspire,UnitTestEx.Azure.Functions, andUnitTestEx.NUnitto 6.1.0, including central versions and generated template pins.Detailed release changes
File processing and File CodeGen
CoreEx.IOruntime for flat, unordered routed, and hierarchical CSV processing, with immutable stream replay, whole-file validation before execution, streamed diagnostics, bounded input/group sizes, strict encoding, and isolated Result-returning unit handlers.CoreEx.CodeGenwithCommandType.File, top-level*-file.yamldiscovery, schema/configuration references, and generated typed records, mappings, validators, contexts, processors, and create-only handlers.CoreEx.Template; update package publishing, changelog, and current-state guidance.Azure Functions and composable scaffolding
CoreEx.Azure.Functionsisolated-worker host support with scoped CoreEx execution context, HTTPWebApiintegration including explicit idempotency/replay support, and explicit Service Bus settlement for CloudEvent and typed native JSON subscribers. Include native Blob/Event Grid integration examples and UnitTestEx invocation/Aspire runtime proofs while preserving Functions-owned trigger and retry behavior.coreex-functions, a minimal Functions host template and matching UnitTestEx test project, supporting shared Application/Infrastructure libraries by default, standalone integrations via--shared-core false, and multiple independently named Function Apps.--test-common-projectfor an explicit shared test-library reference independent of host/shared-library naming; a supplied missing project fails the build.shared-core; use inline pinned versions only when CPM is disabled. Existing CPM roots need corresponding central package entries; project-local opt-outs remain supported.coreex-solutionfor standalone.slnx, central package/build conventions, root guidance, and lightweight Test.Common by default (--test-common falseopts out), without mandatory business layers or infrastructure. Existing-project retrofits preserve package versions, aliases, and host implementation.coreex-codegenconsole scaffolding without database/reference-data prerequisites or runtime-project references. File definitions target existing runtime projects through explicitfilecommands; RefData remains the default command, and no dummy definitions are generated.HTTP response behavior and customization
WithAccept()as an immediate guard against the effective generated JSON media type for MVC and Minimal WebApi. Support wildcards, media parameters, quality exclusions, copied/keyed defaults, 406 for incompatible requests, and 400 for malformed headers while preserving error rendering and existing Vary values. Set response overrides before the guard and call it before business execution; no automatic negotiation/enforcement is introduced.WebApiBase.DefaultJsonMediaTypeand per-operationWithContentType(...), including option-copy support and MVC/Minimal parity. The default remainsapplication/json; explicit results and error representations are unchanged.CreateResultForException(HttpContext, Exception, IExtendedException?)hooks to MVC and Minimal WebApi. Classification, logging, propagation, and applicable Retry-After headers stay outside overrides; renderer failures propagate without re-entry. Keyed DI examples demonstrate independent serializer, success-media-type, and error configuration.Result/Result<T>propagate whenConvertUnhandledExceptionsToProblemDetailsis false. Explicitly enable the flag for conversion inside WebApi. Own-operation-token cancellation propagates consistently, and generated 204/304 responses no longer carry JSON bodies.Validation, Result, and cleanup
Result.ThrowOnError(),Result<T>.ThrowOnError(), and failed result value access instead of wrapping previously thrown errors in a newAggregateException. Explicitly aggregated errors remain unchanged.Action<ValidationException>configuration overloads toValidator<TEntity>.ValidateAndThrowAsyncandValidateWithResultAsync, including overloads withoutValidationArgsand throwing support throughIValidatorEx<T>/ValidatorBase. Configuration runs once only on validation failure and is never retained on shared validators. Configured overloads require an explicit cancellation token; convenience overloads reject null actions.IValidationResult.ThrowOnError(Action<ValidationException>)with a default interface implementation for existing implementers and fluent overloads onValidationContext<TEntity>andMultiValidatorResult. Success returns the same result; non-validation exception identity and stack traces are preserved.DateTimevalues back to writable generated-contract and reflection-backed POCO properties, includingDateTime.Kindchanges. Preserve cleanup exclusions, read-only properties, nested graph identity, wrapped values, and existingCleanAndDefaultsemantics; ordinary object cleanup does not require[String]setters.Messaging and observability
EventSubscriberBase.AutoCleanValue(default false) and nullable per-handler/message-typeSubscribedBase.AutoCleanValueoverrides. CloudEvents and native JSON subscriber bodies use WebApi-equivalent cleaning before required checks, validation, and handling without root/nested collapse or serialized-message/key/metadata changes. Cleanup failures follow existing receive error handling; retry/settlement policies are unchanged.EventFormatter.CaptureBaggage,EventSubscriberBase.PropagateBaggage, or native FunctionsFunctionsServiceBusReceiveArgs.PropagateBaggage. Incoming baggage is bounded; automatic trace-context linking is independent and unchanged.ServiceBusExceptionpublication failures into CoreExTransientExceptionafter configured send retries, preserving the final broker exception asInnerExceptionand enabling logging. Batch-creation failures use the same classification outside send retries; permanent failures and caller cancellation remain unchanged. CoreEx WebApi defaults to HTTP 503 withRetry-After: 90. Publication acceptance can be partial or unconfirmed, so retries still require application-level idempotency; committed outbox-backed requests remain successful.Dependencies and upgrade notes
WithAspireTester<TAppHost>now comes from UnitTestEx rather than CoreEx.UnitTesting, with the same namespace and behavior.AggregateExceptionwrapper from Result failures must catch the underlying exception instead.TransientException.(value, null, cancellationToken)and(value, null, default)become ambiguous with the new configuration overloads; useargs: nullto select the original overload.Verification
The following results are retained from the original implementation history; they are not claims of a fresh run against the final UnitTestEx 6.1.0 release commit:
Final release validation, including the UnitTestEx 6.1.0 dependency update, is delegated to CI at the maintainer's request. The additional local template-pack validation started for that update was still running when this PR description was updated; no new successful result is asserted here.
Whole-file acceptance does not imply whole-file atomicity or exactly-once effects. The local runtime tests do not prove Azure Event Grid provisioning, authentication, cloud delivery, or outbox relay publication.