[BrianMichell]

Summary

Two related fixes to how the S3 kvstore resolves AWS credentials.

1. Don't cache the anonymous fallback of the default chain

MakeAwsCredentialsProvider wrapped every provider in aws_credentials_provider_new_cached. For default, the chain falls back to anonymous credentials, which never expire, so the outer cache kept the anonymous result forever: credentials that became available later (env vars, IMDS, profile) were never picked up.

• MakeCache is now applied per arm (environment, imds, profile, ecs_role). default (which already caches internally) and anonymous are no longer wrapped.
• AsProvider, MakeCache, and MakeDefaultWithAnonymous return nullptr when aws-c-auth fails to construct a provider, instead of dereferencing NULL. MakeAwsCredentialsProvider reports this as an error.

2. Share one credentials provider per aws_credentials context resource

Before, AwsCredentialsResource was config_only and each S3KeyValueStore opened its own provider, so every store re-ran credential resolution (for example IMDS and STS calls). The resource now creates the provider once and every store bound to it reuses it.

Visible effects of removing config_only:

• When a spec is unbound, the default aws_credentials resource is emitted as a context reference instead of inline. Explicit inline specs still round-trip inline.
• S3 driver cache keys now tell apart stores bound to different aws_credentials resource instances.
• Provider construction errors now surface when the context resource is resolved instead of when the store is opened.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[laramiel]

auto default_provider = MakeDefault(profile_name_override);
if (!default_provider) return nullptr;
auto anonymous_provider = MakeAnonymous();
if (!anonymous_provider) return nullptr;

[laramiel]

absl::StrCat("Failed to create credentials provider: ",

[laramiel]

pre-existing: there may be existing errors that should be cleared.
After GetAwsAllocator() add:

aws_reset_error();

[laramiel]

Add

// Subsequent changes to the environment remain cached by the default chain.
SetEnv("AWS_ACCESS_KEY_ID", "AKIDSECOND");
SetEnv("AWS_SECRET_ACCESS_KEY", "secretsecond"); 
{
  auto creds = GetAwsCredentials(provider.get()).result();
  ASSERT_THAT(creds, IsOk());
  EXPECT_FALSE(creds->IsAnonymous());
  EXPECT_EQ(creds->GetAccessKeyId(), "AKIDEXAMPLE");
  EXPECT_EQ(creds->GetSecretAccessKey(), "secretexample");
}

[laramiel]

This is now provided by the Spec and can be removed from the constructor.

[laramiel]

StatusIs(... , HasSubstr("Failed to create credentials provider"))

[laramiel]

IIRC, child may alias parent here.

TENSORSTORE_ASSERT_OK_AND_ASSIGN(
     auto child, Context::FromJson(::nlohmann::json::object_t(), parent));
 ASSERT_NE(child, parent);

Maybe add a negative case as well,

TENSORSTORE_ASSERT_OK_AND_ASSIGN(
    auto child2,
    Context::FromJson({{"aws_credentials", {{"type", "anonymous"}}}},
                      parent));
TENSORSTORE_ASSERT_OK_AND_ASSIGN(auto r4,
                                 child2.GetResource(resource_spec));
EXPECT_NE(r1->credentials_provider.get(),
          r4->credentials_provider.get());

// rename r4 below to r5. :)