tests: Adds e2e-test-auth-img project

[claudiubelu]

The gcr.io/k8s-staging-e2e-test-auth-img and k8s.gcr.io/kubernetes-e2e-test-auth-img registries are meant to be private, to be used in a few E2E test scenarios, and it is meant to replace the gcr.io/authenticated-image-pulling.

An auth token with read-only access will have to be generated afterwards, and then baked into the E2E tests.

Related: #1459
Related: #1528

[ameukam]

@claudiubelu Please add a comment in the images.yaml file as suggested: https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io#creating-staging-repos.

[claudiubelu]

@claudiubelu Please add a comment in the images.yaml file as suggested: https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io#creating-staging-repos.

Ty!

[ameukam]

/lgtm
/hold
/assign @spiffxp @BenTheElder

[spiffxp]

I have this sneaking suspicion we can't get away with k8s.gcr.io/kubernetes-e2e-test-auth-img being private, while the rest of k8s.gcr.io is public.

EDIT: correct, we cannot accomplish what the description is asking for

ref: https://cloud.google.com/container-registry/docs/access-control#public

Within a project's host, it is not possible to publicly serve only specific images. If you have specific images you want to make public:

• Take care to keep them in a separate host location which you make public, or
• Create a new project to hold a publicly accessible images.

[spiffxp]

I also feel like "should the tests that use this even exist?" remains unanswered: kubernetes/kubernetes#97026 (comment)

ok that comment is less relevant, this PR is for authenticated-image-pulling, into which I have not dug deeply

[spiffxp]

An auth token with read-only access will have to be generated afterwards

Can you describe that a little bit more? I feel like that should be part of this PR. I suspect we can get away with either copying or refactoring what we use in ensure-conformance.sh to at least ensure the service account that will be authed is present:

k8s.io/infra/gcp/ensure-conformance-storage.sh

Lines 104 to 131 in 9e2220c

	# ensure_conformance_serviceaccount ensures that:
	# - a serviceaccount of the given name exists in PROJECT
	# - it can write to the given bucket
	# - it has a private key stored in a secret in PROJECT accessible to the given group
	function ensure_conformance_serviceaccount() {
	local name="${1}"
	local bucket="${2}"
	local secret_accessors="${3}"
	local email="$(svc_acct_email "${PROJECT}" "${name}")"
	local secret="${name}-key"
	local private_key_file="${TMPDIR}/key.json"
	color 6 "Ensuring service account exists: ${email}"
	ensure_service_account "${PROJECT}" "${name}" "Grants write access to ${bucket}"
	color 6 "Ensuring ${PROJECT} contains secret ${secret} with private key for ${email}"
	ensure_serviceaccount_key_secret "${PROJECT}" "${secret}" "${email}"
	color 6 "Empowering ${secret_accessors} to access secret: ${secret}"
	ensure_secret_role_binding \
	"projects/${PROJECT}/secrets/${secret}" \
	"group:${secret_accessors}" \
	"roles/secretmanager.secretAccessor"
	color 6 "Empowering ${email} to write to ${bucket}"
	empower_svcacct_to_write_gcs_bucket "${email}" "${bucket}"
	}

[spiffxp]

I'm not sure this is a pattern we want to encourage more of?

[spiffxp]

/cc @thockin @listx @hh @saschagrunert @jeremyrickard
Given that we can't use k8s.gcr.io for this, I have a couple questions/thoughts:

• should we be thinking of promoting to a private repo, or should we just use a one-off private repo? if the latter, should we drop the "k8s-staging" from the name for this one-off repo?
• should we be thinking about cross-cloud private artifacts for registry.k8s.io? or do we just decide to take the hit and have the world hit this one private repo in GCR whenever they run their kubernetes e2e tests?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: claudiubelu
To complete the pull request process, please ask for approval from spiffxp after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS
• groups/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ameukam]

/lgtm
gentle ping @BenTheElder @spiffxp

[BenTheElder]

conceptually seems OK but not comfortable approving the whole thing myself given prior debate about how this should be done / named / ...

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[claudiubelu]

So, the question remains: can this be made private? If so, how? It seems that kubernetes/k8s.io changed since this was initially proposed, so this just simply creates a new project, it's no longer trying to make it private at the moment.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[BenTheElder]

This was always done on a separate registry, and it was always a GCR registry.

I don't think the purpose of the test is to test cloud providers at all, just the flow of an image that requires credentials.

I don't see a reasonable better alternative proposed here, other than perhaps giving up on the test ...

I'm going to drop a link in the SIG slack, I think this PR deserved a follow-up answering @claudiubelu's questions.

[BenTheElder]

should we be thinking of promoting to a private repo, or should we just use a one-off private repo? if the latter, should we drop the "k8s-staging" from the name for this one-off repo?

IMO: yes. we should give it a one-off, more descriptive name if we do this.

should we be thinking about cross-cloud private artifacts for registry.k8s.io? or do we just decide to take the hit and have the world hit this one private repo in GCR whenever they run their kubernetes e2e tests?

Proposing one alternative (maybe) ... can we rely on running a simple authenticated registry in-cluster as part of the test? And just not serve a public endpoint for this purpose at all. We could probably bake up a docker image that is an authenticated registry with some image pre-pushed to it, the question is mostly of if we can run an in-cluster registry sufficiently portably for test purposes. I think the answer might be "no, we can't" (TLS? CRI <-> pod reachability?), but perhaps worth exploring.

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.