Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Prometheus Receiver] Generic OTel Prometheus Receiver configuration for each K8s component that works for all or major certified K8s distros #33937

Open
developer1622 opened this issue Jul 7, 2024 · 5 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed receiver/prometheus Prometheus receiver Stale

Comments

@developer1622
Copy link

Component(s)

receiver/prometheus

Is your feature request related to a problem? Please describe.

I don't know if there was something already exists.

OTel has kubeletstatsreceiver and k8sclusterreceiver, which fetches only some K8s component metrics.

I was wondering if there is a generic or template OpenTelemetry configuration manifest that we can apply to any or major certified K8s distribution such that I would be able to get every K8s component Prometheus metrics (both control plane components and data plane components), assuming that OpenTelemetry could be deployed as Daemonset(for Kubeproxy etc.) and Deployments (etcd, coredns, apiserver, kube-api scheduler, kube-controller etc ).

The major problem is writing scrape configuration for each variant of certified K8s distribution, which might vary with their labels and annotations.

Describe the solution you'd like

I have tried some scrape configurations for the vanilla K8s cluster. Some of the components worked, but not all.

I did try something from the Kubecon event:

https://raw.githubusercontent.com/pavolloffay/kubecon-na-2023-opentelemetry-kubernetes-metrics-tutorial/main/backend/06-collector-k8s-cluster-metrics.yaml

As I said, it is not working for some components and I will have to change the scrape configuration suitable for the running K8s cluster. Some of the components worked, but not all

Describe alternatives you've considered

I am not sure and I did not consider any idea.

Additional context

A single generic standard OTel Prometheus receiver configuration that fetches the metrics of all the K8s cluster components (control plane and data plane) for major or any certified K8s distribution when deployed as both Deamonset and Deployment.

We need two OTel Prometheus receiver configurations: one for Deployment(apiserver, etcd, coredns, cloud controller(if on a cloud) etc.) and another for Kube proxy, CNI, and CSI(if they are exposed).

Kubelet has been covered under OTel native kubeletstats receiver.

Thank you.

@developer1622 developer1622 added enhancement New feature or request needs triage New item requiring triage labels Jul 7, 2024
Copy link
Contributor

github-actions bot commented Jul 7, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the receiver/prometheus Prometheus receiver label Jul 7, 2024
@dashpole dashpole added documentation Improvements or additions to documentation help wanted Extra attention is needed and removed needs triage New item requiring triage labels Jul 8, 2024
@developer1622
Copy link
Author

Thank you @dashpole for adding labels

May I know if there has been any discussion happened on this? Thanks

@dashpole
Copy link
Contributor

I don't know of an example like that today, but anyone is welcome to contribute one. There is a logging example for k8s: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/examples/kubernetes. It might be good to add the metrics example separately.

Otherwise, this is waiting for someone interested in working on it.

@developer1622
Copy link
Author

Hi @dashpole, thank you for your response. I appreciate your help with confirmation.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed receiver/prometheus Prometheus receiver Stale
Projects
None yet
Development

No branches or pull requests

2 participants