Skip to content

Latest commit

 

History

History
111 lines (84 loc) · 6.27 KB

cs_troubleshoot_health.md

File metadata and controls

111 lines (84 loc) · 6.27 KB
Raw
copyright lastupdated keywords subcollection
years
2014, 2020
2020-02-03
kubernetes, iks, logging, help, debug
containers

{:codeblock: .codeblock} {:deprecated: .deprecated} {:download: .download} {:external: target="_blank" .external} {:faq: data-hd-content-type='faq'} {:gif: data-image-type='gif'} {:help: data-hd-content-type='help'} {:important: .important} {:new_window: target="_blank"} {:note: .note} {:pre: .pre} {:preview: .preview} {:screen: .screen} {:shortdesc: .shortdesc} {:support: data-reuse='support'} {:table: .aria-labeledby="caption"} {:tip: .tip} {:troubleshoot: data-hd-content-type='troubleshoot'} {:tsCauses: .tsCauses} {:tsResolve: .tsResolve} {:tsSymptoms: .tsSymptoms}

Logging and monitoring

{: #cs_troubleshoot_health}

As you use {{site.data.keyword.containerlong}}, consider these techniques for troubleshooting issues with logging and monitoring. {: shortdesc}

If you have a more general issue, try out cluster debugging. {: tip}

Kubernetes dashboard does not display utilization graphs

{: #cs_dashboard_graphs}

{: tsSymptoms} When you access the Kubernetes dashboard, utilization graphs do not display.

{: tsCauses} Sometimes after a cluster update or worker node reboot, the kube-dashboard pod does not update.

{: tsResolve} Delete the kube-dashboard pod to force a restart. The pod is re-created with RBAC policies to access heapster for utilization information.

kubectl delete pod -n kube-system $(kubectl get pod -n kube-system --selector=k8s-app=kubernetes-dashboard -o jsonpath='{.items..metadata.name}')

{: pre}


Log lines are too long

{: #long_lines}

{: tsSymptoms} You set up a logging configuration in your cluster to forward logs to an external syslog server. When you view logs, you see a long log message. Additionally, in Kibana, you might be able to see only the last 600 - 700 characters of the log message.

{: tsCauses} A long log message might be truncated due to its length before it is collected by Fluentd, so the log might not be parsed correctly by Fluentd before it is forwarded to your syslog server.

{: tsResolve} To limit line length, you can configure your own logger to have a maximum length for the stack_trace in each log. For example, if you are using Log4j for your logger, you can use an EnhancedPatternLayout{: external} to limit the stack_trace to 15KB.

Feedback, questions, and support

{: #getting_help}

Still having issues with your cluster? Review different ways to get help and support for your {{site.data.keyword.containerlong_notm}} clusters. For any questions or feedback, post in Slack. {: shortdesc}

General ways to resolve issues

  1. Keep your cluster environment up to date.
  2. Make sure that your command line tools are up to date.
    • In the terminal, you are notified when updates to the ibmcloud CLI and plug-ins are available. Be sure to keep your CLI up-to-date so that you can use all available commands and flags.
    • Make sure that your kubectl CLI client matches the same Kubernetes version as your cluster server. Kubernetes does not support{: external} kubectl client versions that are 2 or more versions apart from the server version (n +/- 2).

Reviewing issues and status

  1. To see whether {{site.data.keyword.cloud_notm}} is available, check the {{site.data.keyword.cloud_notm}} status page{: external}.
  2. Filter for the Kubernetes Service component.

Feedback and questions

  1. Post in the {{site.data.keyword.containershort}} Slack.
    • If you are an external user, post in the #general{: external} channel.
    • If you are an IBMer, use the #armada-users channel.

      If you do not use an IBMid for your {{site.data.keyword.cloud_notm}} account, request an invitation{:external} to this Slack.

  2. Review forums such as {{site.data.keyword.containershort}} help, Stack Overflow, and IBM Developer to see whether other users ran into the same issue. When you use the forums to ask a question, tag your question so that it is seen by the {{site.data.keyword.cloud_notm}} development teams.
    • If you have technical questions about developing or deploying clusters or apps with {{site.data.keyword.containerlong_notm}}, post your question on Stack Overflow{: external} and tag your question with ibm-cloud, containers, and openshift.
    • For questions about the service and getting started instructions, use the IBM Developer Answers{: external} forum. Include the ibm-cloud and containers tags.
    • See Getting help for more details about using the forums.

Getting help

  1. Contact IBM Support by opening a case. To learn about opening an IBM support case, or about support levels and case severities, see Contacting support.
  2. In your support case, for Category, select Containers.
  3. For the Offering, select your {{site.data.keyword.containershort}} cluster.

    When you report an issue, include your cluster ID. To get your cluster ID, run ibmcloud ks cluster ls. You can also use the {{site.data.keyword.containerlong_notm}} Diagnostics and Debug Tool to gather and export pertinent information from your cluster to share with IBM Support.