Infrastructure Metrics Not Visible in OpsRamp Portal

Issue

Metrics are not visible in the OpsRamp portal even though the Kubernetes 2.0 agent pods are running.

Cause

1. Metrics not enabled
   • During integration, the metrics option was not enabled.
   • Alternatively, the downloaded/copied manifest does not have metrics enabled for required components.
2. Pipeline limitations
   • The agent fetches metrics from Kubernetes components (API Server, Controller, Scheduler, etcd, kube-proxy, etc.). If metrics are not enabled, these pipelines will not process data.
3. Cluster port changes
   • The Kubernetes 2.0 agent assumes standard ports for Kubernetes components (e.g., kubelet on port 10250). If ports are changed, the agent cannot collect metrics.
4. Kubernetes manifest configuration
   • Some metrics (e.g., volume-related) require Kubernetes manifests to include a ConfigMap or PVC (PersistentVolumeClaim). Without these, metrics cannot be generated, as the limitation comes from OpenTelemetry, not the OpsRamp agent.

Solution

1. Enable Metrics During Installation
   • While configuring the Kubernetes 2.0 integration, ensure that metrics are enabled.
   • Alternatively, edit the manifest and set the required components to true:

     metrics:
       kubeCluster: true
       kubeApiServer: true
       kubeControllerManager: true
       kubeScheduler: true
       kubelet: true
       kubeletStats: true
       kubeProxy: true
       coreDNS: true
       etcd: true
       cAdvisor: true

2. Verify Cluster Ports
   • Confirm that Kubernetes components are running on their default ports.
   • For example, kubelet should be accessible on port 10250. If ports are customized, update the manifests accordingly.
3. Check Manifest Configuration
   • If you need volume-related metrics, ensure your pod manifests include ConfigMaps or PVCs.
   • Example: Attach a PVC in your pod spec to collect storage metrics.
4. Review Infrastructure Metrics Config
   • Check the ConfigMap:

     kubectl get configmap opsramp-k8s-infra-metric-user-config -n <namespace>

   • Adjust the configuration as needed for infrastructure metric collection.
5. Cross-check in Portal
   • Once configurations are corrected, validate that metrics for all Kubernetes components are visible in the OpsRamp portal.

Infrastructure Metrics Troubleshooting

Common Scenarios & Impacts

Debugging Steps

1. Check for Syntax Errors

   kubectl exec -it <worker-pod-name> -- sh
   cd /var/log/opsramp
   cat agent.log | grep "configuration error"

   Example:

   error decoding 'collection_interval': expected type 'time.Duration',
   got type 'bool', value: 'true'

2. Correct the config values (e.g., use proper duration format).
3. Validate Pipelines
   • Once corrected, logs should list all Receivers, Exporters, Processors, Pipelines.
   • If missing → configuration issue.