Release Qualification Test

This dashboard shows results of release qualification pipeline

To view the monitor/alert definition, checkout PrometheusRule list

Test result for 1.8.0-alpha.3-2020110913

Test Info
ProjectID ClusterName Branch/SHA/Tag PrometheusLink GrafanaLink
istio-testing release-qual-18 1.9-alpha.3ca55cf96c3cafd932176bf20264c79ab1f57212 http://prom.release-qual-18.qualistio.org http://grafana.release-qual-18.qualistio.org

Monitor Status
Monitor Status UpdatedTime FiredTimes LastFiredTime Description
HTTP5xxRateHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 The HTTP 5xx errors rate higher than 0.05 in 5 mins
Ingress200RateLow HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 1 The expected rate is 100 per ns, the limit is set based on 15ns
IngressLatencyP99High PENDING 2021-03-07 10:49:43.695255338 +0000 UTC 121 The ingress latency P99 > 250ms
IngressMemoryUsageIncreaseRateHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Ingress proxy Memory Usage increases more than 200 Bytes/sec
IngressTrafficMissing HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 ingress gateway traffic missing, check client logs
IstiodContainerCPUUsageHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Isitod Container CPU usage is above 80%
IstiodContainerNotReady HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 container: discovery not running
IstiodMemoryUsageHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Istiod Container Memory usage is above 80%
IstiodMemoryUsageIncreaseRateHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 26 Istiod Container Memory usage increases more than 1k Bytes/sec
IstiodUnavailableReplica HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Istiod unavailable replica > 0
IstiodxdsPushErrorsHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 istiod push error rate is higher than 0.05
IstiodxdsRejectHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 istiod rejects rate is higher than 0.05
ProxyContainerCPUUsageHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Proxy Container CPU usage is above 80%
ProxyContainerMemoryUsageHigh HEALTHY 2021-03-07 10:49:43.695255338 +0000 UTC 0 Proxy Container Memory usage is above 80%
WorkloadLatencyP99High PENDING 2021-03-07 10:49:43.695255338 +0000 UTC 2 The workload request latency P99 > 160ms

Test result for 1.9.0-2021020515

Test Info
ProjectID ClusterName Branch/SHA/Tag PrometheusLink GrafanaLink
istio-testing release-qual-19 1.9.0 http://prom.release-qual-19.qualistio.org http://grafana.release-qual-19.qualistio.org

Monitor Status
Monitor Status UpdatedTime FiredTimes LastFiredTime Description
HTTP5xxRateHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 The HTTP 5xx errors rate higher than 0.05 in 5 mins
Ingress200RateLow HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 169 The expected rate is 100 per ns, the limit is set based on 15ns
IngressLatencyP99High HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 The ingress latency P99 > 250ms
IngressMemoryUsageIncreaseRateHigh ALERTING 2021-03-07 10:54:27.981418004 +0000 UTC 15 2021-03-07T10:54:27.981418004Z Ingress proxy Memory Usage increases more than 200 Bytes/sec
IngressTrafficMissing ALERTING 2021-03-07 10:54:27.981418004 +0000 UTC 1584 2021-03-07T10:54:27.981418004Z ingress gateway traffic missing, check client logs
IstiodContainerCPUUsageHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 Isitod Container CPU usage is above 80%
IstiodContainerNotReady HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 container: discovery not running
IstiodMemoryUsageHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 Istiod Container Memory usage is above 80%
IstiodMemoryUsageIncreaseRateHigh PENDING 2021-03-07 10:54:27.981418004 +0000 UTC 21 Istiod Container Memory usage increases more than 1k Bytes/sec
IstiodUnavailableReplica HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 Istiod unavailable replica > 0
IstiodxdsPushErrorsHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 istiod push error rate is higher than 0.05
IstiodxdsRejectHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 istiod rejects rate is higher than 0.05
ProxyContainerCPUUsageHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 Proxy Container CPU usage is above 80%
ProxyContainerMemoryUsageHigh HEALTHY 2021-03-07 10:54:27.981418004 +0000 UTC 0 Proxy Container Memory usage is above 80%
WorkloadLatencyP99High PENDING 2021-03-07 10:54:27.981418004 +0000 UTC 0 The workload request latency P99 > 160ms

Test result for master-20200921

Test Info
ProjectID ClusterName Branch/SHA/Tag PrometheusLink GrafanaLink
istio-testing release-qual master-2 http://prom. http://grafana.

Monitor Status
Monitor Status UpdatedTime FiredTimes LastFiredTime Description
HTTP5xxRateHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 The HTTP 5xx errors rate higher than 0.05 in 5 mins
Ingress200RateLow HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 The expected rate is 100 per ns, the limit is set based on 15ns
IstiodContainerCPUUsageHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 Isitod Container CPU usage is above 80%
IstiodContainerNotReady HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 container: discovery not running
IstiodMemoryUsageHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 Istiod Container Memory usage is above 80%
IstiodUnavailableReplica HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 Istiod unavailable replica > 0
IstiodxdsPushErrorsHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 istiod push error rate is higher than 0.05
IstiodxdsRejectHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 istiod rejects rate is higher than 0.05
LatencyHigh PENDING 2021-03-07 10:59:48.295824156 +0000 UTC 0 The workload request latency > 30s
ProxyContainerCPUUsageHigh ALERTING 2021-03-07 10:59:48.295824156 +0000 UTC 223 2021-03-07T10:59:48.295824156Z Proxy Container CPU usage is above 80%
ProxyContainerMemoryUsageHigh HEALTHY 2021-03-07 10:59:48.295824156 +0000 UTC 0 Proxy Container Memory usage is above 80%