The pages in this Configuration section describe most available options for
managing and customizing your Kiali installation.
Unless noted, it is assumed that you are using the Kiali operator and that you
are managing the Kiali installation through a Kiali CR. The provided YAML
snippets for configuring Kiali should be placed in your Kiali CR. For example,
the provided configuration snippet for setting up the Anonymous authentication
strategy is the following:
spec:auth:strategy:anonymous
You will need to take this YAML snippet and apply it to your Kiali CR. As an example, an almost minimal Kiali CR using the previous configuration snippet would be the following:
Read the dedicated page of each authentication strategy to learn more.
1.1 - Anonymous strategy
Access Kiali with no authentication.
Introduction
The anonymous strategy removes any authentication requirement. Users will
have access to Kiali without providing any credentials.
Although the anonymous strategy doesn’t provide any access protection, it’s
valid for some use-cases. Some examples known from the community:
Exposing Kiali through a reverse proxy, where the reverse proxy is providing a custom authentication mechanism.
Exposing Kiali on an already limited network of trusted users.
When Kiali is accessed through kubectl port-forward or alike commands that allow usage of the cluster’s RBAC capabilities to limit access.
When developing Kiali, where a developer has a private instance on his own machine.
It’s worth to emphasize that the anonymous
strategy will leave Kiali unsecured. If you are using this option, make sure
that Kiali is available only to trusted users, or access is protected by other
means.
Set-up
To use the anonymous strategy, use the following configuration in the Kiali CR:
spec:auth:strategy:anonymous
The anonymous strategy doesn’t have any additional configuration.
Access control
When using the anonymous strategy, the content displayed in Kiali is based on
the permissions of the Kiali service account. By default, the Kiali service
account has cluster wide access and will be able to display everything in the
cluster.
OpenShift
If you are running Kiali in OpenShift, access can be customized by changing
privileges to the Kiali ServiceAccount. For example, to reduce permissions to
individual namespaces, first, remove the cluster-wide permissions granted by
default:
oc delete clusterrolebindings kiali
Then grant the kiali role only in needed namespaces. For example:
You can tell the Kiali Operator to install Kiali in “view only”
mode (this does work for either OpenShift or Kubernetes). You do this by
setting the view_only_mode to true in the Kiali CR, which
allows Kiali to read service mesh resources found in the cluster, but it does
not allow any change:
spec:deployment:view_only_mode:true
1.2 - Header strategy
Run Kiali behind a reverse proxy responsible for injecting the user’s token, or a token with impersonation.
Introduction
The header strategy assumes a reverse proxy is in front of Kiali, such as
OpenUnison or OAuth2 Proxy, injecting the user’s identity into each request to
Kiali as an Authorization header. This token can be an OpenID Connect
token or any other token the cluster recognizes. All requests to Kubernetes
will be made with this token, allowing Kiali to use the user’s own RBAC
context.
In addition to a user token, the header strategy supports impersonation
headers. If the impersonation headers are present in the request, then Kiali
will act on behalf of the user specified by the impersonation (assuming the
token supplied in the Authorization header is authorized to do so).
The header strategy will work with any Kubernetes cluster. The token provided
must be supported by that cluster. For instance, most “on-prem” clusters support
OpenID Connect, but cloud hosted clusters do not. For clusters that don’t support
a token, the impersonation
headers can be injected by the reverse proxy.
spec:auth:strategy:header
The header strategy doesn’t have any additional configuration.
HTTP Header
The header strategy looks for a token in the Authorization HTTP header with the
Bearer prefix. The HTTP header should look like:
Authorization: Bearer TOKEN
Where TOKEN is the appropriate token for your cluster. This TOKEN will be
submitted to the API server via a TokenReview to validate the token ONLY
on the first access to Kiali. On subsequent calls the TOKEN is passed through
directly to the API server.
Security Considerations
Network Policies
A policy should be put in place to make sure that the only “client” for Kiali is
the authenticating reverse proxy. This helps limit potential abuse and ensures
that the authenticating reverse proxy is the source of truth for who accessed
Kiali.
Short Lived Tokens
The authenticating reverse proxy should inject a short lived token in the
Authorization header. A shorter lived token is less likely to be abused if
leaked. Kiali will take whatever token is passed into the reqeuest, so as tokens
are regenerated Kiali will use the new token.
Impersonation
TokenRequest API
The authenticating reverse proxy should use the TokenRequest API instead of static
ServiceAccount tokens when possible while using impersonation. The
ServiceAccount that can impersonate users and groups is privileged and having it
be short lived cuts down on the possibility of a token being leaked while it’s being
passed between different parts of the infrastructure.
Drop Incoming Impersonation Headers
The authenticating proxy MUST drop any headers it receives from a remote client
that match the impersonation headers. Not only do you want to make sure that the
authenticating proxy can’t be overriden on which user to authenticate, but also
what groups they’re a member of.
1.3 - OpenID Connect strategy
Access Kiali requiring authentication through a third-party OpenID Connect provider.
Introduction
The openid authentication strategy lets you integrate Kiali to an external
identity provider that implements OpenID Connect, and allows
users to login to Kiali using their existing accounts of a
third-party system.
Currently, Kiali supports the authorization code flow (preferred) and the
implicit flow of the OpenId Connect spec.
Requirements
If you want to enable usage of the OpenId’s authorization code flow, make
sure that the
Kiali’s signing key
is 16, 24 or 32 byte long. If you setup a signing key of a
different size, Kiali will only be capable of using the implicit flow. If you
install Kiali via the operator and don’t set a custom signing key, the operator
should create a 16 byte long signing key.
We recommend using the authorization code flow.
If you don’t need RBAC support, the only requirement is to have a
working OpenId Server where Kiali can be configured as a client application.
A replacement or reverse proxy for the Kubernetes cluster API capable of handling the OIDC authentication.
The first option is preferred if you can manipulate your cluster API server
startup flags, which will result in your cluster to also be integrated with the
external OpenID provider.
The second option is provided for cases where you are using a managed
Kubernetes and your cloud provider does not support configuring OpenID
integration. Kiali assumes an implementation of a Kubernetes API server. For
example, a community user has reported to successfully configure Kiali’s OpenID
strategy by using
kube-oidc-proxy which is a
reverse proxy that handles the OpenID authentication and forwards the
authenticated requests to the Kubernetes API.
Set-up with RBAC support
Assuming you already have a working Kubernetes cluster with OpenId integration
(or a working alternative like kube-oidc-proxy), you should already had
configured an application or a client in your OpenId server (some cloud
providers configure this app/client automatically for you). You must re-use
this existing application/client by adding the root path of your Kiali
instance as an allowed/authorized callback URL. If the OpenID server provided
you a client secret for the application/client, or if you had manually set a
client secret, issue the following command to create a Kubernetes secret
holding the OpenId client secret:
where $NAMESPACE is the namespace where you installed Kiali and
$CLIENT_SECRET is the secret you configured or provided by your OpenId
Server. If Kiali is already running, you may need to restart the Kiali pod so
that the secret is mounted in Kiali.
This secret is only needed if you want Kiali to
use the authorization code flow (i.e. if your Kiali’s signing key is neither
16, 24 or 32 byte long).
It’s worth emphasizing that to configure OpenID
integration you must re-use the OpenID application/client that you created for
your Kubernetes cluster. If you create a new application/client for Kiali in
your OpenId server, Kiali will fail to properly authenticate users.
Then, to enable the OpenID Connect strategy, the minimal configuration you need to
set in the Kiali CR is like the following:
This assumes that your Kubernetes cluster is configured with OpenID Connect
integration. In this case, the client-id and issuer_uri attributes must
match the --oidc-client-id and --oidc-issuer-url flags
used to start the cluster API server.
If these values don’t match, users will fail to login to Kiali.
If you are using a replacement or a reverse proxy for the Kubernetes API
server, the minimal configuration is like the following:
The value of client-id and issuer_uri must match the values of the
configuration of your reverse proxy or cluster API replacement. The api_proxy
attribute is the URI of the reverse proxy or cluster API replacement (only
HTTPS is allowed). The api_proxy_ca_data is the public certificate authority
file encoded in a base64 string, to trust the secure connection.
Set-up with no RBAC support
Register Kiali as a client application in your OpenId Server. Use the root path
of your Kiali instance as the callback URL. If the OpenId Server provides you a
client secret, or if you manually set a client secret, issue the following
command to create a Kubernetes secret holding the OpenId client secret:
where $NAMESPACE is the namespace where you installed Kiali and
$CLIENT_SECRET is the secret you configured or provided by your OpenId
Server. If Kiali is already running, you may need to restart the Kiali pod so
that the secret is mounted in Kiali.
This secret is only needed if you want Kiali to
use the authorization code flow (i.e. if your Kiali’s signing key is neither
16, 24 or 32 byte long).
Then, to enable the OpenID Connect strategy, the minimal configuration you need
to set in the Kiali CR is like the following:
As RBAC is disabled, all users logging into Kiali
will share the same cluster-wide privileges.
Additional configurations
Configuring the displayed user name
The Kiali front-end will, by default, retrieve the string of the sub claim of
the OpenID token and display it as the user name. You can customize which field
to display as the user name by setting the username_claim attribute of the
Kiali CR. For example:
spec:auth:openid:username_claim:"email"
If you enabled RBAC, you will want the username_claim attribute to match the
--oidc-username-claim flag used to start the Kubernetes API server, or the
equivalent option if you are using a replacement or reverse proxy of the API
server. Else, any user-friendly claim will be OK as it is purely informational.
Configuring requested scopes
By default, Kiali will request access to the openid, profile and email
standard scopes. If you need a different set of scopes, you can set the
scopes attribute in the Kiali CR. For example:
The openid scope is forced. If you don’t add it to the list of scopes to
request, Kiali will still request it from the identity provider.
Configuring authentication timeout
When the user is redirected to the external authentication system, by default
Kiali will wait at most 5 minutes for the user to authenticate. After that time
has elapsed, Kiali will reject authentication. You can adjust this timeout by
setting the authentication_timeout with the number of seconds that Kiali
should wait at most. For example:
spec:auth:openid:authentication_timeout:60# Wait only one minute.
Configuring allowed domains
Some identity providers use a shared login and regardless of configuring your
own application under your domain (or organization account), login can succeed
even if the user that is logging in does not belong to your account or
organization. Google is an example of this kind of provider.
To prevent foreign users from logging into your Kiali instance, you can
configure a list of allowed domains:
The e-mail reported by the identity provider is used for the validation. Login
will be allowed if the domain part of the e-mail is listed as an allowed
domain; else, the user will be rejected. Naturally, you will need to
configure the email scope to be requested.
There is a special case: some identity providers include a hd claim in the
id_token. If this claim is present, this is used instead of extracting the
domain from the user e-mail. For example, Google Workspace (aka G Suite)
includes this hd claim for hosted
domains.
Using an OpenID provider with a self-signed certificate
If your OpenID provider is using a self-signed certificate, you can disable
certificate validation by setting the insecure_skip_verify_tls to true in
the Kiali CR:
spec:auth:openid:insecure_skip_verify_tls:true
You should use self-signed certificates only for
testing purposes.
However, if your organization or internal network has an internal trusted
certificate authority (CA), and your OpenID server is using a certificate
issued by this CA, you can configure Kiali to trust certificates from this CA,
rather than disabling verification. For this, create a ConfigMap named
kiali-cabundle containing the root CA certificate (the public component)
under the openid-server-ca.crt key:
apiVersion:v1kind:ConfigMapmetadata:name:kiali-cabundlenamespace:istio-system # This is Kiali's install namespacedata:openid-server-ca.crt:<the public component of your CA root certificate encoded in base64>
After restarting the Kiali pod, Kiali will trust this root certificate for all
HTTPS requests related to OpenID authentication.
Using an HTTP/HTTPS Proxy
In some network configurations, there is the need to use proxies to connect to
the outside world. OpenID requires outside world connections to get metadata and
do key validation, so you can configure it by setting the http_proxy and
https_proxy keys in the Kiali CR. They use the same format as the HTTP_PROXY
and HTTPS_PROXY environment variables.
Passing additional options to the identity provider
When users click on the Login button on Kiali, a redirection occurs to the
authentication page of the external identity provider. Kiali sends a fixed set
of parameters to the identity provider to enable authentication. If you need to
add an additional set of parameters to your identity provider, you can use the
additional_request_params setting of the Kiali CR, which accepts key-value
pairs. For example:
The prompt parameter is a
standard OpenID parameter.
When the login value is passed in this parameter, the
identity provider is instructed to ask for user credentials regardless if the
user already has an active session because of a previous login in some other
system.
If your OpenId provider supports other non-standard parameters, you can specify
the ones you need in this additional_request_params setting.
Take into account that you should not add the client_id, response_type,
redirect_uri, scope, nonce nor state parameters to this list. These are
already in use by Kiali and some already have a dedicated setting.
Provider-specific instructions
Using with Keycloak
When using OpenId with Keycloak, you will need to enable the Standard Flow Enabled
option on the Client (in the Administration Console):
The Standard Flow described on the options is the same as the authorization
code flow from the rest of the documentation.
If you get an error like Client is not allowed to initiate browser login with given response_type. Implicit flow is disabled for the client., it means that
your signing key for Kiali is not a standard size (16, 24 or 32 bytes long).
Enabling the Implicit Flow Enabled option of the client will make the problem
go away, but be aware that the implicit flow is less secure, and not
recommended.
Using with Google Cloud Platform / GKE OAuth2
If you are using Google Cloud Platform (GCP) and its products such as
Google Kubernetes Engine (GKE), it should be straightforward to configure Kiali’s OpenID
strategy to authenticate using your Google credentials.
First, you’ll need to go to your GCP Project and to the Credentials screen which
is available at (Menu Icon) > APIs & Services > Credentials.
On the Credentials screen you can select to create a new OAuth client ID.
If you’ve never setup the OAuth consent screen you will need to
do that before you can create an OAuth client ID. On screen you’ll have multiple
warnings and prompts to walk you through this.
On the Create OAuth client ID screen, set the Application type to Web Application
and enter a name for your key.
Then enter in the Authorized Javascript origins and Authorized redirect URIs for your project.
You can enter in localhost as appropriate during testing. You can also enter multiple URIs as appropriate.
After clicking Create you’ll be shown your newly minted client id and secret. These are important
and needed for your Kiali CR yaml and Kiali secrets files.
You’ll need to update your Kiali CR file to include the following auth block.
spec:auth:strategy:"openid"openid:client_id:"<your client id from GCP>"disable_rbac:trueissuer_uri:"https://accounts.google.com"scopes:["openid","email"]username_claim:"email"
Don’t get creative here. The issuer_uri should be https://accounts.google.com.
Finally you will need to create a secret, if you don’t have one already, that sets the oidc-secret for the openid flow.
apiVersion:v1kind:Secretmetadata:name:kialinamespace:istio-systemlabels:app:kialitype:Opaquedata:oidc-secret:"<base64 encode your client secret from GCP and enter here>"
Once all these settings are complete just set your Kiali CR and the Kiali secret to your cluster. You may need to
refresh your Kiali Pod to set the Secret if you add the Secret after the Kiali pod is created.
Using with Azure: AKS and AAD
The OpenID authentication strategy can be used
with Azure Kubernetes Service (AKS) and Azure Active Directory (AAD) with Kiali
versions 1.33 and later. Prior Kiali versions do not support RBAC on Azure.
AKS has support for a feature named AKS-managed Azure Active Directory, which
enables integration between AKS and AAD. This has the advantage that users can
use their AAD credentials to access AKS clusters and can also use Kubernetes
RBAC features to assign privileges to AAD users.
First, enable the AAD integration on your AKS cluster. See the
official AKS documentation to learn how.
Once it is enabled, your AKS panel should show the following:
Create a web application for Kiali in your Azure AD panel:
Go to AAD > App Registration, create an application with a redirect url like \https://<your-kiali-url>
Go to Certificates & secrets and create a client secret.
After creating the client secret, take note of the provided secret. Create a
Kubernetes secret in your cluster as mentioned in the Set-up
with RBAC support section. Please, note that the suggested name for the
Kubernetes Secret is kiali. If you want to customize the secret name, you
will have to specify your custom name in the Kiali CR. See: secret_name in Kial CR Reference.
Go to API Permissions and press the Add a permission button. In the new page that appears, switch to the
APIs my organization uses tab.
Type the following ID in the search field:
6dae42f8-4368-4678-94ff-3960e28e3630 (this is a shared ID for all Azure
clusters). And select the resulting entry.
Select the Delegated permissions square.
Select the user.read permission.
Go to Authentication and make sure that the Access tokens checkbox is ticked.
Then, create or modify your Kiali CR and include the following settings:
spec:auth:strategy:"openid"openid:client_id:"<your Kiali application client id from Azure>"issuer_uri:"https://sts.windows.net/<your AAD tenant id>/"username_claim:preferred_usernameapi_token:access_tokenadditional_request_params:resource:"6dae42f8-4368-4678-94ff-3960e28e3630"
You can find your client_id and tenant_id in the Overview page of the Kiali
App registration that you just created. See this documentation for more information.
1.4 - OpenShift strategy
Access Kiali requiring OpenShift authentication.
Introduction
The openshift authentication strategy is the preferred and default strategy
when Kiali is deployed on an OpenShift cluster.
When using the openshift strategy, a user logging into Kiali will be
redirected to the login page of the OpenShift console. Once the user provides
his OpenShift credentials, he will be redireted back to Kiali and will be
logged in if the user has enough privileges.
Since openshift is the default strategy when deploying Kiali in OpenShift,
you shouldn’t need to configure anything. If you want to be verbose, use the
following configuration in the Kiali CR:
spec:auth:strategy:openshift
The openshift strategy doesn’t have any additional configuration. The Kiali
operator will make sure to setup the needed OpenShift OAuth resources to register
Kiali as a client.
1.5 - Token strategy
Access Kiali requiring a Kubernetes ServiceAccount token.
Introduction
The token authentication strategy allows a user to login to Kiali using the
token of a Kubernetes ServiceAccount. This is similar to the
login view of Kubernetes Dashboard.
Since token is the default strategy when deploying Kiali in Kubernetes, you
shouldn’t need to configure anything, unless your cluster is OpenShift. If you
want to be verbose or if you need to enable the token strategy in OpenShift,
use the following configuration in the Kiali CR:
spec:auth:strategy:token
The token strategy doesn’t have any additional configuration other than the
session expiration time.
1.6 - Session options
Session timeout and signing key configuration
There are two settings that are available for the user’s session. The first one
is the session expiration time, which is only applicable to
token and header
authentication strategies:
spec:login_token:# By default, users session expires in 24 hours.expiration_seconds:86400
The session expiration time is the amount of time before the user is asked to
extend his session by another cycle. It does not matter if the user is actively
using Kiali, the user will be asked if the session should be extended.
The second available option is the signing key configuration, which is unset by
default, meaning that a random 16-character signing key will be generated
and stored to a secret named kiali-signing-key, in Kiali’s installation
namespace:
spec:login_token:# By default, create a random signing key and store it in# a secret named "kiali-signing-key".signing_key:""
If the secret already exists (which may mean a previous Kiali installation was
present), then the secret is reused.
The signing key is used on security sensitive data. For example, one of the
usages is to sign HTTP cookies related to the user session to prevent session
forgery.
If you need to set a custom fixed key, you can pre-create or modify the
kiali-signing-key secret:
apiVersion:v1kind:Secretmetadata:namespace:"kiali-installation-namespace"name:kiali-signing-keytype:Opaquedata:key:"<your signing key encoded in base64>"
The signing key must be 16, 24 or 32 bytes length. Otherwise, Kiali will fail to start.
If you prefer a different secret name for the signing key and/or a different
key-value pair of the secret, you can specify your preferred names in the Kiali
CR:
It is possible to specify the signing key directly in the Kiali CR, in the
spec.login_token.signing_key attribute. However, this should be only for
testing purposes. The signing key is sensitive and should be treated like a
password that must be protected.
2 - Console Customization
Default selections, find and hide presets and custom metric aggregations.
Custom metric aggregations
The inbound and outbound metric pages, in the Metrics settings drop-down,
provides an opinionated set of groupings that work both for filtering out
metric data that does not match the selection and for aggregating data into
series. Each option is backed by a label on the collected Istio telemetry.
It is possible to add custom aggregations, like in the following example:
Most Kiali pages show metrics per refresh and refresh interval
drop-downs. These are located at the top-right of the page.
Metrics per refresh specifies the time range back from the current
instant to fetch metrics and/or distributed tracing data. By default, a
1-minute time range is selected.
Refresh interval specifies how often Kiali will automatically refresh the
data shown. By default, Kiali refreshes data every 15 seconds.
By default, when Kiali is accessed by the first time, on most Kiali pages users
will need to use the namespace drop-down to choose namespaces they want to view
data from. The selection will be persisted on reloads.
However, it is possible to configure a predefined selection of
namespaces, like in the following example:
Namespace selection will reset to the predefined set on reloads. Also, if for
some reason a namespace becomes deleted, Kiali will simply ignore it from the
list.
Hopefully, the attributes to configure this feature are self-explanatory.
Note that by providing your own presets, you will be overriding the default
configuration. Make sure to include any default presets that you need in case
you provide your own configuration.
Graph default traffic rates
Traffic rates in the graph are fetched from Istio telemetry and there are
several metric sources
that can be used.
In the graph page, you can select the traffic rate metrics using the Traffic
drop-down (next to the Namespaces drop-down). By default, Requests is
selected for GRPC and HTTP protocols, and Sent bytes is selected for the TCP
protocol, but you can change the default selection:
spec:kiali_feature_flags:ui_defaults:graph:traffic:grpc:"requests"# Valid values: none, requests, sent, received and totalhttp:"requests"# Valid values: none and requeststcp:"sent"# Valid values: none, sent, received and total
Note that only requests provide response codes and will allow health to be
calculated. Also, the resulting topology graph may be different for each source.
The name field corresponds to what you can set in the pod annotation kiali.io/dashboards.
The rest of the field definitions are:
runtime: optional, name of the related runtime. It will be displayed on the corresponding Workload Details page. If omitted no name is displayed.
title: dashboard title, displayed as a tab in Application or Workloads Details
discoverOn: metric name to match for auto-discovery. If omitted, the dashboard won’t be discovered automatically, but can still be used via pods annotation.
items: a list of items, that can be either chart, to define a new chart, or include to reference another dashboard
chart: new chart object
name: name of the chart
chartType: type of the chart, can be one of line (default), area, bar or scatter
unit: unit for Y-axis. Free-text field to provide any unit suffix. It can eventually be scaled on display. See specific section below.
unitScale: in case the unit needs to be scaled by some factor, set that factor here. For instance, if your data is in milliseconds, set 0.001 as scale and seconds as unit.
spans: number of “spans” taken by the chart, from 1 to 12, using bootstrap convention
metrics: a list of metrics to display on this single chart:
metricName: the metric name in Prometheus
displayName: name to display on chart
dataType: type of data to be displayed in the chart. Can be one of raw, rate or histogram. Raw data will be queried without transformation. Rate data will be queried using
promQL rate() function. And histogram with histogram_quantile() function.
min and max: domain for Y-values. When unset, charts implementations should usually automatically adapt the domain with the displayed data.
xAxis: type of the X-axis, can be one of time (default) or series. When set to series, only one datapoint per series will be displayed, and the chart type then defaults to bar.
aggregator: defines how the time-series are aggregated when several are returned for a given metric and label set. For example, if a Deployment creates a ReplicaSet of several Pods, you will have at least one time-series per Pod. Since Kiali shows the dashboards at the workload (ReplicaSet) level or at the application level, they will have to be aggregated. This field can be used to fix the aggregator, with values such as sum or avg (full list available in Prometheus documentation). However, if omitted the aggregator will default to sum and can be changed from the dashboard UI.
aggregations: list of labels eligible for aggregations / groupings (they will be displayed in Kiali through a dropdown list)
label: Prometheus label name
displayName: name to display in Kiali
singleSelection: boolean flag to switch between single-selection and multi-selection modes on the values of this label. Defaults to false.
groupLabels: a list of Prometheus labels to be used for grouping. Similar to aggregations, except this grouping will be always turned on.
sortLabel: Prometheus label to be used for the metrics display order.
sortLabelParseAs: set to int if sortLabel needs to be parsed and compared as an integer instead of string.
include: to include another dashboard, or a specific chart from another dashboard. Typically used to compose with generic dashboards such as the ones about MicroProfile Metrics or Micrometer-based JVM metrics. To reference a full dashboard, set the name of that dashboard. To reference a specific chart of another dashboard, set the name of the dashboard followed by $ and the name of the chart (ex: include: "microprofile-1.1$Thread count").
externalLinks: a list of related external links (e.g. to Grafana dashboards)
name: name of the related dashboard in the external system (e.g. name of a Grafana dashboard)
type: link type, currently only grafana is allowed
variables: a set of variables that can be injected in the URL. For instance, with something like namespace: var-namespace and app: var-app, an URL to a Grafana dashboard that manages namespace and app variables would look like:
http://grafana-server:3000/d/xyz/my-grafana-dashboard?var-namespace=some-namespace&var-app=some-app. The available variables in this context are namespace, app and version.
Label clash: you should try to avoid labels clashes within a dashboard.
In Kiali, labels for grouping are aggregated in the top toolbar, so if the same label refers to different things depending on the metric, you wouldn’t be able to distinguish them in the UI. For that reason, ideally, labels should not have too generic names in Prometheus.
For instance labels named “id” for both memory spaces and buffer pools would better be named “space_id” and “pool_id”. If you have control on label names, it’s an important aspect to take into consideration.
Else, it is up to you to organize dashboards with that in mind, eventually splitting them into smaller ones to resolve clashes.
Modifying Built-in Dashboards: If you want to modify or remove a built-in dashboard, you can set its new definition in the Kiali CR’s spec.custom_dashboards. Simply define a custom dashboard with the same name as the built-in dashboard. To remove a built-in dashboard so Kiali doesn’t use it, simply define a custom dashboard by defining only its name with no other data associated with it (e.g. in spec.custom_dashboards you add a list item that has - name: <name of built-in dashboard to remove>.
Dashboard scope
The custom dashboards defined in the Kiali CR are available for all workloads in all namespaces.
Additionally, new custom dashboards can be created for a given namespace or workload, using the dashboards.kiali.io/templates annotation.
This is an example where a “Custom Envoy” dashboard will be available for all applications and workloads for the default namespace:
Some units are recognized in Kiali and scaled appropriately when displayed on charts:
unit: "seconds" can be scaled down to ms, µs, etc.
unit: "bytes-si" and unit: "bitrate-si" can be scaled up to kB, MB (etc.) using SI / metric system. The aliases unit: "bytes" and unit: "bitrate" can be used instead.
unit: "bytes-iec" and unit: "bitrate-iec" can be scaled up to KiB, MiB (etc.) using IEC standard / IEEE 1541-2002 (scale by powers of 2).
Other units will fall into the default case and be scaled using SI standard. For instance, unit: "m" for meter can be scaled up to km.
Prometheus Configuration
Kiali custom dashboards work exclusively with Prometheus, so it must be configured correctly to pull your application metrics.
If you are using the demo Istio installation with addons, your Prometheus instance should already be correctly configured and you can skip to the next section; with the exception of Istio 1.6.x where
you need customize the ConfigMap, or install Istio with the flag--set meshConfig.enablePrometheusMerge=true.
Using another Prometheus instance
You can use a different instance of Prometheus for these metrics, as opposed to Istio metrics. This second Prometheus instance can be configured from the Kiali CR when using the Kiali operator, or ConfigMap otherwise:
You must make sure that this Prometheus instance is correctly configured to scrape your application pods and generates labels that Kiali will understand. Please refer to
this documentation to setup the kubernetes_sd_config section. As a reference,
here is how it is configured in Istio.
It is important to preserve label mapping, so that Kiali can filter by app and version, and to have the same namespace label as defined per Kiali config. Here’s a relabel_configs that allows this:
prometheus.io/scrape tells Prometheus to fetch these metrics or not
prometheus.io/port is the port under which metrics are exposed
prometheus.io/path is the endpoint path where metrics are exposed, default is /metrics
Kiali will try to discover automatically dashboards that are relevant for a given Application or Workload. To do so, it reads their metrics and try to match them with the discoverOn field defined on dashboards.
But if you can’t rely on automatic discovery, you can explicitly annotate the pods to associate them with Kiali dashboards.
kiali.io/dashboards is a comma-separated list of dashboard names that Kiali will look for. Each name in the list must match the name of a built-in dashboard or the name of a custom dashboard as defined in the Kial CR’s spec.custom_dashboards.
Istio recommends adding app and version labels to
pods to attach this information to telemetry. Kiali relies on correctness of these labels for several features.
In Istio, it is possible to use a different set of labels, like
app.kubernetes.io/name and app.kubernetes.io/version, however you must
configure Kiali to the labels you are using. By default, Kiali uses Istio’s
recommended labels:
Although Istio lets you use different labels on different pods, Kiali can only
use a single set.
For example, Istio lets you use the app label in one pod and the
app.kubernetes.io/name in another pod and it will generate telemetry
correctly. However, you will have no way to configure Kiali for this case.
Monitoring port of the IstioD pod
Kiali connects directly to the IstioD pod (not the Service) to check for its
health. By default, the connection is done to port 15014 which is the default
monitoring port of the IstioD pod.
Under some circumstances, you may need to change the monitoring port of the
IstioD pod to something else. For example, when running IstioD in host
network mode
the network is shared between several pods, requiring to change listening ports
of some pods to prevent conflicts.
It is possible to map the newly chosen monitoring port of the IstioD pod in the
related Service to let other services continue working normally. However, since
Kiali connects directly to the IstioD pod, you need to configure the assigned
monitoring port in the Kiali CR:
Istio’s root namespace is the namespace where you can create some resources
to define default Istio configurations and adapt Istio behavior to your
environment. For more information on this Istio configuration, check the Istio
docs Global Mesh options
page and
search for “rootNamespace”.
Kiali uses the root namespace for some of the validations of Istio resources.
If you customized the Istio root namespace, you will need to replicate that
configuration in Kiali. By default, it is unset:
spec:external_services:istio:root_namespace:""
Sidecar injection, canary upgrade management and Istio revisions
Assistance for migrating workloads between Istio revisions when doing a canary
upgrade is turned off by default. This is because it is required to know what
is the revision name that was used when
installing
each Istio control plane. You can enable and configure the canary upgrade
support with the following configuration:
spec:external_services:istio:istio_canary_revision:# Revision string of old Istio versioncurrent:"1-10-3"# Revision string of new Istio versionupgrade:"1-11-0"kiali_feature_flags:# Turns on canary upgrade supportistio_upgrade_action:true
Please note that Kiali will use revision labels to control sidecar injection
policy only when canary upgrade support is enabled; else, non-revision labels
are used. Make sure you finish the canary upgrade before turning off the canary
upgrade support. If you need to disable Kiali’s canary upgrade feature while an
upgrade is unfinished, it will be safer if you also disable the sidecar
injection management feature.
It is important to note that canary upgrades require adding a revision name
during the installation of control planes. You will notice that the revision
name will be appended to the name of Istio resources. Thus, once/if you are
using Kiali with an Istio control plane that has a revision name you will need
to specify what is the name of a few Istio resources that Kiali uses. For
example, if your control plane has a revision name 1-11-0 you would need to
set these configurations:
Reference page for the Kiali CR.
The Kiali Operator will watch for resources of this type and install Kiali according to those resources' configurations.
Example CR
(all values shown here are the defaults unless otherwise noted)
apiVersion:kiali.io/v1alpha1kind:Kialimetadata:name:kialiannotations:ansible.sdk.operatorframework.io/verbosity:"1"spec:additional_display_details:- title:"API Documentation"annotation:"kiali.io/api-spec"icon_annotation:"kiali.io/api-type"installation_tag:""istio_namespace:""version:"default"api:namespaces:exclude:- "^istio-operator"- "^kube-.*"- "^openshift.*"- "^ibm.*"- "^kiali-operator"# default: label_selector is undefinedlabel_selector:"kiali.io/member-of=istio-system"auth:strategy:""openid:# default: additional_request_params is emptyadditional_request_params:openIdReqParam:"openIdReqParamValue"# default: allowed_domains is an empty listallowed_domains:["allowed.domain"]api_proxy:""api_proxy_ca_data:""api_token:"id_token"authentication_timeout:300authorization_endpoint:""client_id:""disable_rbac:falsehttp_proxy:""https_proxy:""insecure_skip_verify_tls:falseissuer_uri:""scopes:["openid","profile","email"]username_claim:"sub"openshift:client_id_prefix:"kiali"# default: custom_dashboards is an empty listcustom_dashboards:- name:"envoy"deployment:accessible_namespaces:["^((?!(istio-operator|kube-.*|openshift.*|ibm.*|kiali-operator)).)*$"]# default: additional_service_yaml is emptyadditional_service_yaml:externalName:"kiali.example.com"affinity:# default: node is emptynode:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key:kubernetes.io/e2e-az-nameoperator:Invalues:- e2e-az1- e2e-az2# default: pod is emptypod:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key:securityoperator:Invalues:- S1topologyKey:topology.kubernetes.io/zone# default: pod_anti is emptypod_anti:preferredDuringSchedulingIgnoredDuringExecution:- weight:100podAffinityTerm:labelSelector:matchExpressions:- key:securityoperator:Invalues:- S2topologyKey:topology.kubernetes.io/zone# default: custom_secrets is an empty listcustom_secrets:- name:"a-custom-secret"mount:"/a-custom-secret-path"optional:truehpa:api_version:"autoscaling/v2beta2"# default: spec is emptyspec:maxReplicas:2minReplicas:1targetCPUUtilizationPercentage:80# default: host_aliases is an empty listhost_aliases:- ip:"192.168.1.100"hostnames:- "foo.local"- "bar.local"image_digest:""image_name:""image_pull_policy:"IfNotPresent"# default: image_pull_secrets is an empty listimage_pull_secrets:["image.pull.secret"]image_version:""ingress:# default: additional_labels is emptyadditional_labels:ingressAdditionalLabel:"ingressAdditionalLabelValue"class_name:"nginx"# default: enabled is undefinedenabled:false# default: override_yaml is undefinedoverride_yaml:metadata:annotations:nginx.ingress.kubernetes.io/secure-backends:"true"nginx.ingress.kubernetes.io/backend-protocol:"HTTPS"spec:rules:- http:paths:- path:"/kiali"pathType:Prefixbackend:service:name:"kiali"port:number:20001instance_name:"kiali"logger:log_level:"info"log_format:"text"sampler_rate:"1"time_field_format:"2006-01-02T15:04:05Z07:00"namespace:"istio-system"# default: node_selector is emptynode_selector:nodeSelector:"nodeSelectorValue"# default: pod_annotations is emptypod_annotations:podAnnotation:"podAnnotationValue"# default: pod_labels is emptypod_labels:sidecar.istio.io/inject:"true"priority_class_name:""replicas:1# default: resources is undefinedresources:requests:cpu:"10m"memory:"64Mi"limits:memory:"1Gi"secret_name:"kiali"# default: service_annotations is emptyservice_annotations:svcAnnotation:"svcAnnotationValue"# default: service_type is undefinedservice_type:"NodePort"# default: tolerations is an empty listtolerations:- key:"example-key"operator:"Exists"effect:"NoSchedule"verbose_mode:"3"version_label:""view_only_mode:falseexternal_services:custom_dashboards:discovery_auto_threshold:10discovery_enabled:"auto"enabled:trueis_core:falsenamespace_label:"namespace"prometheus:auth:ca_file:""insecure_skip_verify:falsepassword:""token:""type:"none"use_kiali_token:falseusername:""cache_duration:10cache_enabled:truecache_expiration:300# default: custom_headers is emptycustom_headers:customHeader1:"customHeader1Value"health_check_url:""is_core:true# default: query_scope is emptyquery_scope:mesh_id:"mesh-1"cluster:"cluster-east"thanos_proxy:enabled:falseretention_period:"7d"scrape_interval:"30s"url:""grafana:auth:ca_file:""insecure_skip_verify:falsepassword:""token:""type:"none"use_kiali_token:falseusername:""dashboards:- name:"Istio Service Dashboard"variables:namespace:"var-namespace"service:"var-service"- name:"Istio Workload Dashboard"variables:namespace:"var-namespace"workload:"var-workload"- name:"Istio Mesh Dashboard"- name:"Istio Control Plane Dashboard"- name:"Istio Performance Dashboard"- name:"Istio Wasm Extension Dashboard"enabled:truehealth_check_url:""# default: in_cluster_url is undefinedin_cluster_url:""is_core:falseurl:""istio:component_status:components:- app_label:"istiod"is_core:trueis_proxy:false- app_label:"istio-ingressgateway"is_core:trueis_proxy:true# default: namespace is undefinednamespace:istio-system- app_label:"istio-egressgateway"is_core:falseis_proxy:true# default: namespace is undefinednamespace:istio-systemenabled:trueconfig_map_name:"istio"envoy_admin_local_port:15000# default: istio_canary_revision is undefinedistio_canary_revision:current:"1-9-9"upgrade:"1-10-2"istio_identity_domain:"svc.cluster.local"istio_injection_annotation:"sidecar.istio.io/inject"istio_sidecar_annotation:"sidecar.istio.io/status"istio_sidecar_injector_config_map_name:"istio-sidecar-injector"istiod_deployment_name:"istiod"istiod_pod_monitoring_port:15014root_namespace:""url_service_version:""prometheus:auth:ca_file:""insecure_skip_verify:falsepassword:""token:""type:"none"use_kiali_token:falseusername:""cache_duration:10cache_enabled:truecache_expiration:300# default: custom_headers is emptycustom_headers:customHeader1:"customHeader1Value"health_check_url:""is_core:true# default: query_scope is emptyquery_scope:mesh_id:"mesh-1"cluster:"cluster-east"thanos_proxy:enabled:falseretention_period:"7d"scrape_interval:"30s"url:""tracing:auth:ca_file:""insecure_skip_verify:falsepassword:""token:""type:"none"use_kiali_token:falseusername:""enabled:truein_cluster_url:""is_core:falsenamespace_selector:trueurl:""use_grpc:truewhitelist_istio_system:["jaeger-query","istio-ingressgateway"]health_config:# default: rate is an empty listrate:- namespace:".*"kind:".*"name:".*"tolerance:- protocol:"http"direction:".*"code:"[1234]00"degraded:5failure:10identity:# default: cert_file is undefinedcert_file:""# default: private_key_file is undefinedprivate_key_file:""istio_labels:app_label_name:"app"injection_label_name:"istio-injection"injection_label_rev:"istio.io/rev"version_label_name:"version"kiali_feature_flags:certificates_information_indicators:enabled:truesecrets:- "cacerts"- "istio-ca-secret"clustering:enabled:trueistio_injection_action:trueistio_upgrade_action:falseui_defaults:graph:find_options:- description:"Find: slow edges (> 1s)"expression:"rt > 1000"- description:"Find: unhealthy nodes"expression:"! healthy"- description:"Find: unknown nodes"expression:"name = unknown"hide_options:- description:"Hide: healthy nodes"expression:"healthy"- description:"Hide: unknown nodes"expression:"name = unknown"traffic:grpc:"requests"http:"requests"tcp:"sent"metrics_per_refresh:"1m"# default: metrics_inbound is undefinedmetrics_inbound:aggregations:- display_name:"Istio Network"label:"topology_istio_io_network"- display_name:"Istio Revision"label:"istio_io_rev"# default: metrics_outbound is undefinedmetrics_outbound:aggregations:- display_name:"Istio Network"label:"topology_istio_io_network"- display_name:"Istio Revision"label:"istio_io_rev"# default: namespaces is an empty listnamespaces:["istio-system"]refresh_interval:"15s"validations:ignore:["KIA1201"]kubernetes_config:burst:200cache_duration:300cache_enabled:truecache_istio_types:- "AuthorizationPolicy"- "DestinationRule"- "EnvoyFilter"- "Gateway"- "PeerAuthentication"- "RequestAuthentication"- "ServiceEntry"- "Sidecar"- "VirtualService"- "WorkloadEntry"- "WorkloadGroup"cache_namespaces:- ".*"cache_token_namespace_duration:10excluded_workloads:- "CronJob"- "DeploymentConfig"- "Job"- "ReplicationController"qps:175login_token:expiration_seconds:86400signing_key:""server:address:""audit_log:truecors_allow_all:falsegzip_enabled:trueobservability:metrics:enabled:trueport:9090tracing:collector_url:"http://jaeger-collector.istio-system:14268/api/traces"enabled:falseport:20001web_fqdn:""web_history_mode:""web_port:""web_root:""web_schema:""
Validating your Kiali CR
A Kiali tool is available to allow you to check your own Kiali CR to ensure it is valid. Simply download the validation script and run it, passing in the location of the Kiali CRD you wish to validate with (e.g. the latest version is found here) and the location of your Kiali CR. You must be connected to/logged into a cluster for this validation tool to work.
For example, to validate a Kiali CR named kiali in the namespace istio-system using the latest version of the Kiali CRD, run the following:
For additional help in using this validation tool, pass it the --help option.
Properties
.spec
(object)
This is the CRD for the resources called Kiali CRs. The Kiali Operator will watch for resources of this type and when it detects a Kiali CR has been added, deleted, or modified, it will install, uninstall, and update the associated Kiali Server installation. The settings here will configure the Kiali Server as well as the Kiali Operator. All of these settings will be stored in the Kiali ConfigMap. Do not modify the ConfigMap; it will be managed by the Kiali Operator. Only modify the Kiali CR when you want to change a configuration setting.
.spec.additional_display_details
(array)
A list of additional details that Kiali will look for in annotations. When found on any workload or service, Kiali will display the additional details in the respective workload or service details page. This is typically used to inject some CI metadata or documentation links into Kiali views. For example, by default, Kiali will recognize these annotations on a service or workload (e.g. a Deployment, StatefulSet, etc.):
Note that if you change this setting for your own custom annotations, keep in mind that it would override the current default. So you would have to add the default setting as shown in the example CR if you want to preserve the default links.
.spec.additional_display_details[*]
(object)
.spec.additional_display_details[*].annotation
(string)*Required*
The name of the annotation whose value is a URL to additional documentation useful to the user.
The name of the annotation whose value is used to determine what icon to display. The annotation name itself can be anything, but note that the value of that annotation must be one of: rest, grpc, and graphql - any other value is ignored.
.spec.additional_display_details[*].title
(string)*Required*
The title of the link that Kiali will display. The link will go to the URL specified in the value of the configured annotation.
.spec.api
(object)
.spec.api.namespaces
(object)
Settings that control what namespaces are returned by Kiali.
.spec.api.namespaces.exclude
(array)
A list of namespaces to be excluded from the list of namespaces provided by the Kiali API and Kiali UI. Regex is supported. This does not affect explicit namespace access.
.spec.api.namespaces.exclude[*]
(string)
.spec.api.namespaces.label_selector
(string)
A Kubernetes label selector (e.g. myLabel=myValue) which is used when fetching the list of
available namespaces. This does not affect explicit namespace access.
If deployment.accessible_namespaces does not have the special value of '**'
then the Kiali operator will add a new label to all accessible namespaces - that new
label will be this label_selector.
Note that if you do not set this label_selector setting but deployment.accessible_namespaces
does not have the special “all namespaces” entry of '**' then this label_selector will be set
to a default value of kiali.io/[<deployment.instance_name>.]member-of=<deployment.namespace>
where [<deployment.instance_name>.] is the instance name assigned to the Kiali installation
if it is not the default ‘kiali’ (otherwise, this is omitted) and <deployment.namespace>
is the namespace where Kiali is to be installed.
The Route resource name and OAuthClient resource name will have this value as its prefix. This value normally should never change. The installer will ensure this value is set correctly.
.spec.auth.strategy
(string)
Determines what authentication strategy to use when users log into Kiali.
Options are anonymous, token, openshift, openid, or header.
Choose anonymous to allow full access to Kiali without requiring any credentials.
Choose token to allow access to Kiali using service account tokens, which controls
access based on RBAC roles assigned to the service account.
Choose openshift to use the OpenShift OAuth login which controls access based on
the individual’s RBAC roles in OpenShift. Not valid for non-OpenShift environments.
Choose openid to enable OpenID Connect-based authentication. Your cluster is required to
be configured to accept the tokens issued by your IdP. There are additional required
configurations for this strategy. See below for the additional OpenID configuration section.
Choose header when Kiali is running behind a reverse proxy that will inject an
Authorization header and potentially impersonation headers.
When empty, this value will default to openshift on OpenShift and token on other Kubernetes environments.
.spec.custom_dashboards
(array)
A list of user-defined custom monitoring dashboards that you can use to generate metrics charts
for your applications. The server has some built-in dashboards; if you define a custom dashboard here
with the same name as a built-in dashboard, your custom dashboard takes precedence and will overwrite
the built-in dashboard. You can disable one or more of the built-in dashboards by simply defining an
empty dashboard.
An example of an additional user-defined dashboard,
A list of namespaces Kiali is to be given access to. These namespaces have service mesh components that are to be observed by Kiali. You can provide names using regex expressions matched against all namespaces the operator can see. The default makes all namespaces accessible except for some internal namespaces that typically should be ignored. NOTE! If this has an entry with the special value of '**' (two asterisks), that will denote you want Kiali to be given access to all namespaces via a single cluster role (if using this special value of '**', you are required to have already granted the operator permissions to create cluster roles and cluster role bindings).
.spec.deployment.accessible_namespaces[*]
(string)
.spec.deployment.additional_service_yaml
(object)
Additional custom yaml to add to the service definition. This is used mainly to customize the service type. For example, if the deployment.service_type is set to ‘LoadBalancer’ and you want to set the loadBalancerIP, you can do so here with: additional_service_yaml: { 'loadBalancerIP': '78.11.24.19' }. Another example would be if the deployment.service_type is set to ‘ExternalName’ you will need to configure the name via: additional_service_yaml: { 'externalName': 'my.kiali.example.com' }. A final example would be if external IPs need to be set: additional_service_yaml: { 'externalIPs': ['80.11.12.10'] }
.spec.deployment.affinity
(object)
Affinity definitions that are to be used to define the nodes where the Kiali pod should be constrained. See the Kubernetes documentation on Assigning Pods to Nodes for the proper syntax for these three different affinity types.
.spec.deployment.affinity.node
(object)
.spec.deployment.affinity.pod
(object)
.spec.deployment.affinity.pod_anti
(object)
.spec.deployment.custom_secrets
(array)
Defines additional secrets that are to be mounted in the Kiali pod.
These are useful to contain certs that are used by Kiali to securely connect to third party systems
(for example, see external_services.tracing.auth.ca_file).
These secrets must be created by an external mechanism. Kiali will not generate these secrets; it
is assumed these secrets are externally managed. You can define 0, 1, or more secrets.
An example configuration is,
The file path location where the secret content will be mounted. The custom secret cannot be mounted on a path that the operator will use to mount its secrets. Make sure you set your custom secret mount path to a unique, unused path. Paths such as /kiali-configuration, /kiali-cert, /kiali-cabundle, and /kiali-secret should not be used as mount paths for custom secrets because the operator may want to use one of those paths.
.spec.deployment.custom_secrets[*].name
(string)*Required*
The name of the secret that is to be mounted to the Kiali pod’s file system. The name of the custom secret must not be the same name as one created by the operator. Names such as kiali, kiali-cert-secret, and kiali-cabundle should not be used as a custom secret name because the operator may want to create one with one of those names.
.spec.deployment.custom_secrets[*].optional
(boolean)
Indicates if the secret may or may not exist at the time the Kiali pod starts. This will default to ‘false’ if not specified.
.spec.deployment.host_aliases
(array)
This is content for the Kubernetes ‘hostAliases’ setting for the Kiali server.
This allows you to modify the Kiali server pod ‘/etc/hosts’ file.
A typical way to configure this setting is,
A specific HPA API version that can be specified in case there is some HPA feature you want to use that is only supported in that specific version.
.spec.deployment.hpa.spec
(object)
The spec specified here will be placed in the created HPA resource’s ‘spec’ section. If spec is left empty, no HPA resource will be created. Note that you must not specify the ‘scaleTargetRef’ section in spec; the Kiali Operator will populate that for you.
.spec.deployment.image_digest
(string)
If deployment.image_version is a digest hash, this value indicates what type of digest it is. A typical value would be ‘sha256’. Note: do NOT prefix this value with a ‘@’.
.spec.deployment.image_name
(string)
Determines which Kiali image to download and install. If you set this to a specific name (i.e. you do not leave it as the default empty string), you must make sure that image is supported by the operator. If empty, the operator will use a known supported image name based on which version was defined. Note that, as a security measure, a cluster admin may have configured the Kiali operator to ignore this setting. A cluster admin may do this to ensure the Kiali operator only installs a single, specific Kiali version, thus this setting may have no effect depending on how the operator itself was configured.
.spec.deployment.image_pull_policy
(string)
The Kubernetes pull policy for the Kiali deployment. This is overridden to be ‘Always’ if deployment.image_version is set to ‘latest’.
.spec.deployment.image_pull_secrets
(array)
The names of the secrets to be used when container images are to be pulled.
.spec.deployment.image_pull_secrets[*]
(string)
.spec.deployment.image_version
(string)
Determines which version of Kiali to install.
Choose ‘lastrelease’ to use the last Kiali release.
Choose ‘latest’ to use the latest image (which may or may not be a released version of Kiali).
Choose ‘operator_version’ to use the image whose version is the same as the operator version.
Otherwise, you can set this to any valid Kiali version (such as ‘v1.0’) or any valid Kiali
digest hash (if you set this to a digest hash, you must indicate the digest in deployment.image_digest).
Note that if this is set to ‘latest’ then the deployment.image_pull_policy will be set to ‘Always’.
If you set this to a specific version (i.e. you do not leave it as the default empty string),
you must make sure that image is supported by the operator.
If empty, the operator will use a known supported image version based on which ‘version’ was defined.
Note that, as a security measure, a cluster admin may have configured the Kiali operator to
ignore this setting. A cluster admin may do this to ensure the Kiali operator only installs
a single, specific Kiali version, thus this setting may have no effect depending on how the
operator itself was configured.
.spec.deployment.ingress
(object)
Configures if/how the Kiali endpoint should be exposed externally.
.spec.deployment.ingress.additional_labels
(object)
Additional labels to add to the Ingress (or Route if on OpenShift). These are added to the labels that are created by default; these do not override the default labels.
.spec.deployment.ingress.class_name
(string)
If class_name is a non-empty string, it will be used as the ‘spec.ingressClassName’ in the created Kubernetes Ingress resource. This setting is ignored if on OpenShift. This is also ignored if override_yaml.spec is defined (i.e. you must define the ‘ingressClassName’ directly in your override yaml).
.spec.deployment.ingress.enabled
(boolean)
Determines if the Kiali endpoint should be exposed externally. If ‘true’, an Ingress will be created if on Kubernetes or a Route if on OpenShift. If left undefined, this will be ‘false’ on Kubernetes and ‘true’ on OpenShift.
.spec.deployment.ingress.override_yaml
(object)
Because an Ingress into a cluster can vary wildly in its desired configuration,
this setting provides a way to override complete portions of the Ingress resource
configuration (Ingress on Kubernetes and Route on OpenShift). It is up to the user
to ensure this override YAML configuration is valid and supports the cluster environment
since the operator will blindly copy this custom configuration into the resource it
creates.
This setting is not used if deployment.ingress.enabled is set to ‘false’.
Note that only ‘metadata.annotations’ and ‘spec’ is valid and only they will
be used to override those same sections in the created resource. You can define
either one or both.
Note that override_yaml.metadata.labels is not allowed - you cannot override the labels; to add
labels to the default set of labels, use the deployment.ingress.additional_labels setting.
Example,
The instance name of this Kiali installation. This instance name will be the prefix prepended to the names of all Kiali resources created by the operator and will be used to label those resources as belonging to this Kiali installation instance. You cannot change this instance name after a Kiali CR is created. If you attempt to change it, the operator will abort with an error. If you want to change it, you must first delete the original Kiali CR and create a new one. Note that this does not affect the name of the auto-generated signing key secret. If you do not supply a signing key, the operator will create one for you in a secret, but that secret will always be named ‘kiali-signing-key’ and shared across all Kiali instances in the same deployment namespace. If you want a different signing key secret, you are free to create your own and tell the operator about it via login_token.signing_key. See the docs on that setting for more details. Note also that if you are setting this value, you may also want to change the installation_tag setting, but this is not required.
.spec.deployment.logger
(object)
Configures the logger that emits messages to the Kiali server pod logs.
.spec.deployment.logger.log_format
(string)
Indicates if the logs should be written with one log message per line or using a JSON format. Must be one of: text or json.
.spec.deployment.logger.log_level
(string)
The lowest priority of messages to log. Must be one of: trace, debug, info, warn, error, or fatal.
.spec.deployment.logger.sampler_rate
(string)
With this setting every sampler_rate-th message will be logged. By default, every message is logged. As an example, setting this to '2' means every other message will be logged. The value of this setting is a string but must be parsable as an integer.
The namespace into which Kiali is to be installed. If this is empty or not defined, the default will be the namespace where the Kiali CR is located.
.spec.deployment.node_selector
(object)
A set of node labels that dictate onto which node the Kiali pod will be deployed.
.spec.deployment.pod_annotations
(object)
Custom annotations to be created on the Kiali pod.
.spec.deployment.pod_labels
(object)
Custom labels to be created on the Kiali pod.
An example use for this setting is to inject an Istio sidecar such as,
sidecar.istio.io/inject: "true"
.spec.deployment.priority_class_name
(string)
The priorityClassName used to assign the priority of the Kiali pod.
.spec.deployment.replicas
(integer)
The replica count for the Kiail deployment.
.spec.deployment.resources
(object)
Defines compute resources that are to be given to the Kiali pod’s container. The value is a dict as defined by Kubernetes. See the Kubernetes documentation (https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container).
If you set this to an empty dict ({}) then no resources will be defined in the Deployment.
If you do not set this at all, the default is,
Custom annotations to be created on the Kiali Service resource.
.spec.deployment.service_type
(string)
The Kiali service type. Kubernetes determines what values are valid. Common values are ‘NodePort’, ‘ClusterIP’, and ‘LoadBalancer’.
.spec.deployment.tolerations
(array)
A list of tolerations which declare which node taints Kiali can tolerate. See the Kubernetes documentation on Taints and Tolerations for more details.
.spec.deployment.tolerations[*]
(object)
.spec.deployment.verbose_mode
(string)
DEPRECATED! Determines which priority levels of log messages Kiali will output. Use deployment.logger settings instead.
.spec.deployment.version_label
(string)
Kiali resources will be assigned a ‘version’ label when they are deployed.
This setting determines what value those ‘version’ labels will have.
When empty, its default will be determined as follows,
If deployment.image_version is ‘latest’, version_label will be fixed to ‘master’.
If deployment.image_version is ‘lastrelease’, version_label will be fixed to the last Kiali release version string.
If deployment.image_version is anything else, version_label will be that value, too.
.spec.deployment.view_only_mode
(boolean)
When true, Kiali will be in ‘view only’ mode, allowing the user to view and retrieve management and monitoring data for the service mesh, but not allow the user to modify the service mesh.
.spec.external_services
(object)
These external service configuration settings define how to connect to the external services
like Prometheus, Grafana, and Jaoeger.
Regarding sensitive values in the external_services ‘auth’ sections:
Some external services configured below support an ‘auth’ sub-section in order to tell Kiali
how it should authenticate with the external services. Credentials used to authenticate Kiali
to those external services can be defined in the auth.password and auth.token values
within the auth sub-section. Because these are sensitive values, you may not want to declare
the actual credentials here in the Kiali CR. In this case, you may store the actual password
or token string in a Kubernetes secret. If you do, you need to set the auth.password or
auth.token to a value in the format secret:<secretName>:<secretKey> where <secretName>
is the name of the secret object that Kiali can access, and <secretKey> is the name of the
key within the named secret that contains the actual password or token string. For example,
if Grafana requires a password, you can store that password in a secret named ‘myGrafanaCredentials’
in a key named ‘myGrafanaPw’. In this case, you would set external_services.grafana.auth.password
to secret:myGrafanaCredentials:myGrafanaPw.
.spec.external_services.custom_dashboards
(object)
Settings for enabling and discovering custom dashboards.
Threshold of the number of pods, for a given Application or Workload, above which dashboards discovery will be skipped. This setting only takes effect when discovery_enabled is set to ‘auto’.
Enable, disable or set ‘auto’ mode to the dashboards discovery process. If set to ‘true’, Kiali will always try to discover dashboards based on metrics. Note that this can generate performance penalties while discovering dashboards for workloads having many pods (thus many metrics). When set to ‘auto’, Kiali will skip dashboards discovery for workloads with more than a configured threshold of pods (see discovery_auto_threshold). When discovery is disabled or auto/skipped, it is still possible to tie workloads with dashboards through annotations on pods (refer to the doc https://kiali.io/docs/configuration/custom-dashboard/#pod-annotations). Value must be one of: true, false, auto.
.spec.external_services.custom_dashboards.enabled
(boolean)
Enable or disable custom dashboards, including the dashboards discovery process.
.spec.external_services.custom_dashboards.is_core
(boolean)
Used in the Components health feature. When true, the unhealthy scenarios will be raised as errors. Otherwise, they will be raised as a warning.
The Prometheus label name used for identifying namespaces in metrics for custom dashboards. The default is namespace but you may want to use kubernetes_namespace depending on your Prometheus configuration.
The Prometheus configuration defined here refers to the Prometheus instance that is dedicated to fetching metrics for custom dashboards. This means you can obtain these metrics for the custom dashboards from a Prometheus instance that is different from the one that Istio uses. If this section is omitted, the same Prometheus that is used to obtain the Istio metrics will also be used for retrieving custom dashboard metrics.
The type of authentication to use when contacting the server. Use bearer to send the token to the Prometheus server. Use basic to connect with username and password credentials. Use none to not use any authentication (this is the default).
When true and if auth.type is bearer, Kiali Service Account token will be used for the API calls to Prometheus (in this case, auth.token config is ignored).
Used in the Components health feature. This is the url which Kiali will ping to determine whether the component is reachable or not. It defaults to url when not provided.
Define this section if Prometheus is to be queried through a Thanos proxy. Kiali will still use the url setting to query for Prometheus metrics so make sure that is set appropriately.
The URL used to query the Prometheus Server. This URL must be accessible from the Kiali pod. If empty, the default will assume Prometheus is in the Istio control plane namespace; e.g. http://prometheus.<istio_namespace>:9090.
.spec.external_services.grafana
(object)
Configuration used to access the Grafana dashboards.
.spec.external_services.grafana.auth
(object)
Settings used to authenticate with the Grafana instance.
.spec.external_services.grafana.auth.ca_file
(string)
The certificate authority file to use when accessing Grafana using https. An empty string means no extra certificate authority file is used.
Set true to skip verifying certificate validity when Kiali contacts Grafana over https.
.spec.external_services.grafana.auth.password
(string)
Password to be used when making requests to Grafana, for basic authentication. May refer to a secret.
.spec.external_services.grafana.auth.token
(string)
Token / API key to access Grafana, for token-based authentication. May refer to a secret.
.spec.external_services.grafana.auth.type
(string)
The type of authentication to use when contacting the server. Use bearer to send the token to the Grafana server. Use basic to connect with username and password credentials. Use none to not use any authentication (this is the default).
When true and if auth.type is bearer, Kiali Service Account token will be used for the API calls to Grafana (in this case, auth.token config is ignored).
.spec.external_services.grafana.auth.username
(string)
Username to be used when making requests to Grafana with basic authentication.
.spec.external_services.grafana.dashboards
(array)
A list of Grafana dashboards that Kiali can link to.
The name of a variable that holds the workload name, if used in that dashboard (else it must be omitted).
.spec.external_services.grafana.enabled
(boolean)
When true, Grafana support will be enabled in Kiali.
.spec.external_services.grafana.health_check_url
(string)
Used in the Components health feature. This is the URL which Kiali will ping to determine whether the component is reachable or not. It defaults to in_cluster_url when not provided.
.spec.external_services.grafana.in_cluster_url
(string)
The URL used for in-cluster access. An example would be http://grafana.istio-system:3000. This URL can contain query parameters if needed, such as ‘?orgId=1’. If not defined, it will default to http://grafana.<istio_namespace>:3000.
.spec.external_services.grafana.is_core
(boolean)
Used in the Components health feature. When true, the unhealthy scenarios will be raised as errors. Otherwise, they will be raised as a warning.
.spec.external_services.grafana.url
(string)
The URL that Kiali uses when integrating with Grafana. This URL must be accessible to clients external to the cluster in order for the integration to work properly. If empty, an attempt to auto-discover it is made. This URL can contain query parameters if needed, such as ‘?orgId=1’.
.spec.external_services.istio
(object)
Istio configuration that Kiali needs to know about in order to observe the mesh.
.spec.external_services.istio.component_status
(object)
Istio components whose status will be monitored by Kiali.
The namespace where the component is installed. It defaults to the Istio control plane namespace (e.g. istio_namespace) setting. Note that the Istio documentation suggests you install the ingress and egress to different namespaces, so you most likely will want to explicitly set this namespace value for the ingress and egress components.
The name of the field that annotates a workload to indicate a sidecar should be automatically injected by Istio. This is the name of a Kubernetes annotation. Note that some Istio implementations also support labels by the same name. In other words, if a workload has a Kubernetes label with this name, that may also trigger automatic sidecar injection.
Set true to skip verifying certificate validity when Kiali contacts Prometheus over https.
.spec.external_services.prometheus.auth.password
(string)
Password to be used when making requests to Prometheus, for basic authentication. May refer to a secret.
.spec.external_services.prometheus.auth.token
(string)
Token / API key to access Prometheus, for token-based authentication. May refer to a secret.
.spec.external_services.prometheus.auth.type
(string)
The type of authentication to use when contacting the server. Use bearer to send the token to the Prometheus server. Use basic to connect with username and password credentials. Use none to not use any authentication (this is the default).
When true and if auth.type is bearer, Kiali Service Account token will be used for the API calls to Prometheus (in this case, auth.token config is ignored).
.spec.external_services.prometheus.auth.username
(string)
Username to be used when making requests to Prometheus with basic authentication.
.spec.external_services.prometheus.cache_duration
(integer)
Prometheus caching duration expressed in seconds.
.spec.external_services.prometheus.cache_enabled
(boolean)
Enable/disable Prometheus caching used for Health services.
Used in the Components health feature. This is the url which Kiali will ping to determine whether the component is reachable or not. It defaults to url when not provided.
.spec.external_services.prometheus.is_core
(boolean)
Used in the Components health feature. When true, the unhealthy scenarios will be raised as errors. Otherwise, they will be raised as a warning.
.spec.external_services.prometheus.query_scope
(object)
A set of labelName/labelValue settings applied to every Prometheus query. Used to narrow unified metrics to only those scoped to the Kiali instance.
.spec.external_services.prometheus.thanos_proxy
(object)
Define this section if Prometheus is to be queried through a Thanos proxy. Kiali will still use the url setting to query for Prometheus metrics so make sure that is set appropriately.
Thanos Scrape interval value expresed as a string.
.spec.external_services.prometheus.url
(string)
The URL used to query the Prometheus Server. This URL must be accessible from the Kiali pod. If empty, the default will assume Prometheus is in the Istio control plane namespace; e.g. http://prometheus.<istio_namespace>:9090.
.spec.external_services.tracing
(object)
Configuration used to access the Tracing (Jaeger) dashboards.
.spec.external_services.tracing.auth
(object)
Settings used to authenticate with the Tracing server instance.
.spec.external_services.tracing.auth.ca_file
(string)
The certificate authority file to use when accessing the Tracing server using https. An empty string means no extra certificate authority file is used.
Set true to skip verifying certificate validity when Kiali contacts the Tracing server over https.
.spec.external_services.tracing.auth.password
(string)
Password to be used when making requests to the Tracing server, for basic authentication. May refer to a secret.
.spec.external_services.tracing.auth.token
(string)
Token / API key to access the Tracing server, for token-based authentication. May refer to a secret.
.spec.external_services.tracing.auth.type
(string)
The type of authentication to use when contacting the server. Use bearer to send the token to the Tracing server. Use basic to connect with username and password credentials. Use none to not use any authentication (this is the default).
When true and if auth.type is bearer, Kiali Service Account token will be used for the API calls to the Tracing server (in this case, auth.token config is ignored).
.spec.external_services.tracing.auth.username
(string)
Username to be used when making requests to the Tracing server with basic authentication.
.spec.external_services.tracing.enabled
(boolean)
When true, connections to the Tracing server are enabled. in_cluster_url and/or url need to be provided.
.spec.external_services.tracing.in_cluster_url
(string)
Set URL for in-cluster access, which enables further integration between Kiali and Jaeger. When not provided, Kiali will only show external links using the url setting. Note: Jaeger v1.20+ has separated ports for GRPC(16685) and HTTP(16686) requests. Make sure you use the appropriate port according to the use_grpc value. Example: http://tracing.istio-system:16685
.spec.external_services.tracing.is_core
(boolean)
Used in the Components health feature. When true, the unhealthy scenarios will be raised as errors. Otherwise, they will be raised as a warning.
Kiali use this boolean to find traces with a namespace selector : service.namespace.
.spec.external_services.tracing.url
(string)
The external URL that will be used to generate links to Jaeger. It must be accessible to clients external to the cluster (e.g: a browser) in order to generate valid links. If the tracing service is deployed with a QUERY_BASE_PATH set, set this URL like https:///. For example, https://tracing-service:8080/jaeger
.spec.external_services.tracing.use_grpc
(boolean)
Set to true in order to enable GRPC connections between Kiali and Jaeger which will speed up the queries. In some setups you might not be able to use GRPC (e.g. if Jaeger is behind some reverse proxy that doesn’t support it). If not specified, this will defalt to ‘false’ if deployed within a Maistra/OSSM+OpenShift environment, ‘true’ otherwise.
The direction that applies for this tolerance (e.g. inbound or outbound). This is a regular expression.
.spec.health_config.rate[*].tolerance[*].failure
(integer)
A failure status will be shown when the telemetry reaches this value (specified as an integer representing a percentage).
.spec.health_config.rate[*].tolerance[*].protocol
(string)
The protocol that applies for this tolerance (e.g. grpc or http). This is a regular expression.
.spec.identity
(object)
Settings that define the Kiali server identity.
.spec.identity.cert_file
(string)
Certificate file used to identify the Kiali server. If set, you must go over https to access Kiali. The Kiali operator will set this if it deploys Kiali behind https. When left undefined, the operator will attempt to generate a cluster-specific cert file that provides https by default (today, this auto-generation of a cluster-specific cert is only supported on OpenShift). When set to an empty string, https will be disabled.
.spec.identity.private_key_file
(string)
Private key file used to identify the Kiali server. If set, you must go over https to access Kiali. When left undefined, the Kiali operator will attempt to generate a cluster-specific private key file that provides https by default (today, this auto-generation of a cluster-specific private key is only supported on OpenShift). When set to an empty string, https will be disabled.
.spec.installation_tag
(string)
Tag used to identify a particular instance/installation of the Kiali server. This is merely a human-readable string that will be used within Kiali to help a user identify the Kiali being used (e.g. in the Kiali UI title bar). See deployment.instance_name for the setting used to customize Kiali resource names that are created.
.spec.istio_labels
(object)
Defines specific labels used by Istio that Kiali needs to know about.
.spec.istio_labels.app_label_name
(string)
The name of the label used to define what application a workload belongs to. This is typically something like app or app.kubernetes.io/name.
.spec.istio_labels.injection_label_name
(string)
The name of the label used to instruct Istio to automatically inject sidecar proxies when applications are deployed.
.spec.istio_labels.injection_label_rev
(string)
The label used to identify the Istio revision.
.spec.istio_labels.version_label_name
(string)
The name of the label used to define what version of the application a workload belongs to. This is typically something like version or app.kubernetes.io/version.
.spec.istio_namespace
(string)
The namespace where Istio is installed. If left empty, it is assumed to be the same namespace as where Kiali is installed (i.e. deployment.namespace).
Flag to enable/disable clustering and federation related features.
.spec.kiali_feature_flags.istio_injection_action
(boolean)
Flag to enable/disable an Action to label a namespace for automatic Istio Sidecar injection.
.spec.kiali_feature_flags.istio_upgrade_action
(boolean)
Flag to activate the Kiali functionality of upgrading namespaces to point to an installed Istio Canary revision. Related Canary upgrade and current revisions of Istio should be defined in istio_canary_revision section.
.spec.kiali_feature_flags.ui_defaults
(object)
Default settings for the UI. These defaults apply to all users.
TCP traffic is measured in sent/received/total bytes. Only request traffic supplies response codes. Value must be one of: none, sent, received, or total.
Additional label aggregation for inbound metric pages in detail pages.
You will see these configurations in the ‘Metric Settings’ drop-down.
An example,
Additional label aggregation for outbound metric pages in detail pages.
You will see these configurations in the ‘Metric Settings’ drop-down.
An example,
Duration of metrics to fetch on each refresh. Value must be one of: 1m, 5m, 10m, 30m, 1h, 3h, 6h, 12h, 1d, 7d, or 30d
.spec.kiali_feature_flags.ui_defaults.namespaces
(array)
Default selections for the namespace selection dropdown. Non-existent or inaccessible namespaces will be ignored. Omit or set to an empty array for no default namespaces.
The automatic refresh interval for pages offering automatic refresh. Value must be one of: pause, 10s, 15s, 30s, 1m, 5m or 15m
.spec.kiali_feature_flags.validations
(object)
Features specific to the validations subsystem.
.spec.kiali_feature_flags.validations.ignore
(array)
A list of one or more validation codes whose errors are to be ignored.
.spec.kiali_feature_flags.validations.ignore[*]
(string)
A validation code (e.g. KIA0101) for a specific validation error that is to be ignored.
.spec.kubernetes_config
(object)
Configuration of Kiali’s access of the Kubernetes API.
.spec.kubernetes_config.burst
(integer)
The Burst value of the Kubernetes client.
.spec.kubernetes_config.cache_duration
(integer)
The ratio interval (expressed in seconds) used for the cache to perform a full refresh. Only used when cache_enabled is true.
.spec.kubernetes_config.cache_enabled
(boolean)
Flag to use a Kubernetes cache for watching changes and updating pods and controllers data asynchronously.
.spec.kubernetes_config.cache_istio_types
(array)
Kiali can cache VirtualService, DestinationRule, Gateway and ServiceEntry Istio resources if they are present on this list of Istio types. Other Istio types are not yet supported.
.spec.kubernetes_config.cache_istio_types[*]
(string)
.spec.kubernetes_config.cache_namespaces
(array)
List of namespaces or regex defining namespaces to include in a cache.
This Kiali cache is a list of namespaces per user. This is typically a short-lived cache compared with the duration of the namespace cache defined by the cache_duration setting. This is specified in seconds.
.spec.kubernetes_config.excluded_workloads
(array)
List of controllers that won’t be used for Workload calculation. Kiali queries Deployment, ReplicaSet, ReplicationController, DeploymentConfig, StatefulSet, Job and CronJob controllers. Deployment and ReplicaSet will be always queried, but ReplicationController, DeploymentConfig, StatefulSet, Job and CronJobs can be skipped from Kiali workloads queries if they are present in this list.
.spec.kubernetes_config.excluded_workloads[*]
(string)
.spec.kubernetes_config.qps
(integer)
The QPS value of the Kubernetes client.
.spec.login_token
(object)
.spec.login_token.expiration_seconds
(integer)
A user’s login token expiration specified in seconds. This is applicable to token and header auth strategies only.
.spec.login_token.signing_key
(string)
The signing key used to generate tokens for user authentication. Because this is potentially sensitive, you have the option to store this value in a secret. If you store this signing key value in a secret, you must indicate what key in what secret by setting this value to a string in the form of secret:<secretName>:<secretKey>. If left as an empty string, a secret with a random signing key will be generated for you. The signing key must be 16, 24 or 32 byte long.
.spec.server
(object)
Configuration that controls some core components within the Kiali Server.
.spec.server.address
(string)
Where the Kiali server is bound. The console and API server are accessible on this host.
.spec.server.audit_log
(boolean)
When true, allows additional audit logging on write operations.
.spec.server.cors_allow_all
(boolean)
When true, allows the web console to send requests to other domains other than where the console came from. Typically used for development environments only.
.spec.server.gzip_enabled
(boolean)
When true, Kiali serves http requests with gzip enabled (if the browser supports it) when the requests are over 1400 bytes.
.spec.server.observability
(object)
Settings to enable observability into the Kiali server itself.
.spec.server.observability.metrics
(object)
Settings that control how Kiali itself emits its own metrics.
.spec.server.observability.metrics.enabled
(boolean)
When true, the metrics endpoint will be available for Prometheus to scrape.
.spec.server.observability.metrics.port
(integer)
The port that the server will bind to in order to receive metric requests. This is the port Prometheus will need to scrape when collecting metrics from Kiali.
.spec.server.observability.tracing
(object)
Settings that control how the Kiali server itself emits its own tracing data.
.spec.server.observability.tracing.collector_url
(string)
The URL used to determine where the Kiali server tracing data will be stored.
.spec.server.observability.tracing.enabled
(boolean)
When true, the Kiali server itself will product its own tracing data.
.spec.server.port
(integer)
The port that the server will bind to in order to receive console and API requests.
.spec.server.web_fqdn
(string)
Defines the public domain where Kiali is being served. This is the ‘domain’ part of the URL (usually it’s a fully-qualified domain name). For example, kiali.example.org. When empty, Kiali will try to guess this value from HTTP headers. On non-OpenShift clusters, you must populate this value if you want to enable cross-linking between Kiali instances in a multi-cluster setup.
.spec.server.web_history_mode
(string)
Define the history mode of kiali UI. Value must be one of: browser or hash.
.spec.server.web_port
(string)
Defines the ingress port where the connections come from. This is usually necessary when the application responds through a proxy/ingress, and it does not forward the correct headers (when this happens, Kiali cannot guess the port). When empty, Kiali will try to guess this value from HTTP headers.
.spec.server.web_root
(string)
Defines the context root path for the Kiali console and API endpoints and readiness probes. When providing a context root path that is not /, do not add a trailing slash (i.e. use /kiali not /kiali/). When empty, this will default to / on OpenShift and /kiali on other Kubernetes environments.
.spec.server.web_schema
(string)
Defines the public HTTP schema used to serve Kiali. Value must be one of: http or https. When empty, Kiali will try to guess this value from HTTP headers. On non-OpenShift clusters, you must populate this value if you want to enable cross-linking between Kiali instances in a multi-cluster setup.
.spec.version
(string)
The version of the Ansible playbook to execute in order to install that version of Kiali.
It is rare you will want to set this - if you are thinking of setting this, know what you are doing first.
The only supported value today is default.
This version setting affects the defaults of the deployment.image_name and
deployment.image_version settings. See the comments for those settings
below for additional details. But in short, this version setting will
dictate which version of the Kiali image will be deployed by default.
Note that if you explicitly set deployment.image_name and/or
deployment.image_version you are responsible for ensuring those settings
are compatible with this setting (i.e. the Kiali image must be compatible
with the rest of the configuration and resources the operator will install).
.status
(object)
The processing status of this CR as reported by the Kiali operator.
6 - Namespace Management
Configuring the namespaces accessible and visible to Kiali.
Introduction
The default Kiali installation (as mentioned in the Installation guide) gives Kiali access to all
namespaces available in the cluster.
It is possible to restrict Kiali to a set of desired namespaces by providing a list
of the ones you want, excluding the ones you don’t want, or filtering by a
label selector. You can use a combination of these options.
Accessible Namespaces
You can configure which namespaces are accessible and observable through
Kiali. You can use regex expressions which will be matched against the operator’s
visible namespaces. If not set in the Kiali CR, the default
makes accessible all cluster namespaces, with the exception of
a predefined set of the cluster’s system workloads.
The list of accessible namespaces is specified in the Kiali CR via the
accessible_namespaces setting, under the main deployment section. As an
example, if Kiali is to be installed in the istio-system namespace, and is
expected to monitor all namespaces prefixed with mycorp_, the setting would
be:
Note that the namespaces declared here (including any regex expressions) are evaluated and discovered at install time. Namespaces that do not exist at the time of install but are created later in the future will not be accessible by Kiali. For Kiali to be given access to namespaces created in the future, you must edit the Kiali CR and update the accessible_namespaces setting to include the new namespaces. The one exception is if you set accessible_namespaces to the special value ["**"] - in that case, all namespaces (including any namespaces created in the future) will be accessible to Kiali.
As you can see in the example, the namespace where Kiali is
installed must be listed as accessible (often, but not always, the same namespace as Istio).
This configuration accepts the special pattern accessible_namespaces: ["**"]
which denotes that Kiali is given access to all namespaces in the cluster, including any namespaces created in the future.
If you install the operator using the Helm Charts,
to be able to use the special pattern accessible_namespaces: ["**"],
you must specify the --set clusterRoleCreator=true flag when invoking helm install.
When installing multiple Kiali instances into a single cluster,
accessible_namespaces must be mutually exclusive. In other words, a namespace
set must be matched by only one Kiali CR. Regular expressions must not have
overlapping patterns.
A cluster can have at most one Kiali instance with the special pattern accessible_namespaces: ["**"].
Maistra supports multi-tenancy and the accessible_namespaces extends that
feature to Kiali. However, explicit naming of accessible namespaces can benefit
non-Maistra installations as well - with it Kiali does not need cluster roles
and the Kiali Operator does not need permissions to create cluster roles.
Excluded Namespaces
The Kiali CR tells the Kiali Operator which accessible namespaces should be excluded from the list of namespaces provided by the API and UI. This can be useful if wildcards are used when specifying Accessible Namespaces. This setting has no effect on namespace accessibility. It is only a filter, not security-related.
For example, if the accessible_namespaces configuration includes mycorp_.* but it is not desirable to see test namespaces, the following configuration can be used:
api:namespaces:exclude:- mycorp_test.*
Namespace Selectors
To fetch a subset of the available namespaces, Kiali supports an optional Kubernetes label selector. This selector is especially useful when spec.deployment.accessible_namespaces is set to ["**"] but you want to reduce the namespaces presented in the UI’s namespace list.
The label selector is defined in the Kiali CR setting spec.api.namespaces.label_selector.
The example below selects all namespaces that have a label kiali-enabled: true:
api:namespaces:label_selector:kiali-enabled=true
For further information on how this api.namespaces.label_selector interacts with spec.deployment.accessible_namespaces read the Kiali CR Reference documentation.
Note that when deploying multiple control planes in the same cluster, you will want to set the label selector’s value unique to each control plane. This allows each Kiali instance to select only the namespaces relevant to each control plane. Because in this “soft-multitenancy” mode spec.deployment.accessible_namespaces is typically set to an explicit set of namespaces (i.e. not ["**"]), you do not have to do anything with this label_selector. This is because the default value of label_selector is kiali.io/member-of: <spec.istio_namespace> when spec.deployment.accessible_namespaces is not set to the “all namespaces” value ["**"]. This allows you to have multiple control planes in the same cluster, with each control plane having its own Kiali instance. If you set your own Kiali instance name in the Kiali CR (i.e. you set spec.deployment.instance_name to something other than kiali), then the default label will be kiali.io/<spec.deployment.instance_name>.member-of: <spec.istio_namespace>.
7 - Prometheus, Jaeger and Grafana
Prometheus and Jaeger are primary data sources for Kiali. This page describes how to configure Kiali to communicate with these dependencies. A minimalistic Grafana integration is also available.
Prometheus configuration
Kiali requires Prometheus to generate the
topology graph,
show metrics,
calculate health and
for several other features. If Prometheus is missing or Kiali
can’t reach it, Kiali won’t work properly.
By default, Kiali assumes that Prometheus is available at the URL of the form
http://prometheus.<istio_namespace_name>:9090, which is the usual case if you
are using the Prometheus Istio
add-on.
If your Prometheus instance has a different service name or is installed to a
different namespace, you must manually provide the endpoint where it is
available, like in the following example:
spec:external_services:prometheus:# Prometheus service name is "metrics" and is in the "telemetry" namespaceurl:"http://metrics.telemetry:9090/"
Notice that you don’t need to expose Prometheus outside the cluster. It is
enough to provide the Kubernetes internal service URL.
Kiali maintains an internal cache of some Prometheus queries to improve
performance (mainly, the queries to calculate Health indicators). It
would be very rare to see data delays, but should you notice any delays you may
tune caching parameters to values that work better for your environment. These
are the default values:
spec:external_services:prometheus:cache_enabled:true# Per-query expiration in secondscache_duration:10# Global cache expiration in seconds. Think of it as# the "reset" or "garbage collection" interval.cache_expiration:300
Compatibility with Prometheus-like servers
Although Kiali assumes a Prometheus server and is tested against it, there are
TSDBs that can be used as Prometheus
replacement despite not implementing the full Prometheus API.
Community users have faced two issues when using Prometheus-like TSDBs:
Kiali may report that the TSDB is unreachable, and/or
Kiali may show empty metrics if the TSBD does not implement the /api/v1/status/config.
To fix these issues, you may need to provide a custom health check endpoint for
the TSDB and/or manually provide the configurations that Kiali reads from the
/api/v1/status/config API endpoint:
spec:external_services:prometheus:# Fix the "Unreachable" metrics server warning.health_check_url:"http://custom-tsdb-health-check-url"# Fix for the empty metrics dashboardsthanos_proxy:enabled:trueretention_period:"7d"scrape_interval:"30s"
By default, Kiali will try to reach Jaeger at the GRPC-enabled URL of the form
http://tracing.<istio_namespace_name>:16685/jaeger, which is the usual case
if you are using the Jaeger Istio
add-on.
If this endpoint is unreachable, Kiali will disable features that use
distributed tracing data.
If your Jaeger instance has a different service name or is installed to a
different namespace, you must manually provide the endpoint where it is
available, like in the following example:
spec:external_services:tracing:# Enabled by default. Kiali will anyway fallback to disabled if# Jaeger is unreachable.enabled:true# Jaeger service name is "tracing" and is in the "telemetry" namespace.# Make sure the URL you provide corresponds to the non-GRPC enabled endpoint# if you set "use_grpc" to false.in_cluster_url:'http://tracing.telemetry:16685/jaeger'use_grpc:true# Public facing URL of Jaegerurl:'http://my-jaeger-host/jaeger'
Minimally, you must provide spec.external_services.tracing.in_cluster_url to
enable Kiali features that use distributed tracing data. However, Kiali can
provide contextual links that users can use to jump to the Jaeger console to
inspect tracing data more in depth. For these links to be available you need to
set the spec.external_services.tracing.url which may mean that you should
expose Jaeger outside the cluster.
Grafana configuration
Istio provides preconfigured Grafana
dashboards for the
most relevant metrics of the mesh. Although Kiali offers similar views in its
metrics dashboards, it is not in Kiali’s goals to provide the advanced querying
options, nor the highly customizable settings, that are available in Grafana.
Thus, it is recommended that you use Grafana if you need those advanced
options.
Kiali can provide a direct link from its metric dashboards to the equivalent or
most similar Grafana dashboard, which is convenient if you need the powerful
Grafana options. For these links to appear in Kiali you need to manually
configure what is the Grafana URL, like in the following example:
spec:external_services:grafana:enabled:true# Grafana service name is "grafana" and is in the "telemetry" namespace.in_cluster_url:'http://grafana.telemetry:3000/'# Public facing URL of Grafanaurl:'http://my-ingress-host/grafana'
8 - Role-based access control
Configuring RBAC based on the authentication strategy.
Introduction
Kiali supports role-based access control (RBAC) when using any authentication strategy other than anonymous.
Kiali uses the RBAC capabilities of the underlying cluster. Thus, RBAC is
accomplished by using the standard RBAC features of the cluster, which is
through ClusterRoles, ClusterRoleBindings, Roles and RoleBindings resources.
Read the Kubernetes RBAC documentation
for details. If you are using OpenShift, read the
OpenShift RBAC documentation.
In general, Kiali will give access to the resources granted to the account used
to login. Specifically, depending on the authentication strategy, this
translates to:
Authentication Strategy
Access To
header
resources granted to the user of the header-supplied token
openid
resources granted to the user of the third-party authentication system
openshift
resources granted to the OpenShift user
token
resources granted to the ServiceAccount token used to login
For example, if you are using the token strategy, you would grant
cluster-wide privileges to a ServiceAccount with this command:
Please read your cluster RBAC documentation to learn how to assign privileges.
Minimum required privileges to login
The get namespace privilege in some namespace is the minimum privilege needed
in order to be able to login to Kiali. This means you need the following
minimal Role bound to the user that wants to login:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
rules:
- apiGroups: [""]
resources:
- namespaces
verbs:
- get
This minimal Role will allow a user to login. Kiali may work partially, but
some pages may be blank or show erroneous information, and errors will be
logged constantly. You will need a broader set of privileges so that Kiali
works fine.
Privileges required for Kiali to work correctly
The default installation of Kiali creates a ClusterRole with the needed
privileges to take the most advantage of all Kiali features. Inspect the
privileges with
kubectl describe clusterrole kiali
If you installed Kiali with view_only_mode: true
option, the ClusterRole will be named kiali-viewer instead of kiali.
You can use this ClusterRole to assign privileges to users requiring access
to Kiali. You can assign privileges either in one namespace, which will result in
users being able to see only resources in that namespace; or assign
cluster-wide privileges.
For example, to assign privileges to the john user and limiting access to the
myApp namespace, you could run either:
But if you need to assign cluster-wide privileges, you could run either:
$ kubectl create clusterrolebinding john-admin-binding --clusterrole=kiali --user="john"
$ oc adm policy add-cluster-role-to-user kiali john # For OpenShift clusters
In case you need to assign a more limited set of privileges than the ones
present in the Kiali ClusterRole, create your own ClusterRole or Role
based off the privileges in Kiali’s ClusterRole and remove the privileges you
want to ban. You must understand that some Kiali features may not work properly
because of the reduced privilege set.
Do not edit the kiali or the kiali-viewerClusterRoles because they are bound to the Kiali ServiceAccount and editing
them may lead to Kiali not working properly.
9 - Traffic Health
Customizing Health for Request Traffic.
There are times when Kiali’s default thresholds for traffic health do not work well for a particular situation. For example, at times 404 response codes are expected. Kiali has the ability to set powerful, fine-grained overrides for health configuration.
Default Configuration
By default Kiali uses the traffic rate configuration shown below. Application errors have minimal tolerance while client errors have a higher tolerance reflecting that some level of client errors is often normal (e.g. 404 Not Found):
For http protocol 4xx are client errors and 5xx codes are application errors.
For grpc protocol all 1-16 are errors (0 is success).
So, for example, if the rate of application errors is >= 0.1% Kiali will show Degraded health and if > 10% will show Failure health.
Custom health configuration is specified in the Kiali CR. To see the supported configuration syntax for health_config see the Kiali CR Reference.
Kiali applies the first matching rate configuration (namespace, kind, etc) and calculates the status for each tolerance. The reported health will be the status with highest priority (see below).
[1] The status code typically depends on the request protocol. The special code -, a single dash, is used for requests that don’t receive a response, and therefore no response code.
Kiali reports traffic health with the following top-down status priority :
Priority
Rule (value=% matching requests)
Status
1
value >= FAILURE threshold
FAILURE
2
value >= DEGRADED threshold AND value < FAILURE threshold
Protocol http if % requests with error code 404 are >= 10 then FAILURE, if they are > 0 then DEGRADED
Protocol http if % requests with others error codes are> 0 then FAILURE.
For namespace beta, all resources
Protocol http if % requests with error code 4xx are >= 40 then FAILURE, if they are >= 30 then DEGRADED
Protocol http if % requests with error code 5xx are > 0 then FAILURE
For other namespaces Kiali will apply the defaults.
Protocol http if % requests with error code 5xx are >= 20 then FAILURE, if they are >= 0.1 then DEGRADED
Protocol grpc if % requests with error code match /^[1-9]$|^1[0-6]$/ are >= 20 then FAILURE, if they are >= 0.1 then DEGRADED
Alpha
Beta
10 - Virtual Machine workloads
Ensuring Kiali can visualize a VM WorkloadEntry.
Introduction
Kiali graph visualizes both Virtual Machine workloads (WorkloadEntry) and pod-based workloads, running inside a Kubernetes cluster. You must ensure that the Istio Proxy is running, and correctly configured, on the Virtual Machine. Also, Prometheus must be able to scrape the metrics endpoint of the Istio Proxy running on the VM. Kiali will then be able to read the traffic telemetry for the Virtual Machine workloads, and incorporate the VM workloads into the graph.
Kiali does not currently distinguish between pod-based and VM-based workloads nor does Kiali support viewing additional details for the VM-based workloads beyond what is displayed on the graph. One way to distinguish between the two is to give the VM-based workloads a different version label than the pod-based workloads.
Configuring Prometheus to scrape VM-based Istio Proxy
Once the Istio Proxy is running on a Virtual Machine, configuring Prometheus to scrape the VM’s Istio Proxy metrics endpoint is the only configuration Kiali needs to display traffic for the VM-based workload.
Configuring Prometheus will vary between environments. Here is a very simple example of a Prometheus configuration that includes a job to scrape VM based workloads:
FAILURE
DEGRADED
HEALTHY
No Health Information
