Deploy traefik2.1 in kubernetes cluster

wzdushu@gmail.com (wanzi) — Tue, 17 Dec 2019 10:22:42 +0800

Architecture & Concepts

Traefik 2.x has a big change compared to 1.7.x architecture. As shown in the architecture diagram above, the main function is to support TCP protocol and add the concept of Router.

Here we use Traefik 2.1 deployed in the kubernetes cluster. Business access is requested to traefik Ingress through haproxy. The following are some concepts involved in the construction process:

EntryPoints: Traefik’s network entry, defining the port where the request is accepted (regardless of http or tcp)
CRD: Extension of Kubernetes API
IngressRouter: forwards incoming requests to services that can handle requests. In addition, Middlewares can dynamically update requests before forwarding requests
Middlewares: dynamically process request parameters before the request reaches the service, such as header or forwarding rules, etc.
TraefikService: If this type is defined in CRD, IngressRouter can directly reference it. It is located between IngressRouter and the service, similar to the Maesh architecture. It is more suitable for more complex scenarios and can be omitted in general.

kubernetes configuration

Configure SSL certificate

Because the business service uses https, configure the SSL certificate here first:

`1`	`kubectl -n cattle-system create secret tls tls-rancher-ingress --cert=test.cn.pem --key=test.cn.key`

Configure cluster access control (RBAC)

apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller

rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions resources: - ingresses verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses/status verbs: - update - apiGroups: - middlewares verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - ingressroutes verbs: - get - list - watch - apiGroups: - traefik. containo.us resources: - ingressroutetcps verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - tlsoptions verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - traefikservices
verbs:
- get
- list
- watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller

roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system

TLS parameter configuration

TLS1.2 is configured by default, of course TLS1.3 can also be used

apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
name: mytlsoption
namespace: kube-system

spec:
minversion: VersionTLS12 snistrict: true ciphersuites: - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 - TLS_RSA_WITH_AES_256_GCM_SHA384 - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 - T LS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ``` ## CRD configuration
IngressRoute, Middleware, TLSOption, IngressRouteTCP, and TraefikService are defined here, of which TraefikService is a new CRD added in version 2.1

```yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutes.traefik.containo.us

spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRoute
plural: ingressroutes
singular: ingressroute
scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: middlewares.traefik.containo.us
spec:
group: traefik.containo.us
version: v1alpha1 names: kind: Middleware plural: middlewares singular: middleware scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tlsoptions.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TLSOption plural: tlsoptions singular: tlsoption scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutetcps.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRouteTCP
plural: ingressroutetcps
singular: ingressroutetcp
scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: traefikservices.traefik.containo.us

spec:
group: traefik.containo.us
version: v1alpha1
names:
kind: TraefikService
plural: traefikservices
singular: traefikservice
scope: Namespaced

TraefikService configuration

TraefikService is somewhat similar to Maesh in solving the calling logic between services, but Maesh relies on coredns; in addition, traefik service can also set the backend service weight and configure the traffic mirroring of the service.

Here we configure the traefik dashboard and rancher’s traefikservice type service. For other service configurations, you can refer to rancher’s traefik service here. The traefik service will forward requests to the kubernetes service type (in the previous section, we have created the rancher service through helm3). Here is just an example:

apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: traefik-webui-traefikservice
namespace: kube-system

spec:
weighted:
services:
- name: traefik-ingress-service
weight: 1
port: 8080

---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:t
name: rancher-traefikservice
namespace: cattle-system

spec:
weighted:
services:
- name: rancher
weight: 1
port: 80

Deployment configuration

Configure k8s standard service
Create traefik configmap, and configure entrypoints and default SSL certificate
Deploy traefik ingress controller in Deployment mode

kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
nodePort: 23456
name: http
- protocol: TCP
port: 443
nodePort: 23457
name: https
type: NodePort

---
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-conf
namespace: kube-system data: traefik.toml: | [global] checkNewVersion = false sendAnonymousUsage = false [log] level = "DEBUG" [api] dashboard = true [metrics.prometheus] buckets = [0.1,0.3,1.2,5.0] entryPoint = "metrics" [entryPoints] [entryPoints.http] address = ":80" [entryPoints.https] address = ":4 43" [tls.stores] [tls.stores.default] [tls.stores.default.defaultCertificate] certFile = "/config/tls/test.cn.crt" keyFile = "/config/tls/test.cn.key" --- kind: Deployment apiVersion: apps/v1 metadata: name: traefik namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: replicas: 1 selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 nodeSelector: node- role.kubernetes.io/traefik: "true" volumes: - name: ssl secret: secretName: traefik-cert - name: config configMap: name: traefik-conf containers: - image: traefik:v2.1.1 name: traefik-ingress-lb volumeMounts: - mountPath: "/config" name: "config" - mountPath: "/config/tls" name: "ssl" resources: limits: cpu: 1000m memory: 800Mi requests: cpu: 500m memory: 600Mi ports: - name: http containerPort: 80 hostPort: 80 - name: https containerPort: 443 hostPort: 443 args: - --entrypoints.http.Address=:8 0 - --entrypoints.https.Address=:443 - --api - --accesslog - --providers.file.directory=/config/ - --providers.file.watch=true - --ping=true - --providers.kubernetescrd ```` ## IngressRouter configuration * Define middleware to configure X-Forwarded-Proto header information for rancher service
* Configure rancher's ingressroute and use the default certificate

```yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: redirect-https
namespace: kube-system
spec:
redirectScheme:
scheme: https

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: http-default-router
namespace: kube-system
spec:
entryPoints:
- http
routes:
- match: HostRegexp(`{host:.+}`)
kind: Rule
services:
- name: traefik-ingress-service
kind: Service
namespaces: kube-system
port: 80
middlewares:
- name: redirect-https
tls:
options:
name: mytlsoption
namespaces: kube-system
certResolver: default

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-webui
namespace: kube-system
spec:
entryPoints:
- https
routes:
- match: Host(`traefik.test.cn`)
kind: Rule
services:
- name: api@internal
kind: TraefikService
namespaces: kube-system

Configure Rancher’s IngressRoute

Since I use rancher to manage the cluster here, the actual environment can be configured according to your own needs

Define middleware to configure the X-Forwarded-Proto header information for the rancher service
Configure rancher’s ingressroute and use the default certificate

apiVersion: traefik.containo.us/v1alpha1 kind: Middleware metadata: name: rancher-https-headers namespace: cattle-system spec: headers: customRequestHeaders: .test.cn`) kind: Rule services: - name: rancher kind: Service namespaces: cattle-system port: 80 middlewares: - name: rancher-https-headers namespaces: cattle-system tls: certResolver: default ```` ## Unified deployment of traefik2.1

```shell
kubectl apply -f 01-crd.yaml
kubectl apply -f 02-rbac.yaml
kubectl apply -f 03-tlsoption.yaml
kubectl apply -f 04-traefikservices.yaml #Not required
kubectl apply -f 05-traefik.yaml
kubectl apply -f 06-ingressrouter.yaml
kubectl apply -f 06-ingressrouter-rancher.yaml

For more configuration information, please visit my github repository: https://github.com/iwz2099/kubecase/tree/master/traefik/v2

Traefik on Wanzi's Blog – Notes on life, tech, and DevOps.