Autoscaling a Dapr app with KEDA
Dapr is a programming model that’s being installed and operated using a sidecar, and thus leaves autoscaling to the hosting layer, for example Kubernetes. Many of Dapr’s bindings overlap with those of KEDA, an Event Driven Autoscaler for Kubernetes.
For apps that use these bindings, it is easy to configure a KEDA autoscaler.
To install KEDA, follow these instructions on the KEDA Github page.
Create KEDA enabled Dapr binding
For this example, we’ll be using Kafka.
You can install Kafka in your cluster by using Helm:
$ helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator $ helm install --name my-kafka incubator/kafka
Next, we’ll create the Dapr Kafka binding for Kubernetes.
Paste the following in a file named kafka.yaml:
apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: kafkaevent spec: type: bindings.kafka metadata: - name: brokers value: "my-kafka:9092" - name: topics value: "myTopic" - name: consumerGroup value: "group1"
The following YAML defines a Kafka component that listens for the topic
myTopic, with consumer group
group1 and that connects to a broker at
Deploy the binding to the cluster:
$ kubectl apply -f kafka.yaml
Create the KEDA autoscaler for Kafka
Paste the following to a file named kafka_scaler.yaml, and put the name of your Deployment in the required places:
apiVersion: keda.k8s.io/v1alpha1 kind: ScaledObject metadata: name: kafka-scaler namespace: default labels: deploymentName: <REPLACE-WITH-DEPLOYMENT-NAME> spec: scaleTargetRef: deploymentName: <REPLACE-WITH-DEPLOYMENT-NAME> triggers: - type: kafka metadata: type: kafkaTrigger direction: in name: event topic: myTopic brokers: my-kafka:9092 consumerGroup: group2 dataType: binary lagThreshold: '5'
Deploy the KEDA scaler to Kubernetes:
$ kubectl apply -f kafka_scaler.yaml
You can now start publishing messages to your Kafka topic
myTopic and watch the pods autoscale when the lag threshold is bigger than
5, as defined in the KEDA scaler manifest.