How-To: Control concurrency and rate limit applications

Typically, in distributed computing, you may only want to allow for a given number of requests to execute concurrently. Using Dapr’s app-max-concurrency, you can control how many requests and events can invoke your application simultaneously.

Default app-max-concurreny is set to -1, meaning no concurrency.

Different approaches

While this guide focuses on app-max-concurrency, you can also limit request rate per second using the middleware.http.ratelimit middleware. However, it’s important to understand the difference between the two approaches:

• middleware.http.ratelimit: Time bound and limits the number of requests per second
• app-max-concurrency: Specifies the number of concurrent requests (and events) at any point of time.

See Rate limit middleware for more information about that approach.

Demo

Watch this video on how to control concurrency and rate limiting.

Configure app-max-concurrency

Without using Dapr, you would need to create some sort of a semaphore in the application and take care of acquiring and releasing it.

Using Dapr, you don’t need to make any code changes to your application.

Select how you’d like to configure app-max-concurrency.

• CLI
• Kubernetes

dapr run --app-max-concurrency 1 --app-port 5000 python ./app.py

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodesubscriber
  namespace: default
  labels:
    app: nodesubscriber
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodesubscriber
  template:
    metadata:
      labels:
        app: nodesubscriber
      annotations:
        dapr.io/enabled: "true"
        dapr.io/app-id: "nodesubscriber"
        dapr.io/app-port: "3000"
        dapr.io/app-max-concurrency: "1"
#...

Limitations

Controlling concurrency on external requests

Rate limiting is guaranteed for every event coming from Dapr, including pub/sub events, direct invocation from other services, bindings events, etc. However, Dapr can’t enforce the concurrency policy on requests that are coming to your app externally.

Related links

Arguments and annotations

Next steps