Skip to main content

Envoy Fusing and Current Limiting Practice (1) Implementing Fusing Based on the Rainbond Plug-in

· 6 min read
info

Envoy can be used as a proxy implementation in the Service Mesh microservice framework. The built-in microservice framework of Rainbond is also implemented based on Envoy.The circuit breaker practice described in this article is implemented based on Rainbond's unique plug-in mechanism.

Introduction to Envoy's circuit breaker mechanism

Circuit breakers are an important part of distributed systems.Failing fast and putting pressure on the downstream as soon as possible can prevent the entire microservice system from entering a bad cascading avalanche state.This is one of the main advantages of the Envoy mesh, Envoy implements enforced circuit breaking restrictions at the network level without having to configure and write each application independently.Envoy supports various types of fully distributed (uncoordinated) circuit：

  • Cluster Maximum Connections (MaxConnections)：The maximum number of connections that Envoy will establish for all hosts in the upstream cluster.In practice, this only works with HTTP/1.1 clusters, since HTTP/2 uses a single connection to each host.

  • Cluster maximum pending requests (MaxPendingRequests)：The maximum number of requests that will be queued while waiting for a ready connection pool connection.In practice, this only works with HTTP/1.1 clusters, as HTTP/2 connection pools do not queue requests.HTTP/2 requests are reused immediately.If this circuit breaker overflows, the cluster'supstream_rq_pending_overflowcounter will be incremented.

  • Cluster Maximum Requests (MaxRequests)：The maximum number of requests that all hosts in the cluster can handle at any given time.In practice, this works for HTTP/2 clusters, since HTTP/1.1 clusters are governed by the max connection circuit breaker.If this circuit breaker overflows, the cluster'supstream_rq_pending_overflowcounter will be incremented.

  • Cluster maximum active retries (MaxRetries)：The maximum number of retries that all hosts in the cluster can perform at any given time.In general, we recommend aggressive circuit break retries so that sporadic failure retries are allowed, but the overall retry volume cannot explode and cause massive cascading failures.If this circuit breaker overflows, the cluster'supstream_rq_retry_overflowcounter will be incremented.

Each circuit breaker threshold can be configured and tracked per upstream cluster and per priority.This allows different components of the distributed system to be independently tuned and have different fusing configurations.

circuit-breaker-1

Fusing based on plug-in mechanism​

The Rainbond cloud-native application management platform uses its own plug-in mechanism to realize the fuse of specified microservices for downstream components.

The default installed Rainbond has integrated export network management plug-in and integrated network management plug-in , both of which are implemented based on Envoy , which can perform more comprehensive network management on the network export direction of the microservices installed with the plug-in.This includes the implementation of the circuit breaker mechanism.

To better describe this process, an example is specially prepared.

The pressure generator based on Locust is used as the client, the integrated network management plug-inis installed, and the Java-maven component is used as the server.The stress generator can set the number of concurrent users according to the graphical interface, and perform stress testing on the service address of Java-maven. During this period, we can collect various phenomena when the circuit breaker mechanism is triggered.

circuit-breaker-9

The installation of the integrated network management plug-in is very simple. Click to install the specified plug-in on the plug-in page of the service component of the client (the pressure generator in the example) that initiates the request.

Set the blowout threshold​

The Java-maven component is implemented based on the Http/1.1 version protocol. According to the explanation of the Envoy circuit breaker mechanism in the first section, we can set it by limiting the maximum number of connections in clusters (MaxConnections) and clusters (MaxPendingRequests) fuse condition.

Click the plug-in of the pressure generator component to view the configuration of the -exit network management plug-in , and you can enter its configuration page.

The integrated network management plug-in is divided into two configuration areas: inbound network management configuration and outbound network management configuration. The setting of the fusing threshold is located in the outbound network management configuration area.

To highlight the effect of the experiment, I set both MaxConnections and MaxPendingRequests to small values.

circuit-breaker-2

The configuration in the figure means that the maximum number of connections to the cluster is 6, and the maximum number of requests waiting is 1 (the default value for both is 1024).This configuration is equivalent to generating the following configuration：for Envoy

"circuit_breakers": {
  "default": {
    "max_connections": 6,
    "max_pending_requests": 1
  }
}

The Domains set for the 5000 port of the downstream application Java-maven is also very important, the pressure generator can apply pressure to the 5000 port of Java-maven by accessing the domain name java-maven.

trigger fuse​

Web pages based on Locust can set concurrency conditions. In this experiment, I set up 97 concurrent requests for domain name http://java-maven. The Locust page will show the total number of requests made, as well as the number of requests in a failed state.

circuit-breaker-4

All bad requests get a 503 status code returned by the circuit breaker mechanism.

circuit-breaker-5

In order to confirm that the number of Tcp connections between the pressure generator and the Java-maven component is indeed limited, you can enter the Java

-maven's web terminal is viewed with the command.

circuit-breaker-3

The 172.20.1.74 in the command is the Pod IP address of the pressure generator component.

It should be noted here that do not query the number of Tcp connections in the pressure generator. This number will be more than 6, but it should actually be 97, because the Locust process that initiates the request will generate Tcp connections according to the number of concurrent users. This process does not Restricted by the circuit breaker mechanism, when the request passes through Envoy, only 6 connections will be successfully established and maintained to the Java-maven server.

Raise the fuse threshold​

Next, by adjusting the configuration of integrated network management plugin , adjust the threshold of fuse, and increase MaxConnections to 66.

circuit-breaker-6

After clicking Update Configuration, the changes will take effect directly without restarting the component.

Appropriately increase the number of concurrent users to 250 in the stress generator, and restart the stress test. It can be found that incorrect requests are no longer generated.

circuit-breaker-7

Re-query the number of tcp connections established in the Java-maven environment, and found that it is no longer 6, but has increased, but has not reached the threshold of 66.

circuit-breaker-8

If the number of concurrent users is continuously increased, the circuit breaker can be triggered again.

Summarize​

Circuit breaker is a very important part of the microservice network governance system.In the ServiceMesh microservice framework implemented by Rainbond combined with Envoy, the fuse mechanism implemented by plug-ins is easy to use and supports dynamic entry, which is very friendly to operators.

In the next article, we will introduce the implementation of full streaming, so stay tuned.