Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upstream keepalive time #8319

Merged
merged 8 commits into from
Apr 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions docs/user-guide/nginx-configuration/configmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ The following table shows a configuration option's name, type, and the default v
|[variables-hash-bucket-size](#variables-hash-bucket-size)|int|128|
|[variables-hash-max-size](#variables-hash-max-size)|int|2048|
|[upstream-keepalive-connections](#upstream-keepalive-connections)|int|320|
|[upstream-keepalive-time](#upstream-keepalive-time)|string|"1h"|
|[upstream-keepalive-timeout](#upstream-keepalive-timeout)|int|60|
|[upstream-keepalive-requests](#upstream-keepalive-requests)|int|10000|
|[limit-conn-zone-variable](#limit-conn-zone-variable)|string|"$binary_remote_addr"|
Expand Down Expand Up @@ -223,13 +224,13 @@ Enables the return of the header Server from the backend instead of the generic

Enables Ingress to parse and add *-snippet annotations/directives created by the user. _**default:**_ `true`

Warning: We recommend enabling this option only if you TRUST users with permission to create Ingress objects, as this
Warning: We recommend enabling this option only if you TRUST users with permission to create Ingress objects, as this
may allow a user to add restricted configurations to the final nginx.conf file

## annotation-value-word-blocklist

Contains a comma-separated value of chars/words that are well known of being used to abuse Ingress configuration
and must be blocked. Related to [CVE-2021-25742](https://github.com/kubernetes/ingress-nginx/issues/7837)
Contains a comma-separated value of chars/words that are well known of being used to abuse Ingress configuration
and must be blocked. Related to [CVE-2021-25742](https://github.com/kubernetes/ingress-nginx/issues/7837)

When an annotation is detected with a value that matches one of the blocked bad words, the whole Ingress won't be configured.

Expand Down Expand Up @@ -769,6 +770,14 @@ _References:_
[http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive)


## upstream-keepalive-time

Sets the maximum time during which requests can be processed through one keepalive connection.
_**default:**_ "1h"

_References:_
[http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_time](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_time)

## upstream-keepalive-timeout

Sets a timeout during which an idle keepalive connection to an upstream server will stay open.
Expand Down Expand Up @@ -1258,7 +1267,7 @@ Configure `memcached` client for [Global Rate Limiting](https://github.com/kuber
* `global-rate-limit-memcached-host`: IP/FQDN of memcached server to use. Required to enable Global Rate Limiting.
* `global-rate-limit-memcached-port`: port of memcached server to use. Defaults default memcached port of `11211`.
* `global-rate-limit-memcached-connect-timeout`: configure timeout for connect, send and receive operations. Unit is millisecond. Defaults to 50ms.
* `global-rate-limit-memcached-max-idle-timeout`: configure timeout for cleaning idle connections. Unit is millisecond. Defaults to 50ms.
* `global-rate-limit-memcached-max-idle-timeout`: configure timeout for cleaning idle connections. Unit is millisecond. Defaults to 50ms.
* `global-rate-limit-memcached-pool-size`: configure number of max connections to keep alive. Make sure your `memcached` server can handle
`global-rate-limit-memcached-pool-size * worker-processes * <number of ingress-nginx replicas>` simultaneous connections.

Expand Down
5 changes: 5 additions & 0 deletions internal/ingress/controller/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,10 @@ type Configuration struct {
// http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
UpstreamKeepaliveConnections int `json:"upstream-keepalive-connections,omitempty"`

// Sets the maximum time during which requests can be processed through one keepalive connection
// https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_time
UpstreamKeepaliveTime string `json:"upstream-keepalive-time,omitempty"`

// Sets a timeout during which an idle keepalive connection to an upstream server will stay open.
// http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout
UpstreamKeepaliveTimeout int `json:"upstream-keepalive-timeout,omitempty"`
Expand Down Expand Up @@ -892,6 +896,7 @@ func NewDefault() Configuration {
ServiceUpstream: false,
},
UpstreamKeepaliveConnections: 320,
UpstreamKeepaliveTime: "1h",
UpstreamKeepaliveTimeout: 60,
UpstreamKeepaliveRequests: 10000,
LimitConnZoneVariable: defaultLimitConnZoneVariable,
Expand Down
4 changes: 2 additions & 2 deletions rootfs/etc/nginx/template/nginx.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ http {
{{ else }}
modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
{{ end }}

{{ if $all.Cfg.EnableOWASPCoreRules }}
modsecurity_rules_file /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf;
{{ end }}
Expand Down Expand Up @@ -508,7 +508,7 @@ http {

{{ if (gt $cfg.UpstreamKeepaliveConnections 0) }}
keepalive {{ $cfg.UpstreamKeepaliveConnections }};

keepalive_time {{ $cfg.UpstreamKeepaliveTime }};
keepalive_timeout {{ $cfg.UpstreamKeepaliveTimeout }}s;
keepalive_requests {{ $cfg.UpstreamKeepaliveRequests }};
{{ end }}
Expand Down
9 changes: 9 additions & 0 deletions test/e2e/settings/keep-alive.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,15 @@ var _ = framework.DescribeSetting("keep-alive keep-alive-requests", func() {
})
})

ginkgo.It("should set keepalive time to upstream server", func() {
f.UpdateNginxConfigMapData("upstream-keepalive-time", "75s")

f.WaitForNginxConfiguration(func(server string) bool {
match, _ := regexp.MatchString(`upstream\supstream_balancer\s\{[\s\S]*keepalive_time\s*75s;`, server)
return match
})
})

ginkgo.It("should set the request count to upstream server through one keep alive connection", func() {
f.UpdateNginxConfigMapData("upstream-keepalive-requests", "200")

Expand Down