2016-03-19 16:11:30 +00:00
|
|
|
# proxy
|
|
|
|
|
|
2017-07-24 08:24:53 -07:00
|
|
|
*proxy* facilitates both a basic reverse proxy and a robust load balancer.
|
|
|
|
|
|
|
|
|
|
The proxy has support for multiple backends. The load balancing features include multiple policies,
|
2017-09-14 09:36:06 +01:00
|
|
|
health checks, and failovers. If all hosts fail their health check the proxy plugin will fail
|
2017-07-24 08:24:53 -07:00
|
|
|
back to randomly selecting a target and sending packets to it.
|
2016-03-19 16:11:30 +00:00
|
|
|
|
|
|
|
|
## Syntax
|
|
|
|
|
|
|
|
|
|
In its most basic form, a simple reverse proxy uses this syntax:
|
|
|
|
|
|
|
|
|
|
~~~
|
2016-11-24 16:57:20 +01:00
|
|
|
proxy FROM TO
|
2016-03-19 16:11:30 +00:00
|
|
|
~~~
|
|
|
|
|
|
2017-02-06 19:32:48 +00:00
|
|
|
* **FROM** is the base domain to match for the request to be proxied.
|
|
|
|
|
* **TO** is the destination endpoint to proxy to.
|
2016-03-19 16:11:30 +00:00
|
|
|
|
|
|
|
|
However, advanced features including load balancing can be utilized with an expanded syntax:
|
|
|
|
|
|
|
|
|
|
~~~
|
2016-10-10 20:13:22 +01:00
|
|
|
proxy FROM TO... {
|
|
|
|
|
policy random|least_conn|round_robin
|
|
|
|
|
fail_timeout DURATION
|
|
|
|
|
max_fails INTEGER
|
|
|
|
|
health_check PATH:PORT [DURATION]
|
|
|
|
|
except IGNORED_NAMES...
|
2016-07-04 21:13:28 +01:00
|
|
|
spray
|
2017-07-29 04:03:55 -07:00
|
|
|
protocol [dns [force_tcp]|https_google [bootstrap ADDRESS...]|grpc [insecure|CACERT|KEY CERT|KEY CERT CACERT]]
|
2016-03-19 16:11:30 +00:00
|
|
|
}
|
|
|
|
|
~~~
|
|
|
|
|
|
2016-10-10 20:13:22 +01:00
|
|
|
* **FROM** is the name to match for the request to be proxied.
|
2017-02-15 08:03:14 +00:00
|
|
|
* **TO** is the destination endpoint to proxy to. At least one is required, but multiple may be
|
|
|
|
|
specified. **TO** may be an IP:Port pair, or may reference a file in resolv.conf format
|
|
|
|
|
* `policy` is the load balancing policy to use; applies only with multiple backends. May be one of
|
|
|
|
|
random, least_conn, or round_robin. Default is random.
|
|
|
|
|
* `fail_timeout` specifies how long to consider a backend as down after it has failed. While it is
|
|
|
|
|
down, requests will not be routed to that backend. A backend is "down" if CoreDNS fails to
|
|
|
|
|
communicate with it. The default value is 10 seconds ("10s").
|
|
|
|
|
* `max_fails` is the number of failures within fail_timeout that are needed before considering
|
|
|
|
|
a backend to be down. If 0, the backend will never be marked as down. Default is 1.
|
plugin/proxy: decrease health timeouts (#1107)
Turn down the timeouts and numbers a bit:
FailTimeout 10s -> 5s
Future 60s -> 12s
TryDuration 60s -> 16s
The timeout for decrementing the fails in a host: 10s -> 2s
And the biggest change: don't set fails when the error is Timeout(),
meaning we loop for a bit and may try the same server again, but we
don't mark our upstream as bad, see comments in proxy.go. Testing this
with "ANY isc.org" and "MX miek.nl" we see:
~~~
::1 - [24/Sep/2017:08:06:17 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.001621221s
24/Sep/2017:08:06:17 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:37420->8.8.8.8:53: i/o timeout
::1 - [24/Sep/2017:08:06:17 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 35.957284ms
127.0.0.1 - [24/Sep/2017:08:06:18 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.002051726s
24/Sep/2017:08:06:18 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:54901->8.8.8.8:53: i/o timeout
::1 - [24/Sep/2017:08:06:19 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 56.848416ms
127.0.0.1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 48.118349ms
::1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 1.055172915s
~~~
So the ANY isc.org queries show up twice, because we retry internally -
this is I think WAI.
The `miek.nl MX` queries are just processed normally as no backend is
marked as unreachable.
May fix #1035 #486
2017-09-24 20:05:36 +01:00
|
|
|
* `health_check` will check **PATH** (on **PORT**) on each backend. If a backend returns a status code of
|
2017-06-30 10:13:45 +01:00
|
|
|
200-399, then that backend is marked healthy for double the healthcheck duration. If it doesn't,
|
|
|
|
|
it is marked as unhealthy and no requests are routed to it. If this option is not provided then
|
plugin/proxy: decrease health timeouts (#1107)
Turn down the timeouts and numbers a bit:
FailTimeout 10s -> 5s
Future 60s -> 12s
TryDuration 60s -> 16s
The timeout for decrementing the fails in a host: 10s -> 2s
And the biggest change: don't set fails when the error is Timeout(),
meaning we loop for a bit and may try the same server again, but we
don't mark our upstream as bad, see comments in proxy.go. Testing this
with "ANY isc.org" and "MX miek.nl" we see:
~~~
::1 - [24/Sep/2017:08:06:17 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.001621221s
24/Sep/2017:08:06:17 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:37420->8.8.8.8:53: i/o timeout
::1 - [24/Sep/2017:08:06:17 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 35.957284ms
127.0.0.1 - [24/Sep/2017:08:06:18 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.002051726s
24/Sep/2017:08:06:18 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:54901->8.8.8.8:53: i/o timeout
::1 - [24/Sep/2017:08:06:19 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 56.848416ms
127.0.0.1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 48.118349ms
::1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 1.055172915s
~~~
So the ANY isc.org queries show up twice, because we retry internally -
this is I think WAI.
The `miek.nl MX` queries are just processed normally as no backend is
marked as unreachable.
May fix #1035 #486
2017-09-24 20:05:36 +01:00
|
|
|
health checks are disabled. The default duration is 4 seconds ("4s").
|
2017-02-15 08:03:14 +00:00
|
|
|
* **IGNORED_NAMES** in `except` is a space-separated list of domains to exclude from proxying.
|
|
|
|
|
Requests that match none of these names will be passed through.
|
|
|
|
|
* `spray` when all backends are unhealthy, randomly pick one to send the traffic to. (This is
|
|
|
|
|
a failsafe.)
|
|
|
|
|
* `protocol` specifies what protocol to use to speak to an upstream, `dns` (the default) is plain
|
|
|
|
|
old DNS, and `https_google` uses `https://dns.google.com` and speaks a JSON DNS dialect. Note when
|
|
|
|
|
using this **TO** will be ignored. The `grpc` option will talk to a server that has implemented
|
2017-03-18 17:08:39 +00:00
|
|
|
the [DnsService](https://github.com/coredns/coredns/pb/dns.proto).
|
2017-09-14 09:36:06 +01:00
|
|
|
An out-of-tree plugin that implements the server side of this can be found at
|
2017-02-15 08:03:14 +00:00
|
|
|
[here](https://github.com/infobloxopen/coredns-grpc).
|
2016-03-19 16:11:30 +00:00
|
|
|
|
|
|
|
|
## Policies
|
|
|
|
|
|
2016-08-22 14:33:40 -07:00
|
|
|
There are three load-balancing policies available:
|
2016-10-10 20:13:22 +01:00
|
|
|
* `random` (default) - Randomly select a backend
|
|
|
|
|
* `least_conn` - Select the backend with the fewest active connections
|
|
|
|
|
* `round_robin` - Select the backend in round-robin fashion
|
2016-03-19 16:11:30 +00:00
|
|
|
|
2016-06-14 18:04:29 +01:00
|
|
|
All polices implement randomly spraying packets to backend hosts when *no healthy* hosts are
|
|
|
|
|
available. This is to preeempt the case where the healthchecking (as a mechanism) fails.
|
|
|
|
|
|
2017-01-15 08:12:58 +00:00
|
|
|
## Upstream Protocols
|
|
|
|
|
|
2017-02-06 19:32:48 +00:00
|
|
|
Currently `protocol` supports `dns` (i.e., standard DNS over UDP/TCP) and `https_google` (JSON
|
|
|
|
|
payload over HTTPS). Note that with `https_google` the entire transport is encrypted. Only *you* and
|
|
|
|
|
*Google* can see your DNS activity.
|
|
|
|
|
|
2017-03-14 21:32:21 +00:00
|
|
|
* `dns`: uses the standard DNS exchange. You can pass `force_tcp` to make sure that the proxied connection is performed
|
|
|
|
|
over TCP, regardless of the inbound request's protocol.
|
2017-02-06 19:32:48 +00:00
|
|
|
* `https_google`: bootstrap **ADDRESS...** is used to (re-)resolve `dns.google.com` to an address to
|
|
|
|
|
connect to. This happens every 300s. If not specified the default is used: 8.8.8.8:53/8.8.4.4:53.
|
|
|
|
|
Note that **TO** is *ignored* when `https_google` is used, as its upstream is defined as
|
|
|
|
|
`dns.google.com`.
|
|
|
|
|
|
|
|
|
|
Debug queries are enabled by default and currently there is no way to turn them off. When CoreDNS
|
|
|
|
|
receives a debug query (i.e. the name is prefixed with `o-o.debug.`) a TXT record with Comment
|
2017-02-15 08:03:14 +00:00
|
|
|
from `dns.google.com` is added. Note this is not always set.
|
2017-02-14 22:20:20 -05:00
|
|
|
* `grpc`: options are used to control how the TLS connection is made to the gRPC server.
|
|
|
|
|
* None - No client authentication is used, and the system CAs are used to verify the server certificate.
|
|
|
|
|
* `insecure` - TLS is not used, the connection is made in plaintext (not good in production).
|
2017-07-29 04:03:55 -07:00
|
|
|
* **CACERT** - No client authentication is used, and the file **CACERT** is used to verify the server certificate.
|
|
|
|
|
* **KEY** **CERT** - Client authentication is used with the specified key/cert pair. The server
|
2017-02-15 08:03:14 +00:00
|
|
|
certificate is verified with the system CAs.
|
2017-07-29 04:03:55 -07:00
|
|
|
* **KEY** **CERT** **CACERT** - Client authentication is used with the specified key/cert pair. The
|
|
|
|
|
server certificate is verified using the **CACERT** file.
|
2017-02-14 22:20:20 -05:00
|
|
|
|
2017-09-14 09:36:06 +01:00
|
|
|
An out-of-tree plugin that implements the server side of this can be found at
|
2017-02-15 08:03:14 +00:00
|
|
|
[here](https://github.com/infobloxopen/coredns-grpc).
|
2017-01-15 08:12:58 +00:00
|
|
|
|
2016-10-28 12:54:49 +01:00
|
|
|
## Metrics
|
|
|
|
|
|
|
|
|
|
If monitoring is enabled (via the *prometheus* directive) then the following metric is exported:
|
|
|
|
|
|
2017-02-07 21:28:47 +00:00
|
|
|
* coredns_proxy_request_count_total{proto, proxy_proto, from}
|
2016-10-28 12:54:49 +01:00
|
|
|
|
2017-02-15 08:03:14 +00:00
|
|
|
Where `proxy_proto` is the protocol used (`dns`, `grpc`, or `https_google`) and `from` is **FROM**
|
|
|
|
|
specified in the config, `proto` is the protocol used by the incoming query ("tcp" or "udp").
|
2016-10-28 12:54:49 +01:00
|
|
|
|
2016-03-19 16:11:30 +00:00
|
|
|
## Examples
|
|
|
|
|
|
2016-03-19 20:53:37 +00:00
|
|
|
Proxy all requests within example.org. to a backend system:
|
|
|
|
|
|
|
|
|
|
~~~
|
2017-08-16 10:00:32 -04:00
|
|
|
proxy example.org 127.0.0.1:9005
|
2016-03-19 20:53:37 +00:00
|
|
|
~~~
|
|
|
|
|
|
2016-03-19 16:11:30 +00:00
|
|
|
Load-balance all requests between three backends (using random policy):
|
2016-03-19 20:53:37 +00:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
|
|
|
|
. {
|
|
|
|
|
proxy . 10.0.0.10:53 10.0.0.11:1053 10.0.0.12
|
|
|
|
|
}
|
2016-03-19 20:53:37 +00:00
|
|
|
~~~
|
2016-03-19 16:11:30 +00:00
|
|
|
|
|
|
|
|
Same as above, but round-robin style:
|
|
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
|
|
|
|
. {
|
|
|
|
|
proxy . 10.0.0.10:53 10.0.0.11:1053 10.0.0.12 {
|
|
|
|
|
policy round_robin
|
|
|
|
|
}
|
2016-03-19 16:11:30 +00:00
|
|
|
}
|
2016-03-19 20:53:37 +00:00
|
|
|
~~~
|
2016-03-19 16:11:30 +00:00
|
|
|
|
|
|
|
|
With health checks and proxy headers to pass hostname, IP, and scheme upstream:
|
|
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
|
|
|
|
. {
|
|
|
|
|
proxy . 10.0.0.11:53 10.0.0.11:53 10.0.0.12:53 {
|
|
|
|
|
policy round_robin
|
|
|
|
|
health_check /health:8080
|
|
|
|
|
}
|
2016-03-19 16:11:30 +00:00
|
|
|
}
|
2016-03-19 20:53:37 +00:00
|
|
|
~~~
|
|
|
|
|
|
2016-03-19 20:59:10 +00:00
|
|
|
Proxy everything except requests to miek.nl or example.org
|
2016-03-19 20:53:37 +00:00
|
|
|
|
|
|
|
|
~~~
|
2017-09-15 11:30:10 +01:00
|
|
|
. {
|
|
|
|
|
proxy . 10.0.0.10:1234 {
|
|
|
|
|
except miek.nl example.org
|
|
|
|
|
}
|
2016-03-19 16:11:30 +00:00
|
|
|
}
|
2016-03-19 20:53:37 +00:00
|
|
|
~~~
|
2016-10-22 10:52:10 -04:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
Proxy everything except `example.org` using the host's `resolv.conf`'s nameservers:
|
2016-10-22 10:52:10 -04:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
|
|
|
|
. {
|
|
|
|
|
proxy . /etc/resolv.conf {
|
|
|
|
|
except miek.nl example.org
|
|
|
|
|
}
|
2016-10-22 10:52:10 -04:00
|
|
|
}
|
|
|
|
|
~~~
|
2017-02-06 19:32:48 +00:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
Proxy all requests within `example.org` to Google's `dns.google.com`.
|
2017-02-06 19:32:48 +00:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
|
|
|
|
. {
|
|
|
|
|
proxy example.org 1.2.3.4:53 {
|
|
|
|
|
protocol https_google
|
|
|
|
|
}
|
2017-02-06 19:32:48 +00:00
|
|
|
}
|
|
|
|
|
~~~
|
|
|
|
|
|
2017-02-15 08:03:14 +00:00
|
|
|
Proxy everything with HTTPS to `dns.google.com`, except `example.org`. Then have another proxy in
|
|
|
|
|
another stanza that uses plain DNS to resolve names under `example.org`.
|
2017-02-06 19:32:48 +00:00
|
|
|
|
2017-09-15 11:30:10 +01:00
|
|
|
~~~ corefile
|
2017-02-15 08:03:14 +00:00
|
|
|
. {
|
|
|
|
|
proxy . 1.2.3.4:53 {
|
2017-05-30 16:03:35 +02:00
|
|
|
except example.org
|
2017-02-15 08:03:14 +00:00
|
|
|
protocol https_google
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
example.org {
|
|
|
|
|
proxy . 8.8.8.8:53
|
2017-02-06 19:32:48 +00:00
|
|
|
}
|
|
|
|
|
~~~
|