mirror of
https://github.com/coredns/coredns.git
synced 2025-11-03 18:53:13 -05:00
124 lines
4.7 KiB
Groff
124 lines
4.7 KiB
Groff
.\" Generated by Mmark Markdown Processer - mmark.miek.nl
|
|
.TH "COREDNS-LOOP" 7 "March 2021" "CoreDNS" "CoreDNS Plugins"
|
|
|
|
.SH "NAME"
|
|
.PP
|
|
\fIloop\fP - detects simple forwarding loops and halts the server.
|
|
|
|
.SH "DESCRIPTION"
|
|
.PP
|
|
The \fIloop\fP plugin will send a random probe query to ourselves and will then keep track of how many times
|
|
we see it. If we see it more than twice, we assume CoreDNS has seen a forwarding loop and we halt the process.
|
|
|
|
.PP
|
|
The plugin will try to send the query for up to 30 seconds. This is done to give CoreDNS enough time
|
|
to start up. Once a query has been successfully sent, \fIloop\fP disables itself to prevent a query of
|
|
death.
|
|
|
|
.PP
|
|
The query sent is \fB\fC<random number>.<random number>.zone\fR with type set to HINFO.
|
|
|
|
.SH "SYNTAX"
|
|
.PP
|
|
.RS
|
|
|
|
.nf
|
|
loop
|
|
|
|
.fi
|
|
.RE
|
|
|
|
.SH "EXAMPLES"
|
|
.PP
|
|
Start a server on the default port and load the \fIloop\fP and \fIforward\fP plugins. The \fIforward\fP plugin
|
|
forwards to it self.
|
|
|
|
.PP
|
|
.RS
|
|
|
|
.nf
|
|
\&. {
|
|
loop
|
|
forward . 127.0.0.1
|
|
}
|
|
|
|
.fi
|
|
.RE
|
|
|
|
.PP
|
|
After CoreDNS has started it stops the process while logging:
|
|
|
|
.PP
|
|
.RS
|
|
|
|
.nf
|
|
plugin/loop: Loop (127.0.0.1:55953 \-> :1053) detected for zone ".", see https://coredns.io/plugins/loop#troubleshooting. Query: "HINFO 4547991504243258144.3688648895315093531."
|
|
|
|
.fi
|
|
.RE
|
|
|
|
.SH "LIMITATIONS"
|
|
.PP
|
|
This plugin only attempts to find simple static forwarding loops at start up time. To detect a loop,
|
|
the following must be true:
|
|
|
|
.IP \(bu 4
|
|
the loop must be present at start up time.
|
|
.IP \(bu 4
|
|
the loop must occur for the \fB\fCHINFO\fR query type.
|
|
|
|
|
|
.SH "TROUBLESHOOTING"
|
|
.PP
|
|
When CoreDNS logs contain the message \fB\fCLoop ... detected ...\fR, this means that the \fB\fCloop\fR detection
|
|
plugin has detected an infinite forwarding loop in one of the upstream DNS servers. This is a fatal
|
|
error because operating with an infinite loop will consume memory and CPU until eventual out of
|
|
memory death by the host.
|
|
|
|
.PP
|
|
A forwarding loop is usually caused by:
|
|
|
|
.IP \(bu 4
|
|
Most commonly, CoreDNS forwarding requests directly to itself. e.g. via a loopback address such as \fB\fC127.0.0.1\fR, \fB\fC::1\fR or \fB\fC127.0.0.53\fR
|
|
.IP \(bu 4
|
|
Less commonly, CoreDNS forwarding to an upstream server that in turn, forwards requests back to CoreDNS.
|
|
|
|
|
|
.PP
|
|
To troubleshoot this problem, look in your Corefile for any \fB\fCforward\fRs to the zone
|
|
in which the loop was detected. Make sure that they are not forwarding to a local address or
|
|
to another DNS server that is forwarding requests back to CoreDNS. If \fB\fCforward\fR is
|
|
using a file (e.g. \fB\fC/etc/resolv.conf\fR), make sure that file does not contain local addresses.
|
|
|
|
.SS "TROUBLESHOOTING LOOPS IN KUBERNETES CLUSTERS"
|
|
.PP
|
|
When a CoreDNS Pod deployed in Kubernetes detects a loop, the CoreDNS Pod will start to "CrashLoopBackOff".
|
|
This is because Kubernetes will try to restart the Pod every time CoreDNS detects the loop and exits.
|
|
|
|
.PP
|
|
A common cause of forwarding loops in Kubernetes clusters is an interaction with a local DNS cache
|
|
on the host node (e.g. \fB\fCsystemd-resolved\fR). For example, in certain configurations \fB\fCsystemd-resolved\fR will
|
|
put the loopback address \fB\fC127.0.0.53\fR as a nameserver into \fB\fC/etc/resolv.conf\fR. Kubernetes (via \fB\fCkubelet\fR) by default
|
|
will pass this \fB\fC/etc/resolv.conf\fR file to all Pods using the \fB\fCdefault\fR dnsPolicy rendering them
|
|
unable to make DNS lookups (this includes CoreDNS Pods). CoreDNS uses this \fB\fC/etc/resolv.conf\fR
|
|
as a list of upstreams to forward requests to. Since it contains a loopback address, CoreDNS ends up forwarding
|
|
requests to itself.
|
|
|
|
.PP
|
|
There are many ways to work around this issue, some are listed here:
|
|
|
|
.IP \(bu 4
|
|
Add the following to your \fB\fCkubelet\fR config yaml: \fB\fCresolvConf: <path-to-your-real-resolv-conf-file>\fR (or via command line flag \fB\fC--resolv-conf\fR deprecated in 1.10). Your "real"
|
|
\fB\fCresolv.conf\fR is the one that contains the actual IPs of your upstream servers, and no local/loopback address.
|
|
This flag tells \fB\fCkubelet\fR to pass an alternate \fB\fCresolv.conf\fR to Pods. For systems using \fB\fCsystemd-resolved\fR,
|
|
\fB\fC/run/systemd/resolve/resolv.conf\fR is typically the location of the "real" \fB\fCresolv.conf\fR,
|
|
although this can be different depending on your distribution.
|
|
.IP \(bu 4
|
|
Disable the local DNS cache on host nodes, and restore \fB\fC/etc/resolv.conf\fR to the original.
|
|
.IP \(bu 4
|
|
A quick and dirty fix is to edit your Corefile, replacing \fB\fCforward . /etc/resolv.conf\fR with
|
|
the IP address of your upstream DNS, for example \fB\fCforward . 8.8.8.8\fR. But this only fixes the issue for CoreDNS,
|
|
kubelet will continue to forward the invalid \fB\fCresolv.conf\fR to all \fB\fCdefault\fR dnsPolicy Pods, leaving them unable to resolve DNS.
|
|
|
|
|