diff --git a/man/coredns-shed.7 b/man/coredns-shed.7 new file mode 100644 index 000000000..471ca551e --- /dev/null +++ b/man/coredns-shed.7 @@ -0,0 +1,106 @@ +.\" Generated by Mmark Markdown Processor - mmark.miek.nl +.TH "COREDNS-SHED" 7 "August 2026" "CoreDNS" "CoreDNS Plugins" + +.SH "NAME" +.PP +\fIshed\fP \- serializes UDP response writes per listener socket and sheds load when the socket cannot keep up. + +.SH "DESCRIPTION" +.PP +UDP responses written back through one listener socket serialize on the Go runtime's internal +fdMutex, which allows at most 2^20\-1 concurrent operations (holders plus waiters) per file +descriptor and terminates the process with + +.PP +.RS + +.nf +panic: too many concurrent operations on a single file or socket (max 1048575) + +.fi +.RE + +.PP +when that is exceeded. CoreDNS serves UDP with one goroutine per query, all writing back through +the shared packet connection, so when queries arrive faster than the socket's serialized writes +drain, every excess in\-flight query parks its goroutine in that wait queue and nothing bounds the +pile. Observed in production: ~2.8M goroutines and 60GiB RSS before the panic. + +.PP +The \fIshed\fP plugin makes that panic structurally unreachable, per UDP listener socket: + +.IP \(bu 4 +\fBSingle writer\fP \- responses are not written by the handler goroutine. The packed response is +pushed onto a bounded per\-socket stack (fixed depth 1024) and one writer goroutine per socket +performs the wire writes, so the file descriptor never sees more than one writer. The stack +evicts the oldest entry when full and the writer pops the newest first, so under overload the +socket's residual capacity always goes to the freshest response. The depth is a fixed burst +budget (roughly 12\-16ms of a typical socket's drain rate), not a tunable. +.IP \(bu 4 +\fBCoupled shedding\fP \- while a socket's stack is full, arriving queries on that socket are +dropped before any plugin runs; work admitted then would only produce a response destined for +eviction. There is no configuration: the stack's fullness is the signal. + + +.PP +Drops are silent \- no response is written, so the client's resolver retries against another +server, the standard load\-shedding contract for UDP DNS. Every drop is counted. + +.PP +The plugin only acts on UDP; TCP queries pass through untouched. It can only be used in plain DNS +server blocks (not \fItls\fP, \fIgrpc\fP, \fIhttps\fP or \fIquic\fP), which is enforced at startup. It should be +listed before (above) the \fIprometheus\fP plugin in the plugin chain, so that shed drops are never +counted as handled requests by the \fIprometheus\fP plugin \- which is where this plugin sits by +default. + +.PP +When several server blocks share a listener, any block with \fIshed\fP installs the write discipline +for every write on that socket, while the pre\-chain shedding only runs in blocks that carry the +directive \- keep it uniform across blocks sharing a listener. The discipline covers every response +written through \fB\fCWriteMsg\fR, which is how every plugin responds; a plugin writing raw bytes with +\fB\fCResponseWriter.Write\fR would bypass it. + +.SH "SYNTAX" +.PP +.RS + +.nf +shed + +.fi +.RE + +.PP +The plugin takes no arguments. + +.SH "METRICS" +.PP +If monitoring is enabled (via the \fIprometheus\fP plugin) then the following metric is exported: + +.IP \(bu 4 +\fB\fCcoredns_shed_dropped_total{server, reason}\fR \- counter of dropped queries and responses. The +\fB\fCreason\fR label is \fB\fCquery\fR for queries dropped before the plugin chain because the socket's +stack was full, and \fB\fCresponse\fR for responses dropped at the write boundary (evicted by a newer +response, failed to reach the wire, or arriving during shutdown). + + +.SH "EXAMPLES" +.PP +Protect the UDP listener while forwarding: + +.PP +.RS + +.nf +\&. { + shed + forward . 8.8.8.8 +} + +.fi +.RE + +.SH "SEE ALSO" +.PP +The fdMutex limit is enforced in \fB\fCGOROOT/src/internal/poll/fd_mutex.go\fR. +