mirror of
https://github.com/coredns/coredns.git
synced 2026-08-20 23:08:28 -04:00
76056dd2e5
* plugin/shed: add UDP overload protection plugin
UDP responses written back through one listener socket serialize on the
Go runtime's internal fdMutex, which allows at most 2^20-1 concurrent
operations per file descriptor and panics the process when exceeded.
CoreDNS serves UDP with one goroutine per query, all writing through the
shared packet connection, so a sustained overload parks every excess
in-flight query in that wait queue until the process dies with
"too many concurrent operations on a single file or socket". Observed
in production: ~2.8M goroutines and 60GiB RSS before the panic.
The shed plugin makes the panic structurally unreachable. It installs,
via Config.UDPDecorateWriterFunc, a per-socket bounded evict-oldest
stack drained newest-first by a single writer goroutine, so the fd
never sees more than one writer and residual capacity under overload
always goes to the freshest response. While a socket's stack is full,
arriving queries are dropped before any plugin runs. Drops are silent
(the client's resolver retries elsewhere) and counted in
coredns_shed_dropped_total{server, reason}.
plugin/shed/fdmutex_test.go demonstrates the failure and the fix with
one shared flood harness. Two subprocess tests reproduce the exact
runtime panic without the plugin's write discipline - one deterministic
(a held write plus >2^20 queued writers), one with nothing held or
mocked; both exercise the Go runtime rather than the plugin, so they
are gated behind SHED_FLOOD_TEST=1. The counterfactual - the same load
through the plugin's stack, completing with every response accounted
for as written or dropped - runs in every test invocation, including
-race, at 50k responders, and at the full 1.5M with SHED_FLOOD_TEST=1:
SHED_FLOOD_TEST=1 go test ./plugin/shed/
Signed-off-by: Ryan Brewster <rpb@anthropic.com>
* test: add shed e2e test
Query a shed-enabled server over UDP (the plugin's deferred
single-writer path) and TCP (which shed passes through), and check
that coredns_shed_dropped_total is exported with its reason label.
No-Verification-Needed: test-only change
Signed-off-by: Ryan Brewster <rpb@anthropic.com>
---------
Signed-off-by: Ryan Brewster <rpb@anthropic.com>
68 lines
2.8 KiB
Go
68 lines
2.8 KiB
Go
// generated by directives_generate.go; DO NOT EDIT
|
|
|
|
package plugin
|
|
|
|
import (
|
|
// Include all plugins.
|
|
_ "github.com/coredns/caddy/onevent"
|
|
_ "github.com/coredns/coredns/plugin/acl"
|
|
_ "github.com/coredns/coredns/plugin/any"
|
|
_ "github.com/coredns/coredns/plugin/auto"
|
|
_ "github.com/coredns/coredns/plugin/autopath"
|
|
_ "github.com/coredns/coredns/plugin/azure"
|
|
_ "github.com/coredns/coredns/plugin/bind"
|
|
_ "github.com/coredns/coredns/plugin/bufsize"
|
|
_ "github.com/coredns/coredns/plugin/cache"
|
|
_ "github.com/coredns/coredns/plugin/cancel"
|
|
_ "github.com/coredns/coredns/plugin/chaos"
|
|
_ "github.com/coredns/coredns/plugin/clouddns"
|
|
_ "github.com/coredns/coredns/plugin/debug"
|
|
_ "github.com/coredns/coredns/plugin/dns64"
|
|
_ "github.com/coredns/coredns/plugin/dnssec"
|
|
_ "github.com/coredns/coredns/plugin/dnstap"
|
|
_ "github.com/coredns/coredns/plugin/erratic"
|
|
_ "github.com/coredns/coredns/plugin/errors"
|
|
_ "github.com/coredns/coredns/plugin/etcd"
|
|
_ "github.com/coredns/coredns/plugin/file"
|
|
_ "github.com/coredns/coredns/plugin/forward"
|
|
_ "github.com/coredns/coredns/plugin/geoip"
|
|
_ "github.com/coredns/coredns/plugin/grpc"
|
|
_ "github.com/coredns/coredns/plugin/grpc_server"
|
|
_ "github.com/coredns/coredns/plugin/header"
|
|
_ "github.com/coredns/coredns/plugin/health"
|
|
_ "github.com/coredns/coredns/plugin/hosts"
|
|
_ "github.com/coredns/coredns/plugin/https"
|
|
_ "github.com/coredns/coredns/plugin/https3"
|
|
_ "github.com/coredns/coredns/plugin/k8s_external"
|
|
_ "github.com/coredns/coredns/plugin/kubernetes"
|
|
_ "github.com/coredns/coredns/plugin/loadbalance"
|
|
_ "github.com/coredns/coredns/plugin/local"
|
|
_ "github.com/coredns/coredns/plugin/log"
|
|
_ "github.com/coredns/coredns/plugin/loop"
|
|
_ "github.com/coredns/coredns/plugin/metadata"
|
|
_ "github.com/coredns/coredns/plugin/metrics"
|
|
_ "github.com/coredns/coredns/plugin/minimal"
|
|
_ "github.com/coredns/coredns/plugin/multisocket"
|
|
_ "github.com/coredns/coredns/plugin/nomad"
|
|
_ "github.com/coredns/coredns/plugin/nsid"
|
|
_ "github.com/coredns/coredns/plugin/pprof"
|
|
_ "github.com/coredns/coredns/plugin/proxyproto"
|
|
_ "github.com/coredns/coredns/plugin/quic"
|
|
_ "github.com/coredns/coredns/plugin/ready"
|
|
_ "github.com/coredns/coredns/plugin/reload"
|
|
_ "github.com/coredns/coredns/plugin/rewrite"
|
|
_ "github.com/coredns/coredns/plugin/root"
|
|
_ "github.com/coredns/coredns/plugin/route53"
|
|
_ "github.com/coredns/coredns/plugin/secondary"
|
|
_ "github.com/coredns/coredns/plugin/shed"
|
|
_ "github.com/coredns/coredns/plugin/sign"
|
|
_ "github.com/coredns/coredns/plugin/template"
|
|
_ "github.com/coredns/coredns/plugin/timeouts"
|
|
_ "github.com/coredns/coredns/plugin/tls"
|
|
_ "github.com/coredns/coredns/plugin/trace"
|
|
_ "github.com/coredns/coredns/plugin/transfer"
|
|
_ "github.com/coredns/coredns/plugin/tsig"
|
|
_ "github.com/coredns/coredns/plugin/view"
|
|
_ "github.com/coredns/coredns/plugin/whoami"
|
|
)
|