『CoreDNS at 100K RPS: ndots, Negative Caching, and Autopath』のカバーアート

CoreDNS at 100K RPS: ndots, Negative Caching, and Autopath

CoreDNS at 100K RPS: ndots, Negative Caching, and Autopath

無料で聴く

ポッドキャストの詳細を見る
概要
 A real-world CoreDNS latency incident at high query volume reveals how ndots and Kubernetes search domains silently multiply DNS lookups into a full-blown query storm. You'll learn: Why a single external hostname like api.stripe.com generates 4 upstream queries under Kubernetes' default ndots:5 — and how that 4x amplifier compounds at scaleHow to override search behavior per-pod via dnsConfig.ndots, and why a trailing dot in your FQDN collapses four queries into oneTuning the CoreDNS cache plugin: setting denial TTLs high enough that NXDOMAIN responses actually stick, and the staleness tradeoff you must articulate in interviewsWhat the autopath plugin actually does differently — moving search-path resolution server-side — and when it helps versus when it shifts the problemWhy interviewers use this question to separate engineers who've read the docs from those who've debugged a DNS storm at 3am Keywords: CoreDNS tuning, Kubernetes DNS ndots, negative caching NXDOMAIN, CoreDNS cache plugin, Kubernetes networking interview 🎧 Listen, then go deeper — DevOps & Cloud interview-prep ebooks at DevOpsInterview.Cloud ▶ Daily 30-second interview drills: DevOps Interview Cloud on YouTube Transcript Picture this. Your cluster is doing something like a hundred thousand requests per second across a few hundred services, and suddenly every outbound call to a third-party API starts timing out intermittently. Nothing in your application changed. No deploy went out. But your p99 latency for anything that touches DNS resolution just tripled, and your CoreDNS pods are pegged at max CPU. This is a real failure pattern, and it almost always traces back to two things: how ndots is configured in every pod's resolver, and how CoreDNS is caching, or failing to cache, negative responses. Interviewers ask about this because it separates people who've read the Kubernetes docs from people who've actually had to explain a production DNS storm to their team. Anyone can say 'CoreDNS handles DNS in the cluster.' Far fewer people can explain why a single hostname lookup can turn into four or five actual queries hitting your DNS servers, and what that does to your query volume when you're already running near capacity. This question tests whether you understand resolution mechanics, not just that a service called CoreDNS exists. Here's the mental model. Every pod in a Kubernetes cluster gets a resolv.conf file, and that file has two important settings: a search list and an ndots value. The search list typically looks like namespace dot svc dot cluster dot local, then svc dot cluster dot local, then cluster dot local, and depending on your cloud provider, maybe an additional domain from the node itself. The ndots value, five by default in Kubernetes, tells the resolver: if the name you're looking up has fewer than five dots in it, don't treat it as fully qualified. Instead, try appending each entry in the search list first, in order, before falling back to treating it as an absolute name. Now think about what that means for a completely ordinary external lookup, something like api dot stripe dot com. That name has two dots. Since two is less than the ndots threshold of five, the resolver assumes it might be a relative name inside your cluster. So it tries api dot stripe dot com dot your-namespace dot svc dot cluster dot local first. That fails, comes back NXDOMAIN. Then it tries api dot stripe dot com dot svc dot cluster dot local. Fails again. Then api dot stripe dot com dot cluster dot local. Fails again. Only on the fourth attempt does it try api dot stripe dot com as an absolute name, with a trailing dot, and finally succeeds. So a single application-level DNS call just became four queries hitting CoreDNS, three of which were guaranteed to fail. Now multiply that across your fleet. If you're doing a hundred thousand real DNS-triggering requests per second, and a meaningful chunk of those are external hostnames going through this same four-query pattern, your actual query volume against CoreDNS isn't a hundred thousand queries per second. It's closer to three hundred or four hundred thousand, most of it wasted work resolving things that were never going to exist in your cluster domain in the first place. That's the amplification, and it's the single biggest reason CoreDNS falls over under load that looks survivable on paper. There are two levers here, and good candidates know both. First, ndots itself. You can override the search behavior per pod using the dnsConfig field in the pod spec, setting ndots to something lower, like one, for workloads that mostly talk to external services. Or, simpler and often better, you just get your application code or your service mesh sidecar to use fully qualified domain names with a trailing dot for external calls, which bypasses the search list entirely regardless of ndots. That trailing dot is doing real work. It tells the resolver 'this is already absolute, skip the search list,' and ...
adbl_web_anon_alc_button_suppression_t1
まだレビューはありません