[LINK] RFI: Telstra DNS outage

Sun May 15 09:46:52 AEST 2016

Thanks Geoff, that clarifies quite a few things.

One query remains though:  if resolvers stopped responding, resulting in the service dying, surely that still means that there was a single-point-of-failure - although in a different part of the system from where I was inferring.

__________________________________________

At 9:23 +1000 15/5/16, Geoff Huston wrote:
>Hi Roger,
>
>Yes you are making unfounded accusations here based on poor evidence and insufficient analysis.
>
>Firstly, you are confusing resolvers and authoritative name servers. The article you quote was about Telstra's resolvers not answering DNS queries from Telstra customers. I.e. Telstra's resolvers stopped responding. Your note looks at the authoritative name servers for the telstra.net domain.
>
>Secondly, you should've seen that two of the four servers are operated by APNIC, rather than Telstra. So there is no single point of name failure in serving telstra.net
>
>Thirdly, in the DNS too much is sometimes as bad as too little. More servers for a name can cause slower responses to resolution requests in some cases. Telstra's design of its server infrastructure, using 2 organizations and 4 server addresses looks like a good decision.
>
>Fourthly you are inferring way too much from the IPv4 address. I have not bothered to check but that fact that these are numerically adjacent addresses still permits the possibility that these are the addresses of two anycast clouds and there many be a number of servers that respond to the same address. It may also be the case that the internal routing infrastructure treats these as distinct /32s and they may well be provisioned using diverse internal paths.
>
>I would hesitate to hurl around accusations of "utter incompetence" in this case. I would tend to say that the server design for serving 'telstra.net' looks like decent service engineering, and the "problems" you appear to identify may well reflect your understanding of DNS and network engineering.
>
>
>Regards,
>
>    Geoff

__________

>> On 13 May 2016, at 09:02, Roger Clarke <Roger.Clarke at xamax.com.au> wrote:
>> 
>> itNews reports:
>>> Telstra suffered a nationwide network outage last night, as two of its internet domain name servers ceased to respond to queries from thousands of customer systems.
>> 
>> Am I missing something here?
>> 
>> I've chastised small-time ISPs in the past for having both or all of their DNS-servers on the same sub-net and therefore (under IPv4 at least) subject to the same threats.  They thereby represent a single-point-of-failure, rather than the redundancy that is the whole point of having >1 DNS-server
>> But Telstra currently shows
>> telstra.net.        NS    dns1.telstra.net.
>> telstra.net.        NS    sec1.apnic.net.
>> telstra.net.        NS    sec3.apnic.net.
>> telstra.net.        NS    dns0.telstra.net.
>> 
>> dns1.telstra.net.    A    203.50.5.200
>> dns0.telstra.net.    A    203.50.5.199
>> 
>> Is the largest provider in the country utterly incompetent?
>> 
>> Or is there something important about Internet architecture that I fail to understand?
>> 
>> ______________
>> 
>> Telstra DNS outage causes customer grief
>> By Juha Saarinen on May 13, 2016 6:51AM
>> Two-hour interruption to services.
>> http://www.itnews.com.au/news/telstra-dns-outage-causes-customer-grief-419496
>> 
>> Telstra suffered a nationwide network outage last night, as two of its internet domain name servers ceased to respond to queries from thousands of customer systems.
>> 
>> Two Telstra name servers used by customers for domain resolution, ns0 and ns1.telstra.net, went offline just after eight o'clock last night, users reported.
>> 
>> Domain name system servers are used to look up and point client systems to the correct IP address for human readable URLs such as www.telstra.net.
>> 
>> Without working DNS resolution, web browsers and other applications are unable to locate the IP address of the server they need to communicate with.
>> 
>> The name servers appear to have come back up around 11pm yesterday.
>> 
>> Telstra's service status web page made no mention of the DNS server problem.
>> 
>> While many Telstra customers took to Twitter and Facebook to complain about the outage, the telco did not confirm the service interruption until this morning, when it said the issue had been dealt with.
>> 
>>    @crakd67 Sorry for the delay in replying - the DNS issue has since been resolved - Steph
>>    - Telstra (@Telstra) May 12, 2016
>> 
>> iTnews has contacted Telstra for comment on the outage.
>> 
>> The telco earlier this month pledged to pour an extra $50 million into its mobile network after a series of damaging outages in the early months of this year.
>> 
>> 
>> -- 
>> Roger Clarke                                 http://www.rogerclarke.com/
>>                        
>> Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA
>> Tel: +61 2 6288 6916                        http://about.me/roger.clarke
>> mailto:Roger.Clarke at xamax.com.au                http://www.xamax.com.au/
>> 
>> Visiting Professor in the Faculty of Law            University of N.S.W.
>> Visiting Professor in Computer Science    Australian National University

-- 
Roger Clarke                                 http://www.rogerclarke.com/

Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916                        http://about.me/roger.clarke
mailto:Roger.Clarke at xamax.com.au                http://www.xamax.com.au/ 

Visiting Professor in the Faculty of Law            University of N.S.W.
Visiting Professor in Computer Science    Australian National University