Last Friday at around 14:05 we noticed that our website (www.scrt.ch) along with some other services we use internally were no longer accessible. We immediately tried to figure out why that was and quickly noticed that our DNS requests were not returning the correct IP addresses.
As these responses were not only returned by our DNS server, but all servers globally, we had a short moment of anguish where we wondered whether we had forgotten to renew our domain name or if our account at our registrar had been hacked!
Thankfully none of these had happened as we observed that the DNS configuration at the registrar and in our name servers was not altered. So at 14:17 we contacted nic.ch to ask them whether they could tell us any more about what had happened. They informed us that they had received a change request from our registrar (Gandi) and that we should check with them. Following further discussion with them, they found that similar changes had been requested for 94 .ch and .li domain names.
We then contacted Gandi directly who told us that 751 of their domain names had been hijacked and they subsequently started to rollback the changes. By 15:05, everything was back in order, though we still haven’t received any news as to what made all of this possible in the first place.
Here we are
The domain scrt.ch is registered at Gandi where we configure the IP addresses of our name servers. Gandi is responsible for propagating this information to nic.ch, enabling the resolution for our domain scrt.ch globally.
Last Friday, an attacker was able to compromise a technical provider used by Gandi to communicate with various TLD registries. This compromise allowed him to request changes to registries, including nic.ch, to modify the name server information for several domains, including ours.
At this point, a rogue DNS server was introduced in the DNS resolution path. Looking at the DNS resolution process described above, the hijack happened in step 3 where nic.ch was providing the rogue DNS server instead of the valid one to any resolver, allowing the attacker to redirect any requests for impacted domains to IP addresses owned by the attacker himself.
Gandi’s incident notification can be found here.
On our end, the impacts this attack had were:
- For an hour, the www.scrt.ch website was redirected to another host serving up an exploit kit that would try to infect vulnerable browsers. However, if you had visited our site previously, you would not have been affected as HTTP Strict-Transport-Security would have forced your browser to use a valid HTTPS which the attacker could not emulate, therefore resulting in a connection error. The only case where you might have been at risk is if you had never visited our site previously and you had an outdated vulnerable browser (and/or browser plugin), though if this is the case, attackers would probably not wait for you to visit our website to compromise you.
- During the attack, all emails sent to scrt.ch addresses were not delivered to our server but to a foreign mail server which, according to tests performed during the attack, was not configured to accept emails for our domain. While it doesn’t seem to have been the objective here, this type of attack could potentially be used to read incoming emails while the name servers are poisoned.
In our case, we have always treated emails as inherently insecure and make sure to use other channels and encryption to transfer sensitive information.
Despite the fact that this incident was entirely out of our control, we will be taking steps to further reduce the impact such an attack could have in the future and of course recommend these to all of you reading this, as such attacks can happen to anyone:
- Preload Strict-Transport-Security into browsers for our website in order to protect even first-time visitors.
- Monitor DNS resolution by not only querying authoritative name servers directly but by crawling the full hierarchy.
- Discuss with nic.ch whether detection of such events could be improved. Having over 90 domains owned by different entities changed at the same time to point to a single name server is really suspicious.
- Implement DNSSEC. While in this case the attacker could have been able to either suppress or update the DS record in the TLD zone, the cache mechanism would have mitigated the attack until DS record TTL’s expiration, as confirmed by a discussion with SWITCH-CERT.