Last year I posted on how we prevented subdomain takeovers and saved $000s.
But our 400 Bug Bounty researchers upped their game - so we did too, by adding optional automated subdomain takeover to our open source tool Domain Protect.
The problem
In my last post, you’ll see an explanation of subdomain takeover and how we developed Domain Protect to scan all AWS accounts in our organisation.
This is a real problem for most larger enterprises, for example a UK Department for Transport subdomain was recently taken over:
With Domain Protect scanning our Amazon Web Services (AWS) infrastructure every 24 hours, we identified and fixed a significant number of vulnerable subdomains, and the corresponding quantity of Bug Bounty findings - and cash paid out - reduced substantially.
We then developed Domain Protect for GCP to also scan our Google Cloud Platform estate, which detected more issues.
However new subdomain vulnerabilities can arise at any time, for example when an engineer deletes a S3 bucket but forgets to remove the corresponding DNS entry. Domain Protect runs once per day, with the on-duty security team member passing the issue via Slack or email to the owning team, who goes on to delete or fix the DNS entry. Typically this process might take a few days.
After a while, we noticed that some Bug Bounty researchers were quickly taking over OVO subdomains after new vulnerabilities arose, before they were even detected by Domain Protect, let alone fixed. We weren’t just focused on the money these Bug Bounty findings were costing us - we were also concerned that malicious attackers could take over vulnerable subdomains, damage OVO’s reputation and potentially use them for convincing phishing attacks.
We needed to up our game.
Introducing automated takeover in our security account
If we could take over our own vulnerable subdomains, before anyone else, we would prevent takeover by attackers and Bug Bounty researchers. So that’s exactly what we implemented:
Active takeover in Domain Protect currently supports DNS records in Amazon Route53 backed by:
- Elastic Beanstalk environments
- S3 buckets
Those tests which support active takeover now run much more frequently - so we detect new issues quickly, after which automated takeover typically takes a few minutes.
Managing automated takeovers
The system notifies the security team on successful or failed takeover
with daily reports on all takeover resources created within the central security account:
How automated takeover works in Domain Protect
We created a new serverless function to take over resources, and another for daily reports, and gave them appropriate permissions in the central security account.
When a new issue is detected, the code deploys a CloudFormation template to build the takeover resource, and tags it with details of the vulnerable domain. The daily reporting Lambda function searches the security account for CloudFormation stacks with Domain Protect tags.
After the owning team has removed or fixed the DNS record, security engineering deletes the takeover resource and CloudFormation stack, to tidy up and minimise costs.
What’s next?
Across OVO, not all of our DNS records are implemented in Amazon Route53 and Google Cloud DNS. So we’re planning to extend to other platforms.
We’re also constantly looking to improve Domain Protect, by adding new tests, and enhancing our automated takeover capabilities.