When you navigate to your project in CircleCI's UI, Javascript from eight different analytics companies gets loaded and executed in your browser.
- Pusher
- Intercom
- Launch Darkly
- Amplitude
- Appcues
- Quora (??)
- elev.io
- Optimizely
You can see this in my Network tab here:
This is a problem because the CircleCI browser context has full access to the CircleCI API, which is hosted on the same domain, so all eight of those companies' scripts can make requests to CircleCI API endpoints. Furthermore CircleCI customers frequently either include credentials in source code or as environment variables in CircleCI. Set these, and you are trusting that CircleCI won't get compromised, or at least, your application is at most as secure as CircleCI is.
However, with eight different companies running Javascript in your browser with access to the CircleCI API, your source code and secrets are at most as secure as the union of eight different analytics companies' Javascript environments. If any of those eight gets compromised, it's trivial to execute Javascript that creates a new API token for your account. Once that token is created, an attacker can easily export it to a domain controlled by the attacker. Once an attacker has the token, they can use the "Test Commands" API to add new commands that will dump your environment variables and/or all files in source code to the logs, then download your logs or artifacts via the same API.
Analytics companies frequently get hacked, serve malware or otherwise contain vulnerabilities. Just last week it was revealed that "Code Hive" was running coin mining code on CBS's website. In 2014, jQuery.com was compromised and used to serve malware. If the same happened to any of the eight companies, you would be screwed.
This is frankly unacceptable for a company that manages source code and secrets for a large number of companies in the industry. It's unacceptable enough that your browsing data on CircleCI is potentially exposed to eight different companies, let alone API access to your source code. There are a number of steps you can take to mitigate the issue:
-
Put all domains used by the above servers in /etc/hosts, for each machine and each person on your team that accesses CircleCI. I have a tool that you can use to automate this process. However, this may break Javascript on CircleCI or other sites that are not coded in a defensive style.
-
Only use command line tools + API's to access CircleCI. I have written one such tool.
-
If there is an API to disable the "Test Commands" setting, turn it off, as an attacker can use that to put data in logs or artifacts without being able to control the circle.yml file or push to Github.
-
Don't put any sensitive keys in CircleCI, or in source code; inject them directly into the production runtime.
-
Demand that CircleCI take steps to make their dashboard and your source code more secure.
Finally, there are a number of steps CircleCI should enable immediately:
-
Don't load analytics scripts from eight different companies on pages that contain sensitive content, or that have access to the CircleCI "create token" API. Host the dashboard and API on a separate domain from the marketing page.
-
Send an email any time an API access token is created. Add a setting to allow org-wide disabling of API token creation.
-
Add an option to disable the "Test Commands" API, as this would allow an attacker without access to Github to place whatever content they want (source code/environment variables) in logs.
-
Add an option to delete old logs. If you have ever dumped env vars to the log file, an attacker can export these.
-
Enable subresource integrity, or serve JS from each of these companies from the CircleCI domain.
With embarrassing compromises literally every week, and new Congressional scrutiny into Facebook and Google's advertising practices in the 2016 election, as an industry we need to do better on issues like this.
Update: CircleCI posted their response.
Notes
Why don't you include a POC? I have one that demonstrates this attack, but I don't want to show script kiddies how. If you can't figure out how to construct it by reading the above description and the network traffic, you don't deserve to know how.
Liked what you read? I am available for hire.
So basically if you store credentials in your source code and use CircleCI you could get hit with this if one of those 8 providers got hacked? Is that a fair synopsis?
Yes
Can you compare and contrast this against any other online CI vendor? Like Atlassian Bamboo perhaps, or Travis?
I suspect they’ll all have similar linkages. Either you trust your source code out on the intertubes or you don’t.
Travis CI does not load any analytics tools, zero. Neither does Gitlab, according to the founder. https://news.ycombinator.com/item?id=15442746
“Either you trust your source code out on the intertubes or you don’t” is a fatalistic analysis which ignores that, at the margin, there is a lot that can be done to improve security and reduce risk
“This is a problem because the CircleCI browser context has full access to the CircleCI API”
That seems… unlikely at best. The Circle API request require authentication. They appear to be using cookies for this, so JS loaded from another domain would not be able to authenticate against the Circle API. I think you misunderstand the level of access externally loaded JS has to a page.
This isn’t correct, the Same Origin policy would prevent AJAX requests from circleci.com to mycrookedsite.com, but it will do nothing about circleci.com to circleci.com. It doesn’t matter where the JS is loaded from, otherwise CDN’s wouldn’t work. https://stackoverflow.com/a/12543984/329700 has more detail.
Any modern browser associates the scripts with their origin and wouldn’t let them send requests to CircleCI anyway as long as it’s protected properly on the Origin header, yeah?
Javascript doesn’t work that way.
Hi there,
Emma from CircleCI here.
We take security at CircleCI very seriously, and I wanted to share our thoughts + reply to this post in case anyone is interested:
https://discuss.circleci.com/t/circleci-response-to-kevin-burke-s-blog-post-about-third-party-services/17276
Thanks for all the thoughtful discussion on this topic.
I’ve been saying this to the guys at Codeship for a while, actually it’s why I didn’t sign up after the evaluation. It’s a really bad sign for the rest of their security efforts when they don’t lock down the back-end. This stuff is a massive potential attack vector, it should be secured properly, not treated as a playground. I’ve seen a number of other companies behaving like this, they should be named and shamed.
Laughing… while it wasn’t this particular exploit that lead to the security breach… a third-party vendor was exploited: https://support.circleci.com/hc/en-us/articles/360034852194-Security-Incident-on-8-31-2019-Details-and-FAQs-