iOS CI Broken due to Personal Accounts used to build the client.
Postmortems | Links:
LESSON LEARNED: Personal private accounts should never be used inside the Organization we are working for. We need to be more strict with knowledge transfer. Take ownership and more responsibilities over the code we write.
“Ultimately there is no such thing as failure. There are lessons learned in different ways.”
Platform/Client
- iOS - Bitrise CI
Context Situation
At the time of this writing, our iOS Client is using Bitrise for Continues Integration. The app builds using this CI service and also performs some checks, which we rely on for PR Reviews, Testing and Deployments.
Basically the CI builds the iOS Client from a Github Repository, first including the main application module (from the main code repository) plus a sub-module (it is a private repository called POD on iOS terminology). This last point is IMPORTANT to understand the route cause of the issue.
Issue Summary
At the organization, since we were using a Legacy Subscription Plan on Github, there was a need to upgrade it and also do some clean up by removing legacy accounts and former contributors.
Once these maintenance tasks were performed, the iOS app was not building anymore.
We were using former employees and personal private accounts at CI level for building the iOS Client.
There was neither knowledge nor documentation, on how things were set up and running.
Impact
2 days of having to manually build the application and not being able to distribute it internally, which caused a big impact on developer’s productivity.
Resolution
In order to solve the mentioned issue we had to acquire knowledge on the fly (exploratory approach) and read the documentation, which stated:
IMPORTANT info to consider!: “Registering the same SSH key for every repository is the best, most secure way - but not all services support it. GitLab and Bitbucket does support it, GITHUB, however, DOESN’T. If your code is stored on GITHUB, read on!”
** GITHUB, however, DOESN’T **
These are the conclusions which lead to an effective resolution of the problem:
- A common account for the CI Service (CiBot user) must be present as a user with read privileges on both repositories: the main one and the sub module one.
- This common account (CiBot user) should be added as QA user on Bitrise.
- Bitrise Settings of the project must be set up in such a way that is possible for this common user (CiBot user) to have a “public ssh key” for both repositories (main module and sub module).
Lessons Learned
- We need to be more strict with knowledge transfer (Bus Factor) and document how things work, at least the big picture.
- Take ownership and more responsibilities over the code we write and the ecosystem around it.
- Personal private accounts should never be used in order to avoid coupling outside the organization we are working for.
