At viki we like to release frequently as often as multiple times a day or atleast once a day. This can be challenging in an agile aggressive team1 where there is always lot of things going in parallel and everybody still commits to a single main-line of the repository. It requires a discipline to stick to continuous deployment and that is done by following a workflow that everybody in team is comfortable with. When features are released everyday, various teams (Marketing/Customer Support/Business) need to be aware of whats in the pipeline and how things are being rolled-out. In most companies, there is a deployment team which takes care of release managment + communication with rest of the world about what’s being released.
- Your Product Managers write feature/bug stories and prioritize them anytime during the day.
- Your engineering team believes in Continuous Integration and all engineers commit to Master branch (or Trunk) several times a day.
- If the build (CI) is green, that tag from Master Branch might get pushed to QA environment for stories to be delivered.
- Once all stories are accepted, (QA) Staging Tag is deployed to production, Happy Ending of the day!
- You have 5 Product Managers who request feature/bug stories and prioritize them anytime during the day.
- You have 10 engineers working on 5 stories and commits to Master branch (or Trunk) several times a day.
- Build is green only 70% of time during the day.
- By the time build is green there are commits to 4 finished & intermediate commits to an 1 un-finished story.
- QA delivers the stories(4) which has been finished and 3 are accepted and 1 is rejected.
- Among 3 accepted stories Marketing Team takes a call to hold 1 story even though its ready.
- Now we have 2 production ready, 1 held by marketing, 1 rejected, 1 un-finished.
- Commits related to 2 prod ready are shuffled between commits related to all non-ready stories.
- Among 2 ready-to-deploy , 1 is marked urgent but it’s commits are part of the day when CI build was RED (not necessarily due to this commit).
- What and how would you deploy today? (cherry-pick commits for a feature with no green build? FAIL) You post-pone release!
- ==Day Rolls Over==
- 3 new stories requested by Product Managers.
- An engineer finishes 2 of new stories quickly whose commits goes to master. 3rd new story might take long to finish, but gets its commits pushed to master.
- Build is green and a tag on master is pushed to QA environment.
- 2 new stories are delivered and accepted.
- Which staging tag on master would you deploy to production? At any given time there are commits from un-finished/un-delivered stories.
- Sure you can use feature toggle, but it only makes sense for long running (for weeks) set of stories. When every story starts to have a feature toggle, then system gets polluted with if-else everywhere, which again is difficult to manage and error prone
- You can also ask engineers to have separate feature branches for each stories and rebase with master often. This brings in its own over heads…
- Time spent in merging changes
- It needs a single controller of release branch who pulls the changes and makes sure what goes live is vetted. (This controller can soon become a bottleneck in the process)
- Engineers work in isolation and can not commit intermediate commits unless feature is complete.
- Use All-Accepted Marker : This is a commit on master below which all stories have been accepted and there are few commits (shuffled with other un-finished) above the marker that can be cleanly cherry-picked.
Workflow for All-Accepted Marker Deployment:
- Find a suitable commit below which all stories are accepted
- Branch out to new “Release” Branch
- Inform stake-holders about what features are being released and locked down release marker
- Cherry-pick related commits from above the marker to release branch
- Build the release branch and wait for it to be green
- Deploy release branch to production
- Automate this entire process
Deployment Pipeline is the tool that helps a deployment manager in above workflow
developers-machine:~/workspace/repository (master)$ pipeline help Tasks: pipeline help [TASK] # Describe available tasks or one specific task pipeline release_plan # Prepares a release plan pipeline setup # Setup Deployment Pipeline Tool pipeline status # lists all stories with their status pipeline suitable_release # Suggests a release commit to be picked and also includes a release plan Options: [--config=CONFIG] # A ruby file that defines relevant constants & configs. accepts ENV $PIPELINE_CONFIG # Default: /Users/dev_home/.pipeline_config
: Team which is motivated for release-often philosophy so much that it releases to production multiple times a day. It uses DVCS like Git and agile story tracker like PIVOTAL TRACKER. It has adopted TDD & Continious Integration as way of life. Every engineer commits to master all the time.