Part-1 | Part-2
Our platform is based on our beloved e-CMS and Experience platform Sitecore.
For any deployment at high-level there are three key artifacts code, config & data. In usual plain ASP.NET project data means T-SQL scripts or database backups or if ORM is there, then entities/modals. But in case of Sitecore usually data exchanged between systems is in serialized form of Item which is fundamental unit of data in Sitecore. Because of this we have variety of developer tools which deals with Sitecore Serialization, popular among them is HedgeHog's TDS and Kam's Unicorn.
In early stages of project we use to manual way of serializing the Sitecore Items with help of Sitecore shell itself and de-serialize back into Sitecore using same process..but when we started doing that in bulk people do miss items which they change during development or other human mistakes which lead us to significant time to roll out changes for platform. So, we started using TDS which is convenient for developers(after training) and our item(data) deliverable were in form of update packages.
Another concern was deploying every time full build from code result in huge package which is concern from speed of deployment, as large update and code packages will take more time to deploy & debug if deployment fails. This issue is resolved by simply creating a tag in source control on each successful build and compare your current successful build with given build tag using little magic from PowerShell, PSake, PowerCore then generate a delta in package. This type of packages we called as Incremental Package. How to create such package would be subject for another article with code snippets.
There are large number of configuration files in Sitecore, for any development project that multiplies when you have multiple deployment environment like SIT, UAT, NNIT(Non Negative Impact testing), PROD etc., But it grows further when you have 10 different production environment running same code but with obvious different configuration. Here a environment means a collection of machines(it could be Sitecore instances too) CM/CD/Job server with there own dedicated databases. So in order to manage this situation there are two solutions one is TDS based environment project where you maintain build profile which mimic your number of environment and than replaces the config from relevant folder(you create copies of config in folder structure per build config) at time of build using appropriate build profile(Like Debug, Release, SIT, UAT etc.,). This approach work marvelously until you are limited to 5-6 configuration targets, but if you are running more than 10 targets then using Config Transformation is best approach and that can work in accordance with Sitecore patch mechanism.
In our deployment architecture we always use three minimum roles of Sitecore Core server first is CM(Content Management/Authoring Node), CD(Content Delivery) and Job Server(All active background jobs) before advent of publishing services and reporting services in Sitecore. Once you have CM/CD/Job split we need to deploy different configs and different data(update packages to different role). For example Code will be deployed to all server subsequently but after testing in CM. Configs will be taken care at deployment time due to use of config transformation technique.
Items, we only need to deploy in CM and that too only once as we have multiple CM servers. Due to this we need our Code and items packages to be separated. We don't compromise with this policy even for public Sitecore Module.
Later after testing Items will be published to first Internal Target to test and further to external target(Live URL for that environment).
Another concern was to analyses of quality of code team is contributing which can be done by employing right tool at time of build. We are using Sonar on same Jenkins server to analyze the code.
Over choice of Jenkins Vs Team City & Bamboo, we did choose Jenkins as economic(Open Source /Free), scale-able, flexible, robust product.
Over choice of multiple source controls we did choose on-premise version of BitBucket because of following reason:
Last we need a tool which can deploy our Nuget package to running Sitecore instances that at different roles.(CM/CD/Job). So we used Octopus Server which is perfect for .Net based Orchestrator. This allow us to deploy seamlessly and also open other opportunities like Sitecore environment creation automation, Environment Syncing from source, taking code and db backup and restoring whenever we need. We can employ many deployment practices whichever suits to project and team. My colleague Badal has written a detailed series already on how we used Octopus Server here.
Following diagram explain the summary of the Dev-ops process used for our Sitecore Platform.
You have notice so far that i am not going in detail for each of these tools and scripts because they are elemental part of it and there are already many articles out which explain already, but in case if you need to know anything specific after reading this article, please let me know, i will go through that in detail. But In next article it will be all about scripts and sample, as in next article we will discuss a new concept of SDM-(Sitecore Dependency Manager) which allows you to create a nugget package which will install all dependencies in server and in local Sitecore instance. This will help in managing same configuration(Dependency + Configuration) at server and at local seamlessly.
Update : Unfortunately Will not be able to release details of SDM as it was designed specifically for Customer and as per their policy can't make it public :(
Our platform is based on our beloved e-CMS and Experience platform Sitecore.
For any deployment at high-level there are three key artifacts code, config & data. In usual plain ASP.NET project data means T-SQL scripts or database backups or if ORM is there, then entities/modals. But in case of Sitecore usually data exchanged between systems is in serialized form of Item which is fundamental unit of data in Sitecore. Because of this we have variety of developer tools which deals with Sitecore Serialization, popular among them is HedgeHog's TDS and Kam's Unicorn.
In early stages of project we use to manual way of serializing the Sitecore Items with help of Sitecore shell itself and de-serialize back into Sitecore using same process..but when we started doing that in bulk people do miss items which they change during development or other human mistakes which lead us to significant time to roll out changes for platform. So, we started using TDS which is convenient for developers(after training) and our item(data) deliverable were in form of update packages.
Another concern was deploying every time full build from code result in huge package which is concern from speed of deployment, as large update and code packages will take more time to deploy & debug if deployment fails. This issue is resolved by simply creating a tag in source control on each successful build and compare your current successful build with given build tag using little magic from PowerShell, PSake, PowerCore then generate a delta in package. This type of packages we called as Incremental Package. How to create such package would be subject for another article with code snippets.
There are large number of configuration files in Sitecore, for any development project that multiplies when you have multiple deployment environment like SIT, UAT, NNIT(Non Negative Impact testing), PROD etc., But it grows further when you have 10 different production environment running same code but with obvious different configuration. Here a environment means a collection of machines(it could be Sitecore instances too) CM/CD/Job server with there own dedicated databases. So in order to manage this situation there are two solutions one is TDS based environment project where you maintain build profile which mimic your number of environment and than replaces the config from relevant folder(you create copies of config in folder structure per build config) at time of build using appropriate build profile(Like Debug, Release, SIT, UAT etc.,). This approach work marvelously until you are limited to 5-6 configuration targets, but if you are running more than 10 targets then using Config Transformation is best approach and that can work in accordance with Sitecore patch mechanism.
In our deployment architecture we always use three minimum roles of Sitecore Core server first is CM(Content Management/Authoring Node), CD(Content Delivery) and Job Server(All active background jobs) before advent of publishing services and reporting services in Sitecore. Once you have CM/CD/Job split we need to deploy different configs and different data(update packages to different role). For example Code will be deployed to all server subsequently but after testing in CM. Configs will be taken care at deployment time due to use of config transformation technique.
Items, we only need to deploy in CM and that too only once as we have multiple CM servers. Due to this we need our Code and items packages to be separated. We don't compromise with this policy even for public Sitecore Module.
Later after testing Items will be published to first Internal Target to test and further to external target(Live URL for that environment).
Another concern was to analyses of quality of code team is contributing which can be done by employing right tool at time of build. We are using Sonar on same Jenkins server to analyze the code.
Over choice of Jenkins Vs Team City & Bamboo, we did choose Jenkins as economic(Open Source /Free), scale-able, flexible, robust product.
Over choice of multiple source controls we did choose on-premise version of BitBucket because of following reason:
- We use larger suite of Atlassian tools like, for project management we use Jira & Agile Module while for documentation we use Confluence so BitBucket was a natural fit for us.
- Our project policy don't allow us to store code in cloud(SaaS) so we had On-Prem(On premise for us is non public cloud data center) instance of it.
- Our team needed feature branches concept and love the Git-Flow Model, so bit bucket was good fit for us.
Last we need a tool which can deploy our Nuget package to running Sitecore instances that at different roles.(CM/CD/Job). So we used Octopus Server which is perfect for .Net based Orchestrator. This allow us to deploy seamlessly and also open other opportunities like Sitecore environment creation automation, Environment Syncing from source, taking code and db backup and restoring whenever we need. We can employ many deployment practices whichever suits to project and team. My colleague Badal has written a detailed series already on how we used Octopus Server here.
Following diagram explain the summary of the Dev-ops process used for our Sitecore Platform.
| Figure 2.1 |
You have notice so far that i am not going in detail for each of these tools and scripts because they are elemental part of it and there are already many articles out which explain already, but in case if you need to know anything specific after reading this article, please let me know, i will go through that in detail. But In next article it will be all about scripts and sample, as in next article we will discuss a new concept of SDM-(Sitecore Dependency Manager) which allows you to create a nugget package which will install all dependencies in server and in local Sitecore instance. This will help in managing same configuration(Dependency + Configuration) at server and at local seamlessly.
Update : Unfortunately Will not be able to release details of SDM as it was designed specifically for Customer and as per their policy can't make it public :(
