14 October 2018

Sitecore Symposium 2018

Sitecore last week concluded just another wonderful Symposium. This Symposium was mix bag for me, just because Sitecore had promise lot last year while yet to deliver on few. While this had been echoed by many MVPs & customer alike.

So here I summarize key highlights, which are as follows:

Figure- 1.1
  1. Sitecore acquire StyleLabs : As per Sitecore, this will resolve the content crisis which is faced by their customers who had to depend upon other third parties for DAM, MRM, PIM etc., Notably SkyLabs is 100% SaaS solution build natively for Azure. Sitecore offered for their customer through this Acquisition. 

  2. Sitecore JSS will be available with Sitecore 9.1: Sitecore is scheduled to release Sitecore 9.1, with this release they will also release most awaited Headless Solution from last two years & after 4 Tech previews. Following are related notes with this subject:
    1. Sitecore Differentiate & make its case heard: In one of the keynotes "Headless CMS and the great uncoupling" by Deane Barker, Deane made following opinions, where few are right while few debatable.
    2. Figure 1.2
      1.  Most of Modern Headless CMS are Database at core with modern UI & API to serve. Headless CMS because they don't have burden of legacy they are good with this core principle of serving Content. No disagreement. 
      2. But performing any logic before serving content or adding value to that content by means of optimizations like adding context is left to solution developers. Deane is right here. But my only point here is, now atleast solution designer has choice what to choose & not everybody requires Logic or other suggested tiers. So overall headless is good.
        Figure 1.3
      3. This is true that over period of time due to urge of fulfilling Edge Cases, Capture Adjacent Markets or simply adding features most of CMS may end up with bloated feature library & implied cost, which make them look fatter in future from what they are now. Deane is right here. But my Hope is running in SaaS mode & committed to subscription based low pricing comes with inherited duty that Headless CMS vendor can't wander too far, they may have different offerings for different edge cases or markets. Anyway time will only tell who is right or wrong.
        Figure 1.4
        Figure 1.5
        Enabling customer, "Pay what they use is Mantra for every product vendor" or somebody else will come & disrupt market.
      4. Summary as per Dean is if Solution developer/Customer intend to use Contextual Content marketing then Customers should stick to Solution which provide that all. Which I agree but Sitecore need to be better, faster & cheaper.
    3. Sitecore JS Web Tracker : This tracker is rest API build on top of existing Web Tracker which provide Analytics Capabilities & offer personalized content(Not Sure here). This is different from Universal Tracker & yet not documented in current JSS documentation. 

  3. Other Sitecore 9.1 Updates: Sitecore made 9.1 kind of major release without disrupting versioning & saving next major version for Horizon & Zenith. Following are key updates:
    1. Performance Updates & Upgrades:
      Figure 1.6
       I am glad that Sitecore is on right path. Sitecore is improving performance of its Experience Management(Xm)  by 50% & reducing technical debt. They have promised to improve this further & take similar improvement to Xp also. They have also upgrade regular third parties they consume & made promise to do it earlier than usual.
    2. Figure 1.7
        1. Universal Tracker: It is Rest API to be consumed by any device or channel, extremely fast & scale-able. This will be tracker which will eventually replace current Web Tracker & intermittent JS Web Tracker. It has been built on using Sitecore Host. As of now in 9.1 it doesn't offer Content Personalization but will offer in future. 
        2. Figure 1.8
          1. Sitecore Identity Service: Finally Sitecore embraced modern Identity Management & shrugged of decades old Microsoft Membership Database. From 9.1 this is default login provider. This enabled Single Sign-On integration with third party provider more standardized solutions. This was on road for sometime as Commerce already had this.  
          2. Core DB is optional for CD: This nice & useful update will enable upcoming cloud adopters & current hosting solution provider lifer much easy.
          3. Sitecore Host:
            Figure 1.9
            This is pattern/technology which will enable most of new & upcoming/migrating services to enable services based world. Sitecore is re-architect-ing it's own solution to make it service oriented, which is easy to scale & host & technologically make it candidate to run as SaaS in future. 
          4. Sitecore Offers GrapQL Endpoint: This is another big announcement as this enable developers to write very precise queries to fetch Sitecore Data over wire. As of now this service only offers Content & Templates. But i hope that in future this offers more than content like layout service etc., 
          5. Sitecore Cortex:
            Figure 1.10
            To me this is like those bells & whistle which may make difference to Customer. Anyway, anything offered out of box is to delight if configured correctly. This uses Microsoft Cognitive Services & have capability to use custom Models. The only glitch I see here is how many enterprise today wan't to ship their consumer's personal & behavioral data to MS Cloud for getting insights, lot of such enterprise customers are either building their own serious ML gigs or already have it. This is interesting area to me & community. While checkout this article by Amitabh documenting Core concepts & collating articles on Cortex so far. 
          6. Sitecore Horizon:
            Figure 1.11
            Sitecore will be releasing Horizon as Technical Preview in Sitecore 9.1 while publicly admitting that it was harder than thought earlier to implement. Well this was expected. Anyway as community we are happy that Sitecore is pursuing this goal with not be just nice face-lift but as earlier indicated doing engine overhaul of Sitecore bit by bit so that next major release. The only question left is will be Next Sitecore Symposium 2019 will be that moment or will it be 2020 Symposium to get fully re-designed & re-invented. Last there was not enough mention of Zenith Project expect one time in MVP summit i heard which is it is still in development(it could be that I missed it). 

        3. Sitecore .NET Core: The effort which started 2 years back & exposed to us in 2016 MVP summit is yet not completed but it is on right path. I predict here it will take next 2 years to be fully based on .Net Core & become service oriented. This opens up many possibilities like hosting it on Linux Platform, Containarize it fully, run large complex robust & scalable Sitecore Solutions etc., Possibilities are endless. Sitecore is not to blame for slow progress but i admit it is overall technology improvement which is dependent on Microsoft & Open Source Community, at end Product Vendors are always later in queue to implement any new technologies. 
        Being developer I understand project delays, which are even sometime not in control of development team, so I have full belief on Sitecore that next Symposium will be best.
        Is this Sitecore Release is worth to upgrade from last stable release of Sitecore 8.2 ?
        • I would say wait till JFM of 2019 to finally commit to production, just because the sharp edges of 9.1 release are finished usually by an update.
        • But that doesn't mean, customers should wait for it, they must start early to test the risks(Through means of POC) & secure needed resources.
        Is JSS worth looking ?
        • Yes Headless is the way forward, if you wan't your development cost low & scale it better while making blazing fast Web Experience. 
        What are ways to implement JSS ?
        • Honestly there could be many possible ways out.
        • I will experiment with Sitecore(GraphQL Endpoint) + React + Gatsby + JS Web Tracker/Universal Tracker + Netlify + Cloudinary
        • Above Experiment, just depend upon stars to be in right position.
        Note: All images are content & property of Sitecore, they have been taken from Sitecore Symposium 2018 for purpose of summarizing the event. 

        27 November 2017

        Easy UnInstall Sitecore 9

        As being Sitecore developer, I do Sitecore Installations often. Earlier when we used to Install, it was relatively easy(1, 2, 3) to un-install Sitecore. Though it was never too easy, but still we can remove website by taking care of IIS node & couple of databases in local machine to get rid of old or corrupted Sitecore Instalations.

        Recently Sitecore launched new Instalation experience, called it SIF(Sitecore Instalation Framework), many(Kam, Toby) of our MVP colleagues has already blogged about it on how to quickly learn & do it yourself. But with this new way, it introduces many moving pieces. So now it contains SSL Certificates, Solr Cores, Windows Services, multiple IIS websites and these can be less or more depend upon the type of Instalation(Topologies) you choose.

        During my own learning, I failed with Sitecore 9 Instalations & end up having these zombies pieces like Certs, DB, Solr Core etc., I see this problem is not only with me, Kam did take a shot to it.

        Well, I am trying to bring my own solution for same problem. Following is to summarise my solution in quick points:

        • Create Uniform PowerShell Script(Cmdlet) for Instalation.
          • I take this script straight from Sitecore Instalation Manual & made little modifications.
        • As part of this instalation script generate 'AntiInstance' file in JSON format.
          • As part of this modification I write code to generate this JSON file.
        • With help of 'AntiInstance' file & another Powershell script Un-Install the Sitecore Instance.
          • This should be run & forget as a forceful removal.
        Key here are two PowerShell files which are still in WIP, you can find source code here, which I plan to complete over in nex few weekends.

        Following are quick steps to Install & then Un-Install Sitecore, Here deliberately i will fail the Installation to mock incomplete Installations. 
        • Dump all files in one folder(as shown in Figure 1.1), for required topology, in this example I used xp0.
          • This folder should includes license file & two Powershell files. Powershell files you should get from GitHub.
        Figure 1.1
        • Make sure that you use SitecoreInstall-Better.ps1 file for Instalation while updating all parameters stated at start of file(as shown in Figure 1.2).
          • Offcourse here, I assumed that you already have all pre-requisite(for example Solr on SSL) to Install Sitecore(Otherwise follow blogs stated earlier).
        Figure 1.2
        • Install Sitecore while firing up above file after due changes. 
        Figure 1.3
        • This should  result in In-Complete Installation.
        Figure 1.4
        • Now let's Un-Install this Sitecore Instalation, while using Un-InstallSitecore.ps1 file. It will confirm before deleting Website root folders, to be quick pass A for 'Accept All'.
        Figure 1.5
        • It should clear all mess which earlier Instalation script it left, for example certificate installed earlier deleted through this script as shown in Figure 1.6 & 1.7
        Figure 1.6

        Figure 1.7

        Again this is not yet production ready & take it at your own risk. I will try to complete it for xp0 topology and will seek feedback to take it further, if this is right way to uninstall for actual adoption.

        Let me know what you think about this, especially Kam.

        09 November 2017

        Install Tyk Gateway

        While creating NodeJS based services for one of the project, I realise we need an good open source API Gateway.

        By the way, what is API gateway ?
        "An API Gateway is a server that is the single entry point into the system. It is similar to the Facade pattern from object‑oriented design. The API Gateway encapsulates the internal system architecture and provides an API that is tailored to each client. It might have other responsibilities such as authentication, monitoring, load balancing, caching, request shaping and management, and static response handling" - Quoted from here.

        But you must be thinking, go and use AWS or Azure for hosting Services and they all come up with API mangement or gateway features. Our issue was PII data which we only host with trusted datacenter partner. So we need on-premise API Gateway which can scale if required & atleast free for initial instalation on production to tryout how it goes. 

        So we zoomed on Tyk after couple of options. We like it because of simplicty of its instalation, interface & easy to follow documentation. 

        In our case, Tyk Gateway will take inbound requests, run them through a set of middleware components which apply transforms and any other service-specific operations, and then proxy the request out again to the origin, intercepting the response, running a set of response middleware and then returning

        Sharing my quick instalation script for Tyk Gateway on RHEL, it may help somebody else. Script assume it is clean & fresh RHEL instance to which you have Sudo access, still in some place you may need to change or add additional commands as per your context.

        But  before starting with Gateway script following is script to setup machine quick


        Machine Setup Commands
        --------------------
        yum upgrade
        yum install -y NetworkManager-tui nano wget
        #use ip addr command to see the current network interface name then set it up to be used by network manager
        nano /etc/sysconfig/network-scripts/ifcfg-ens33
        #change the address to below    
        nmtui edit ens33
        #add the google DNS & local dns
        nano /etc/resolv.conf
        #add the local hostname
        nano /etc/hosts
        127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 dev.apigateway.com
        ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
        192.168.171.130 dev.apigateway.com
        192.168.171.130 portal.apigateway.com
        #restart the network service
        systemctl restart network.service
        ping dev.apigateway.com
        ping portal.apigateway.com
        sudo firewall-cmd --zone=public --add-port=27017/tcp --permanent
        sudo firewall-cmd --reload
        
        Now following is Gateway instalation code..
        STEP-A - Gateway Instalation
        --------------------
        TYK-Gateway Installation
        --------------------
        Step-1
        ------
        sudo firewall-cmd --zone=public --add-port=8080/tcp --permanent
        sudo firewall-cmd --reload
        sudo yum install pygpgme yum-utils wget
        Step-2
        ------
        nano /etc/yum.repos.d/tyk_tyk-gateway.repo
        #Copy & Paste below
        [tyk_tyk-gateway]
        name=tyk_tyk-gateway
        baseurl=https://packagecloud.io/tyk/tyk-gateway/el/7/$basearch
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        [tyk_tyk-gateway-source]
        name=tyk_tyk-gateway-source
        baseurl=https://packagecloud.io/tyk/tyk-gateway/el/7/SRPMS
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        Step-3
        ------
        #Make sure you refer latest epel version for example i changed it to epel 7-8 to 7-10
        cd GatewayInstall
        wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm
        sudo rpm -ivh epel-release-7-10.noarch.rpm
        sudo yum -q makecache -y --disablerepo='*' --enablerepo='tyk_tyk-gateway' 
        --enablerepo=epel info zabbix
        Step-4
        ------
        ##In order to allow your custom port from default linux firewalls
        sudo firewall-cmd --zone=public --add-port=6379/tcp --permanent
        sudo firewall-cmd --reload
        sudo yum install -y redis tyk-gateway
        sudo /opt/tyk-gateway/install/setup.sh --listenport=8080 --redishost=localhost 
        --redisport=6379 --domain=""
        Step-5
        ------
        sudo service redis start
        ----------------------
        
        
        
        STEP-B - Dashboard Instalation
        --------------------
        TYK-Dashboard Installation
        --------------------
        Step-1
        ------
        #ignore if already done
        yum install pygpgme yum-utils wget
        Step-2
        ------
        nano /etc/yum.repos.d/tyk_tyk-dashboard.repo
        #Copy & Paste below
        [tyk_tyk-dashboard]
        name=tyk_tyk-dashboard
        baseurl=https://packagecloud.io/tyk/tyk-dashboard/el/7/$basearch
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        [tyk_tyk-dashboard-source]
        name=tyk_tyk-dashboard-source
        baseurl=https://packagecloud.io/tyk/tyk-dashboard/el/7/SRPMS
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        Step-4
        ------
        nano /etc/yum.repos.d/mongodb-org-3.0.repo
        #Copy & Paste below
        [mongodb-org-3.0]
        name=MongoDB Repository
        baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.0/x86_64/
        gpgcheck=0
        enabled=1
        
        Step-5
        ------
        sudo yum -q makecache -y --disablerepo='*' --enablerepo='tyk_tyk-dashboard' info zabbix
        
        Step-6
        ------
        sudo yum install -y mongodb-org tyk-dashboard
        
        Step-7
        ------
        sudo firewall-cmd --zone=public --add-port=3000/tcp --permanent
        sudo firewall-cmd --reload
        sudo /opt/tyk-dashboard/install/setup.sh --listenport=3000 --redishost=localhost
        --redisport=6379 --mongo=mongodb://127.0.0.1/tyk_analytics --tyk_api_hostname=$HOSTNAME
        --tyk_node_hostname=http://localhost --tyk_node_port=8080 --portal_root=/portal
        --domain="apigateway.com"
        sudo service tyk-dashboard start
        #enter the license by browsing the url & restart the service
        sudo service tyk-dashboard restart 
        #Configuring Tyk Gateway with Dashboard
        sudo /opt/tyk-gateway/install/setup.sh --dashboard=1 --listenport=8080 
        --redishost=localhost --redisport=6379
        #Boostrap the Dashboard
        sudo /opt/tyk-dashboard/install/bootstrap.sh apigateway.com:3000
        
        TYK-PUMP Installation
        ---------------------
        Step-1
        ------
        nano /etc/yum.repos.d/tyk_tyk-pump.repo
        #copy paste below
        [tyk_tyk-pump]
        name=tyk_tyk-pump
        baseurl=https://packagecloud.io/tyk/tyk-pump/el/7/$basearch
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        [tyk_tyk-pump-source]
        name=tyk_tyk-pump-source
        baseurl=https://packagecloud.io/tyk/tyk-pump/el/7/SRPMS
        repo_gpgcheck=1
        enabled=1
        gpgkey=http://keyserver.tyk.io/tyk.io.rpm.signing.key
           https://packagecloud.io/gpg.key
        sslverify=1
        sslcacert=/etc/pki/tls/certs/ca-bundle.crt
        
        Step-2
        ------
        sudo yum -q makecache -y --disablerepo='*' --enablerepo='tyk_tyk-pump' info zabbix
        sudo yum install -y tyk-pump
        
        Step-3
        ------
        sudo /opt/tyk-pump/install/setup.sh --redishost=localhost --redisport=6379
        --mongo=mongodb://127.0.0.1/tyk_analytics
        sudo service tyk-pump start
        
        Following are quick service commands

        Service Start Commands for gateway
        -------------------
        service mongod stop
        service redis stop
        service tyk-gateway stop
        service tyk-dashboard stop
        service tyk-pump stop
        
        service mongod restart
        service redis restart
        service tyk-gateway restart
        service tyk-dashboard restart
        service tyk-pump restart
        service mongod start
        service redis start
        service tyk-gateway start
        service tyk-dashboard start
        service tyk-pump start
        -------------------
        
        After installing Mongo as default, i realised that this will not scale as we analytics portion of gateway will quickly eat GB's, so need to shift mongo data directory to its dedicated drive. While good thing with this gateway is even if mongo server is choked, API gateway still continue to work, which saved the day.
        #Better to change dbpath so that mongo don't run under default directory, assume here
        new dbpath to be is /data02/mongodb/
        #http://blog.carl.pro/2016/07/installing-mongodb-on-rhel-defining-a-custom-data-directory
        sudo service mongod stop
        cd /var/lib/mongodb
        cp -rf mongodb /data02/mongodb/
        cd /data02/
        sudo mkdir /data02/mongodb/
        cd /data02/mongodb/
        sudo chown -R mongod:mongod  /data02/mongodb/
        sudo chmod -R 700 mongodb/
        #Change dbpath to above example directory /data02/mongodb/
        nano /etc/mongod.conf
        #if SELinux enabled
        sestatus
        yum install policycoreutils-python
        semanage fcontext -a -t mongod_var_lib_t '/data02/mongodb/'
        restorecon -v '/data02/mongodb/'
        semanage port -a -t mongod_port_t -p tcp 27017
        sudo service mongod start
        # test if mongodb user can access new location:
        sudo -u mongodb -s cd /data02/mongodb/
        
        Will write more on how we develop our NodeJS service & share our learning.