Deployment - CD Pipeline

Running deployments on your local machine works great during development, but let's be honest - what we really want is for these processes to happen automatically when we push our code. This guide explains how Continuous Deployment (CD) works in a multistore setup, helping you create an efficient and reliable deployment process that runs without manual intervention.

What You'll Learn

• How to set up CD pipeline using GitHub Actions
• How to detect and process only changed stores
• How to efficiently deploy multiple stores in parallel
• Advanced topics like change detection mechanism and CLI-driven approach

Core Concepts

What is CD in Multistore?

In a multistore setup, CD is designed to:

1. Automatically detect which stores have changed
2. Build and deploy only the changed stores

This targeted approach ensures:

• Faster deployments
• Efficient resource usage
• Less redundancy (unchanged stores are skipped in CD pipeline)

Continuous Deployment (CD)

The continuous deployment process ensures that your stores are built and deployed efficiently. Here's how it works:

Deployment Triggers

There are several ways to configure when your stores get deployed:

Automatic Deployment

Deploy automatically whenever code is pushed to the main branch:

on:
  push:
    branches: [main]  # Deploy on every push to main

Manual Deployment

Allow manual deployments with optional store selection:

on:
  workflow_dispatch:
    inputs:
      store_ids:
        description: 'Space separated list of store IDs to deploy'
        required: false  # When not provided, detect changed stores

You can use the automatic deployment for staging and manual deployment for production environment.

1. Building and Deploying Stores

The store deploy command handles both building and deploying your stores. There's no need to run a separate build step:

steps:
  - name: Deploy stores
    run: |
      yarn store deploy \
        --cloud-username ${{ vars.CLOUD_USERNAME }} \
        --cloud-password ${{ secrets.CLOUD_PASSWORD }} \
        $storeIdsFlag \
        --verbose

2. Parallel Deployment

For efficient deployment of multiple stores, use a matrix strategy:

jobs:
  deploy:
    strategy:
      matrix:
        store_id: ${{ fromJson(storeIds) }}
    steps:
      - name: Deploy store
        run: |
          yarn store deploy \
            --cloud-username ${{ vars.CLOUD_USERNAME }} \
            --cloud-password ${{ secrets.CLOUD_PASSWORD }} \
            --store-id ${{ matrix.store_id }} \
            --verbose

Advanced Topics

Understanding Change Detection

The change detection system is a core part of our CI/CD pipeline, implemented in the CLI, with yarn store changed command, to ensure consistent behavior across different CI platforms. Here's how it works:

The change detection system analyzes git differences and determines which stores are affected based on several rules:

1. Root Changes When files in the base applications change (e.g., /apps/storefront-middleware/src/index.ts), all stores inheriting from that application are affected, unless they override the changed file.
   Example: A change in base middleware:
   • Changed file: /apps/storefront-middleware/src/index.ts
   • Affected stores: All stores which don't have their own version of the file
2. Parent Store Changes When a parent store changes, all its child stores are affected, unless they override the changed file.
   Example: A change in parent store:
   • Changed file: /apps/stores/fashion-brand/middleware/index.ts
   • Affected stores: All stores which don't have their own version of the file
3. Direct Store Changes When files within a store directory change, only that store is affected.
   Example: A change in specific store:
   • Changed file: /apps/stores/sports-brand/middleware/index.ts
   • Affected store: only sports-brand
4. Dependency Changes Changes to dependency files (e.g., yarn.lock) affect all stores.
   Example: A change in dependencies:
   • Changed file: yarn.lock
   • Affected: All stores
5. Global Dependencies Changes to globally configured paths (e.g., shared packages) affect all stores. You can mark global dependencies for the yarn store changed command by adding the --global-dependencies flag. For example:

   # Mark all packages in the packages directory as global dependencies
   yarn store changed --global-dependencies="packages/**"
   

6. File Overrides If a store overrides a file from its parent or base application, changes to the original file won't affect that store.
   Example: A file override:
   • Changed file: /apps/stores/fashion-brand/middleware/index.ts
   • Skipped: fashion-brand-us (has its own version of the file)
   • Affected: Other child stores without overrides

The system provides detailed information about why each store was affected:

• STORE_CHANGED: Direct changes to the store
• ANCESTOR_CHANGED: Changes in parent store or base application
• PACKAGE_LOCK_CHANGED: Dependency file changes
• GLOBAL_DEPENDENCIES_CHANGED: Changes in globally configured paths

Why Parallel Store Deployments?

We recommend running deployments in parallel (one job per store) for several reasons:

1. Isolation
   • Each store deployment runs in its own environment
   • Failures in one store don't affect others
   • Resource limits are per-store, preventing one store from consuming all resources
2. Performance
   • Multiple stores deploy simultaneously
   • Overall deployment time is significantly reduced
3. Maintainability
   • Easy to retry failed deployments
   • Clear logs per store
   • Simpler debugging and monitoring

CLI-Driven Approach

Our CI/CD heavily relies on the CLI for several important reasons:

1. Platform Independence
   • The same commands work across all platforms
   • No need to rewrite logic for different platforms
   • Consistent behavior everywhere
2. Local Build and Development Parity
   • Developers can run the same commands locally
   • Easy to debug CI/CD issues
   • No surprises between local and CI environments
3. Encapsulated Logic
   • Complex operations are packaged in simple commands
   • CI configuration focuses on workflow, not implementation
   • Updates to deployment logic don't require CI changes

For example, instead of implementing store change detection in CI:

# Don't do this
- name: Detect changes
  run: |
    # Complex git diff logic
    # Parse file paths
    # Check store inheritance
    # Handle overrides

We use a single CLI command:

# Do this instead
- name: Detect changes
  run: yarn store changed --since $SINCE_SHA --to $TO_SHA

This approach makes it easy to implement our CI/CD pipeline in any CI system that can run shell commands.