Deployment process

Track the process

You can track build and deploy actions in real-time using the terminal or the Project Web Interface. The status displays in-progress, pending, success, or failed. You can view the logs in the interface.

If you are using external GitHub repositories, the log of the operations does not display in the GitHub session. You can still follow activity in their interface and in the Magento Commerce (Cloud) Project Web Interface.

Project configuration

A set of YAML configuration files located in the project root directory define your Magento installation and describe its dependencies. If you intend to make changes, modify the YAML files in your local branch. The build and deploy scripts access those files for specific settings.

For all Starter environments and Pro Integration environments, pushing your Git branch updates all settings and configurations dependent on these files. For Pro Staging and Production environments, you will need to enter a Support ticket. We will configure those environments using configurations from the Git files.

• .magento.app.yaml—defines how to build and deploy Magento. Enter specific build and deploy options to the hooks section.
• .magento.env.yaml—centralizes the management of build and deploy actions across all of your environments, including Pro Staging and Production, using environment variables.
• .magento/routes.yaml—defines how Magento Commerce processes an incoming URL.
• .magento/services.yaml—defines the services Magento uses by name and version. For example, this file may include versions of MySQL, PHP extensions, and Elasticsearch. These are referred to as services.

• app/etc/config.php—defines the system-specific settings Magento uses to configure your store. Magento auto-generates this file if it does not detect it during the build phase and includes a list of modules and extensions. If the file exists, the build phase continues as normal, compresses static files using gzip, and deploys the files. If you follow Configuration Management at a later time, the commands update the file without requiring additional steps.

  The app/etc/config.php file includes a scopes setting that defines how static files deploy during the build phase. Magento Commerce (Cloud) 2.1 supports the standard strategy only. Static file deployment takes a long time to complete, so doing it during the build phase reduces deployment and site downtime.

Required files for your Git branch

Your Git branch must have the following files for building and deploying in your local environment and to Integration, Staging, and Production environments:

• auth.json—in the root Magento directory. This file includes the Magento Authentication keys entered when creating the project. The file is generated as part of autoprovisioning a new project using a blank template. If you need to verify the file and settings, see Troubleshoot deployment.
• app/etc/config.php—auto-generated during the build phase if it does not exist.
• .magento.app.yaml—updated and saved to the root directory.
• .magento/services.yaml—updated and saved to magento/.
• .magento/routes.yaml—updated and saved to magento/.

Best practices for builds and deployment

We highly recommend the following best practices and considerations for your deployment process:

• Always follow the deployment process to ensure your code is THE SAME in Integration, Staging, and Production. This is vital. Pushing code from Integration environments may become important or needed for upgrades, patches, and configurations. This deployment overwrites Production and any differences in code in that environment.
• Always add new extensions, integrations, and code in iterated branches to then build and deploy using the process. Some extensions and integrations must be enabled and configured in a specific order due to dependencies. Adding these in groups can make your build and deploy process much easier and help determine where issues occur.
• Enter the same variables environment-to-environment. The values for these variables may differ across environments, but the variables may be required for your code.
• Keep sensitive configuration values and data in environment-specific variables. This includes an env.php file, CLI-entered variables, and Project Web Interface-entered variables. The values can differ, but having the variables is important.
• Test your build and deploy locally and in Staging before deploying to Production. Extensions and custom code work great in development. Some users push to production only to have failures and issues. Staging gives you an opportunity to fully test your code and implementation in a production environment without extended downtime if something goes wrong in Production.

Five phases of Integration build and deployment

The following phases occur in your local development environment and the Integration environment. For Pro plans, the code is not deployed to the Staging or Production environments in these initial phases. See Deploy code to Staging and Production.

Integration build and deployment consists of the following phases:

Phase 1: Configuration validation and code retrieval
Phase 2: Build
Phase 3: Prepare slug
Phase 4: Deploy slugs and cluster
Phase 5: Deployment hooks
Post-deployment: configure routing

For detailed instructions, see Build and deploy full steps.

Phase 1: Code and configuration validation

When you initially set up a project from a template, we retrieve the code from the the Magento Commerce template. This code repo is cloned to your project as the master branch.

• For Starter—master branch is your Production environment.
• For Pro—master begins as origin branch for the Integration environment.

You should create a branch from master for your custom code, extensions and modules, and third party integrations. We provide a full Integration environment for testing your code in the cloud.

When you push your code from your local workspace to the remote repository, a series of checks and code validation completes prior to build and deploy scripts. The built-in Git server checks what you are pushing and makes changes. For example, you may want to add an Elasticsearch instance. The built-in Git server detects this and verifies that the topology of your cluster is modified to your new needs.

If you have a syntax error in a configuration file, our Git server refuses the push. For details, see Protective Block.

This phase also runs composer install to retrieve dependencies.

Phase 2: Build

This phase builds the codebase and runs hooks in the build section of .magento.app.yaml. The default Magento build hook is the magento-cloud:build command and performs the following:

• Applies patches located in vendor/magento/ece-patches, as well as optional, project-specific patches in m2-hotfixes
• Regenerates code and the dependency injection configuration (that is, the var/generation and var/di directories) using bin/magento setup:di:compile.
• Checks if the app/etc/config.php file exists in the codebase. Magento auto-generates this file it does not detect it during the build phase and includes a list of modules and extensions. If it exists, the build phase continues as normal, compresses static files using gzip, and deploys the files, which reduces downtime in the deployment phase. Refer to Magento build options to learn about customizing or disabling file compression.

After the application builds, it is mounted on a read-only file system. You can configure specific mount points that are going to be read/write. You cannot FTP to the server and add modules. Instead, you must add code to your local repository and run git push, which builds and deploys the environment. For the project structure, see Local project directory structure.

Phase 3: Prepare the slug

The result of the build phase is a read-only file system referred to as a slug. In this phase, we create an archive and put the slug in permanent storage. The next time you push code, if a service did not change, it uses the slug from the archive.

• Makes continuous integration build faster by reusing unchanged code
• If code changes, updates the slug for the next build to reuse
• Allows for instantaneous reverting of a deployment, if needed
• Includes static files if the app/etc/config.php file exists in the codebase

The slug includes all files and folders excluding the following mounts configured in magento.app.yaml:

• "var": "shared:files/var"
• "app/etc": "shared:files/etc"
• "pub/media": "shared:files/media"
• "pub/static": "shared:files/static"

Phase 4: Deploy slugs and cluster

Now we provision your applications and all of the backend services you need:

• Mounts each service in a container (web server, Elasticsearch, RabbitMQ)
• Mounts the read-write file system (mounted on a highly available distributed storage grid)
• Configures the network so Magento services can “see” each other (and only each other)

Phase 5: Deployment hooks

The last step runs a deployment script, which you can use to anonymize data in development environments, clear caches, and ping external, continuous integration tools. When this script runs, you have access to all the services in your environment, such as Redis.

If the app/etc/config.php file does not exist in the codebase, static files are compressed using gzip and deployed during this phase. This increases the length of your deploy phase and site maintenance.

There are two default deploy hooks. The pre-deploy.php hook completes necessary cleanup and retrieval of resources and code generated in the build hook. The php ./vendor/bin/m2-ece-deploy hook runs a series of commands and scripts:

• If Magento is not installed, it installs Magento with bin/magento setup:install, updates the deployment configuration, app/etc/env.php, and the database for your specified environment, such as Redis and website URLs. Important: When you completed the First time deployment during setup, Magento Commerce was installed and deployed across all environments.

• If Magento is installed, performs any necessary upgrades. The deployment script runs bin/magento setup:upgrade to update the database schema and data (which is necessary after extension or core code updates), and also updates the deployment configuration, app/etc/env.php, and the database for your environment. Finally, the deployment script clears the Magento cache.

• The script optionally generates static web content using the command magento setup:static-content:deploy.

• Uses scopes (-s flag in build scripts) with a default setting of standard for static content deployment strategy. You cannot change the strategy in 2.1.x.

Post-deployment: configure routing

While the deployment is running, we freeze the incoming traffic at the entry point for 60 seconds. We are now ready to configure routing so your web traffic arrives at your newly created cluster.

Successful deployment removes the maintenance mode to allow for normal access and creates backup (BAK) files for the app/etc/env.php and the app/etc/config.php configuration files.

If you enabled static content generation using the SCD_ON_DEMAND variable and you configured the post_deploy hook, this clears the cache and pre-loads (warms) the cache after the container begins accepting connections and during normal, incoming traffic.

To review build and deploy logs, see Use logs for troubleshooting.

Build and deploy full steps

With an understanding of the process, we provide the following instructions for build and deploy for your local, Integration, Staging, and finally Production environments:

• Build and deploy to your local
• Prepare to deploy
• Deploy code and data
• Test deployment
• Go live and launch