Magento 2 E-Commerce Marketplace on AWS

Context & Pain Points

A Magento 2 marketplace carries two kinds of load that pull against each other. Category and product pages are read constantly and change rarely, so they should almost never reach PHP. Cart, checkout and account pages belong to one shopper each and cannot be cached at all. The marketplace also ran a separate Symfony application next to the storefront. Both had to answer on the same domain, scale on their own, and ship without taking the other down. Product images were a third problem. Magento writes media to local disk by default. That breaks the moment there is a second web server, because an image uploaded through one instance does not exist on the others. On top of all that, any single server, zone or database instance that could take the store offline was a direct revenue risk.

What We Had To Solve

• Keeping PHP out of the read path. Category and product pages make up most storefront traffic, and rendering each one in Magento costs far more than serving a stored copy. The cache had to sit in front of both applications, not inside one of them.
• Running two applications on one domain. Magento and Symfony scale differently and ship on different schedules, but shoppers reach both through the same hostname. One entry point had to hand each request to the right fleet without tying their deployments together.
• Sharing media across a fleet that grows and shrinks. Magento writes product images to local disk. Add a second web server, or let Auto Scaling replace one, and images go missing on some instances unless every instance reads the same files.
• Surviving the loss of a zone. A store that loses its only database instance or its only web server stops taking orders. Every tier, including the cache, the sessions and the data, needed a copy in a second Availability Zone.
• Releasing to instances that come and go. Auto Scaling can launch an instance in the middle of a deployment, and that instance has to come up on the same build as the rest of its group.

How We Built It

• Placed an Nginx and Varnish proxy tier behind the main Elastic Load Balancer, with an instance in each Availability Zone. Cacheable pages are served from Varnish, and only cart, checkout and account traffic reaches PHP.
• Gave Magento 2 and Symfony their own internal load balancer and their own Auto Scaling group spanning both zones. The proxy tier sends each request to the right application, so either fleet can scale or be redeployed without touching the other.
• Moved product media onto Amazon EFS, mounted by every Magento and Symfony instance, and served it to shoppers through CloudFront so image traffic stays off the web tier.
• Ran Aurora with a writer in one zone and a standby in the other. An ElastiCache Redis cluster with a node in each zone holds sessions and the Magento cache. A shopper keeps their cart when the instance serving them is replaced.
• Built the release path on CodeCommit, CodePipeline and CodeDeploy, with build artifacts in S3. A push to the repository becomes a deployment to both Auto Scaling groups, and instances that scaling adds later pick up the same revision. CloudFormation defines the network and the stack, CloudWatch alarms notify the team through SNS, and order emails go out through Amazon SES.

Outcomes That Mattered