Feature request: Support different stages/environments by default

[DanielOrtel]

Is your feature request related to a problem? Please describe.
Deploying to different stages is not just a recommended practice, it is a required practice for most web applications, so it's pretty strange that this component doesn't support it out of the box. Not having a dedicated testing stage can easily create weird bugs on the live site which users visit regularly

I am aware that there is an example, but that needs you to add and maintain your own solution for handling stages, which is not ideal, since if there's a package update, you'll have to work around any breaking changes.

Describe the solution you'd like
A configurable option in the serverless.yml which creates a dedicated .serverless folder for the deployed stage.

MyNextApp:
  component: serverless-next.js
  inputs:
    stage: 'prod'

Describe alternatives you've considered
Rolling out my own solution based upon the example provided here. But then I ran into the same issue as mentioned in this issue. Correction: using a modified version of the example above, and committing the .serverless folder solved the issue I was experiencing and the process creates the dedicated Deploy.<environment>.<config>.jsonfiles.

[barrysteyn]

@DanielOrtel I created the example you referenced. Unfortunately, what you are asking for needs to be done by Serverless component infrastructure (i.e. the Serverless project needs to build this into their infrastructure). There are two problems with this: Firstly, it seems like the core team behind serverless components are not supporting the project anymore (as evidenced by looking at the component templates which have not been updated for months). Secondly, the way serverless did things, there is a component that loads the yml file (baked in the serverless.js library, called @serverless/core). That is the only infrastructure/mechanism that has the ability to create instances of state for each stage. All components are built on top of that, and (as mentioned), it has not been updated since it seems like it has been abandoned.

You can try asking the Serverless team directly to address your concerns by adding the functionality to @serverless/core. Good luck though! When I created the example, I had spent almost 3 weeks going back and forth asking for this, in the end, I was able to get agreement with core engineering that this was an acceptable solution. That was almost 8 months ago. Nothing since then!

[krzysztof-miemiec]

I created a simple component, so I could map branch name to a "stage" and then use variables defined per each stage:
https://gist.github.com/krzysztof-miemiec/a1a9e13d424ee71a1918e69af3334523

It supports simple interpolation in target.inputs section, but the rest of config has to be constant. Also, you can reuse it in other components.

Edit: I later found out that serverless remove does not work with my component - no fix from my side as I switched to Terraform-based solution.

[uclaeamsavino]

@DanielOrtel - I'm trying to work through this now. Can you share what you had to modify to get the example to work on your end?

Are you running this through a CI/CD flow? That's my biggest concern. If there's anything that needs to be checked in to .serverless or .serverless-next folders after running deploy - that won't work from inside a CI/CD build job. Even having to check in after running next build is less than ideal but maybe there's no way around it.

[krzysztof-miemiec]

I made some further research and found out serverless components are not feasible for production deployments (no automation is possible without storing state). From what I saw here this functionalities are not yet ready (definitely not if you don't use serverless cloud). I guess you could tinker with creating your own remote store, by doing something like:

# Before running serverless fetch remotely stored folders
aws s3 sync "s3://my-serverless-state-bucket/my-component/${STAGE}/.serverless" .serverless

serverless

# After running serverless sync it's folders back to S3 bucket
aws s3 sync .serverless "s3://my-serverless-state-bucket/my-component/${STAGE}/.serverless"

As we don't really have to do SSR, I ended up reusing frontend infrastructure I made some time ago with Terraform. It's much more stable, features proper tagging and feels more production-grade. You can wrap it with other AWS resources (like Hosted Zones, IAM roles/users for CI/CD). For Terraform-Serverless integration, I found a blog post: The definitive guide to using Terraform with the Serverless Framework

[DanielOrtel]

This is a new project that's starting up, so we don't have a CI release pipeline yet(that's what we're trying to figure out atm). @uclaeamsavino when I mentioned that it worked, I should've mentioned that this worked locally, not from a CI. For that, we're considering keeping the .serverless folder on S3 and syncing on deploy(committing back to the repo on deploy is also an option, though not a particularly good one).

But it looks like Terraform might be a better option, so I'm investigating that atm.

FYI: as far I could tell, you only need the .serverless folder, you don't need to commit the .serverless_nextjs one.

[uclaeamsavino]

@DanielOrtel - I'm using the method @krzysztof-miemiec describes and so far so good. We don't check in .serverless - but we do cache it in S3 for each environment. The build job syncs the folder from S3, runs the serverless command, then syncs it back to S3 when done.

Our buildspec.yml currently looks like this (I am still working on injecting the necessary environment variables into react at build time):

phases:
  install:
    commands:
      - echo CURRENT_ENVIRONMENT=${CURRENT_ENVIRONMENT}
      - npm install -g serverless
      - npm install

  pre_build:
    commands:
      - aws s3 sync s3://my-bucket/my-repo-name/${CURRENT_ENVIRONMENT}/.serverless .serverless --delete

  build:
    commands:
      - serverless
      - aws s3 sync .serverless s3://my-bucket/my-repo-name/${CURRENT_ENVIRONMENT}/.serverless --delete

  post_build:
    commands:

[hadynz]

@uclaeamsavino @krzysztof-miemiec How do you work around the different local paths referenced in the .serverless folder?

Does the serverless cli actually ignore them? Or did you have to look at setting up aliases, or ensuring that all developers on your teams and environments are using the same path?

[uclaeamsavino]

Each dev has their own unique copy of the .serverless folder, which never gets checked in.

[hadynz]

Each dev has their own unique copy of the .serverless folder, which never gets checked in.

That would mean that the only way to deploy to a particular environment will have to be from the chosen CI/CD environment (e.g. GitHub actions) as the .serverless folder that is stored in S3 will always have match code paths?

[hadynz]

If anyone is interested in a GitHub action workflow that uses @krzysztof-miemiec's solution check out this gist that works well for me.

[uclaeamsavino]

@hadynz = that's basically what our buildspec.yml looks like.

[bhall2001]

Based on some of the ideas here, I've now deployed 2 websites with a GitHub Actions process that deploys to both a staging and production environment using standard GitHub practices. It's working quite well and as a result I've created a repo that demonstrates how I set it all up.

At a high level when you commit to the master branch, "staging" is deployed while creating a release by tagging a commit with a version number (ie. v1.0.0) results in a production deployment.

It's not a trivial process for sure to setup and it does require an initial deployment then a config change for it all to work. But the change is pretty simple. After the initial deploy, it just works (quite nicely for me). Have a look and see if it might be something you can use.

serverless-nextjs-github-ci-cd

[dphang]

@bhall2001 great solution, I've been using a similar way to copy various serverless-*.yml files to serverless.yml to support multiple stages (I have 3), though this way uses a deployment script to execute npx serverless and the aws cli so it's not tied to GitHub Actions (although I am using GitHub Actions for my deployments).

Gist: https://gist.github.com/dphang/7395ee09f6182f6b34f224660bed8e8c

Feel free to use and adapt as needed.

[NoelBaron]

NOTE: This is not a production-ready solution

If you're looking to get started with a quick npm-script solution without CI/CD or github actions:

Create these files & folders:
.resources/dev/serverless.yml // your dev config
.resources/dev/.serverless // empty folder

Add these OSX-specific scripts to package.json:

"state:clear": "rm -rf .serverless && rm -rf serverless.yml && rm -rf .next && rm -rf .serverless_nextjs",
"state:load:dev": "yarn state:clear && cp -af .resources/dev/.serverless ./ && cp -af .resources/dev/serverless.yml ./",
"state:save:dev": "rm -rf .resources/dev/.serverless && cp -af .serverless .resources/dev",
"deploy:dev": "yarn state:load:dev && yarn serverless && yarn state:save:dev",
"serverless": "AWS_SDK_LOAD_CONFIG=1 AWS_CONFIG_FILE=.aws serverless"

This will allow you to run deploy:dev, which will perform the following tasks:

• replace serverless.yml and .serverless with stage-specific state.
• execute serverless (remove env variables if you're relying on global config)
• save the updated serverless state back into .resources/dev.

Add !/.resources/**/* to .gitignore to make sure state saves to your repo.