CodeCollab Handbook

Appearance

Return to top
On this page

Setting up CI/CD for your project

Currently, all pipelines (except frontends) use GitLab CI. Your pipelines should handle:

  • Linting
  • Testing
  • Docker build and push
  • Deploy
  • Cleanup

The config should handle the above to both QA and Release environments.

GitLab CI configs vary by language, here’s an example for Go: https://gitlab.com/codecollab-io/cc-projects-service/-/blob/main/build/ci/.gitlab-ci.yml

Initial deployment

When your project is ready for the first deployment, you have to run all stages of the pipeline except Deploy and Cleanup. This will build your project and make its image available on Artifact Registry. After which, you can apply the Terraform plan to create the new Cloud Run instance. Deployments after this will work as expected.

Configuring credentials (repository)

Often, your pipelines will need to read from another repository (especially for our Go projects). There are two CI/CD variables available to do that:

  • CI_DEPLOY_USERNAME
  • CI_DEPLOY_TOKEN

After you include these into a .netrc file, you’ll be able to read from repositories across the CodeCollab group:

bash
echo -e "machine gitlab.com\nlogin ${CI_DEPLOY_USERNAME}\npassword ${CI_DEPLOY_TOKEN}" > ~/.netrc

For more advanced scenarios where you need to push to repositories, you can make use of a Deploy Key. Current this is only used for https://gitlab.com/codecollab-io/protobuf/cc-protos as it needs to update stub repositories. If you need to use a Deploy Key, please @Qin Guan.

Configuring credentials (GitLab Registry)

Your pipelines will need to pull and push images to the GitLab Registry. GitLab provides three CI/CD variables to do that:

  • CI_REGISTRY_USER
  • CI_REGISTRY_PASSWORD
  • CI_REGISTRY

You can include this in a before_script for your Docker push pipeline. For example:

bash
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

Configuring credentials (Google Cloud)

Your pipelines will need to push images and deploy to Cloud Run. To do that, a service account is provided. The service account has the following permissions:

  • artifactregistry.tags.delete
  • artifactregistry.versions.delete
  • iam.serviceAccounts.actAs
  • run.services.create
  • run.services.get
  • run.services.update

Two CI/CD variables are provided to use the service account:

  • CI_SERVICE_ACCOUNT
  • CI_SERVICE_KEY

The following script will authenticate with Google Cloud and configure Docker:

bash
gcloud config set project codecollab-io

echo $CI_SERVICE_KEY > key.json
gcloud auth activate-service-account $CI_SERVICE_ACCOUNT --key-file=key.json
gcloud auth configure-docker "${ARTIFACT_REGISTRY_REGION}-docker.pkg.dev"

Depending on which region your Artifact Registry is, you may want to replace line 5 with the correct region.

Testing

Try to include JUnit test reports in your pipeline and export the generated XML file as an artifact. GitLab will process this automatically. For your project, also set up Test Coverage Parsing, which varies by language.

If running your tests requires a connection to an external service, make a config file for it and utilize GitLab CI services. Refer to the GitLab CI config for cc-projects-service.

Build and Push

The build and push stage must push the final image to both GitLab Container Registry and Google Artifact Registry. Google Artifact Registry is used as a connector between GitLab Container Registry and Cloud Run. No images should be stored there after the deployment is complete. It must be removed in the clean up stage.

For build, since our CI/CD is quite simple, just include the production Dockerfile in the repository and build from that. For development, you can include a Dockerfile.dev.

Configuration files

If you require a config/env file (you should), do not include it with the image. Instead, use Secrets Manager. This is configured in Terraform and Google Cloud.

Since GitLab’s build minutes are still free for us now (and also because we can run our own GitLab CI Runners) we can use that for now.

To build an image with GitLab CI, use Docker in Docker. The GitLab runner does not seem to work with TLS, so you’ll have to disable it in your GitLab CI config. For example:

yaml
variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_TLS_CERTDIR: ""
image:
  name: gcr.io/google.com/cloudsdktool/cloud-sdk:latest
services:
  - docker:dind

If building your image requires reading from other repositories, create a .netrc file in the GitLab CI job, and pass it as a --build-arg to docker build. For example, in gitlab-ci.yml:

yaml
before_script: &configure_credentials
  - echo -e "machine gitlab.com\nlogin ${CI_DEPLOY_USERNAME}\npassword ${CI_DEPLOY_TOKEN}" > ~/.netrc
  - export GOPRIVATE="gitlab.com/codecollab-io"
script:
  - docker build -f Dockerfile --build-arg CI_NETRC="$(cat ~/.netrc)" -t $CONTAINER_QA_IMAGE -t $ARTIFACT_REGISTRY_QA_IMAGE .

Then in Dockerfile:

docker
ARG CI_NETRC
ENV GOPRIVATE=gitlab.com/codecollab-io
RUN echo "${CI_NETRC}" > ~/.netrc

The image must be tagged accordingly:

  • QA - test
  • Release - $CI_COMMIT_TAG

Using scratch

If you are building from scratch, remember to disable dynamic linking. For Go: use CGO_ENABLED=0

If you are building from scratch and require connecting to MongoDB, you need to add `ca-certificates:

docker
FROM alpine:latest as certificates

RUN apk --no-cache add ca-certificates

FROM scratch as runner

COPY --from=certificates /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

After the image is built, push it to GitLab’s container registry and Artifact Registry. The build and push should be under the same job. To push, you will need to be authenticated with GitLab Registry, refer to the section above on how to authenticate. You will also need to be authenticated with Google Cloud, refer to the section above as well.

Deploy

Your project should have two deployments:

  • Release - when a new tag is created
  • QA - when a new commit is pushed to the test branch

A new revision of the service can be created on Cloud Run with:

bash
gcloud run deploy $CLOUD_RUN_QA_SERVICE_NAME --image $ARTIFACT_REPOSITORY_QA_IMAGE --region $CLOUD_RUN_QA_SERVICE_REGION

Cleanup

After the deployment has completed, your pipeline needs to clear up the images pushed to Artifact Registry:

This is no longer necessary as it screws up redeployments from Terraform when Cloud Run tries to find the image.

bash
gcloud artifacts docker images delete $ARTIFACT_REPOSITORY_QA_IMAGE

Frontend projects

Frontend projects are deployed on Netlify. The CI/CD process is not handled by GitLab CI. Contact @Carl Voller or @Qin Guan if you’re working on a frontend project.