Internal Steps

Internal steps are developed by Wercker and are baked into the Wercker CLI. Internal steps interact with the Docker API that is external from the container. From a technical perspective, it is not possible to interact with the Docker daemon from within a container. This is why we created these internal steps.

This article explains the different internal steps that are available:

internal/docker-push

The internal/docker-push step can be used in two ways.

You can use it to push an image that has been built using the internal/docker-build step to a Docker registry. You must set the image-name property to identify the image you wish to push, as shown in the following example. This example uses Docker Hub; but you can use any image registry.

- internal/docker-push:
    # specify the image to be pushed - this is the one we created earlier
    image-name: my-new-image 
    username: $USERNAME # Registry username  
    password: $PASSWORD # Registry password
    registry: https://hub.docker.com
    repository: $USERNAME/docker-build-golang

Alternatively, you can use this step to create a new image from the currently running container, and immediately push that image to an image registry. In this case the image-name property is not set.

The internal/docker-push step takes the current pipeline image (specified either globally or per pipeline) in its current state and pushes that as an image to a Docker registry. This includes the result of all the Wercker steps that have been run up until that point.

deploy:
  steps:
    - internal/docker-push:
        username: $USERNAME
        password: $PASSWORD
        repository: turing/bar
        tag: my-amazing-tag
        ports: "5000, 8080"

The internal/docker-push step supports the following properties:

  • image-name: This property is only used when pushing an image created previously using a docker-build step. It indicates the name of the image to be pushed. If you choose to set this property, then specify the same name that you specified while running the internal/docker-build step. However, if you omit this property, then the currently running pipeline container is committed and pushed. The internal/docker-push step supports the following properties:

You can use the following properties irrespective of whether the image-name property is specified or not:

  • username: Indicates the user name that is used to authinticate the registry.
  • password: Indicates the password that is used to authinticate the registry.
  • email: Indicates the e-mail address that is used to authinticate the registry.
  • repository: Indicates the name of the repository. If you are using a repository other than Docker Hub, prefix the value with the hostname of the private repository.
  • tag: A space- or comma-separated list of Docker tags that are applied to the built container. Wercker automatically applies a tag with the build number, and if a tag is left blank, Wercker uses the docker default of latest.
  • registry: Indicates the end point of the registry. Leave this property empty if you’re pushing the image to Docker hub. However, if you’re pushing the image to other registries, then begin the registry name with https://, and make sure that it has the same prefix as that of the repository. Append /v2 to the URL to push to registries that support V2 of the registry API.

The following properties are available only when you create an image from the current pipeline container. These should not be specified if the image is created from a Dockerfile using an internal/docker-build step - as they modify the image.

  • ports: Indicates the comma-separated list of ports that are exposed. The port number can end with the /tcp or the /udp protocols. If you do not specify a protocol, then /tcp is used. This property is equivalent to the EXPOSE instruction in a Dockerfile.
  • volumes: Indicates the comma-separated list of volumes that are exposed. This property is equivalent to the VOLUME instruction in a Dockerfile.
  • working-dir: Used to override the working directory of the container. This property is equivalent to the WORKDIR instruction in a Dockerfile.
  • author: Used to set the author of the container. This property is equivalent to the MAINTAINER instruction in a Dockerfile.
  • cmd: Used to set the cmd for the new container. This is equivalent of the CMD instruction in a Dockerfile. Wercker only supports a single string value, which is in turn converted to exec form using go-shlex.
  • entrypoint: Used to set the entry point for the new container. This is equivalent of the ENTRYPOINT instruction in a Dockerfile. Wercker only supports a single string value, which is in turn converted to exec form using go-shlex.
  • disable-sync: Used to disable the synchronization of the environment variables before running the docker-push step. Set this property to true for the containers that do not support syncronization of the environment variables. This does mean that any of the exported environment variables will not be available for use in these properties.
  • message: Used to set a comment on a layer.
  • user: A string value used to specify a user inside a container.
  • env: A set of environment variables that you must set inside an image. Set the environment variables with one or more key-value pairs in the following format: VAR=value, with multiple values separated by spaces. Enclose the resulting string in double-quotes. If the value contains spaces, you must enclose it within single quotes. For example, “VAR1=value1 VAR2=‘Three word value’”
  • labels: A set of arbitrary key-value labels that you must set on an image. Set the labels with one or more key-value pairs in the following format: label=value, with multiple values separated by spaces. Enclose the resulting string in double-quotes. If the value contains spaces, you must enclose it within single quotes. For example, “Label1=value1 ‘Three word key’=value2 Label3=‘Three word value’”.
  • stopsignal: Indicates a signal to stop a container as a string or an unsigned integer. SIGTERM is the default value.

It is possible to use environment variables inside all properties, which will be expanded. Environment variables that are exported during a build are also available, unless disable-sync is set to true.

More information about the internal/docker-push step can be found here.

internal/docker-scratch-push

The docker-scratch-push step works the same as the normal docker-push step. The main difference is that this step uses a scratch base image provided by Docker. This image contains only the bare essentials, and as such is very lightweight in terms of size. To set it up correctly however, it requires some additional steps.

It injects the files present in the $WERCKER_ROOT environment variable into the root of the container.

To help you build your first scratch-enabled application, you can follow our tutorial

internal/store-container

The internal/store-container step will store the result of a build as a container artifact. You will then be able to download that container and execute it locally. This is useful if you want to pull your container to introspect your build.

internal/watch

The internal/watch step gives you a long-running step that can be configured to reload on file changes. A very common use case for this step is front-end development, here’s an example from our getting-started-nodejs project:

box: nodesource/trusty
dev:
  steps:
    - npm-install
    - internal/watch:
        code: node app.js
        reload: true

With this dev pipeline we run one initial setup step, npm-install, to prepare our container’s environment. We then execute our node app, reloading on changes.

To run this, you would do:

wercker dev --publish 5000

And once it loads, browse to your docker host on port 5000 to see your app running. For your convenience, we’ll tell you the IP once the step runs.

As you make changes to your code, the app will be reloaded, but the npm-install steps will not be run again.

Without the “reload: true” the code will only be run once, which is useful if your development server has its own reloading semantics (or is only loading static files).

Tuning open file limits on macOS

The internal/watch step uses kqueue on macOS. Kqueue must open each file that it watches, which means projects with many files may run into system limits. This usually manifests itself as a “too many open files” error. You can adjust these limits by running

sysctl -w kern.maxfiles=20480 (or whatever number you choose)
sysctl -w kern.maxfilesperproc=18000 (or whatever number you choose)

Thanks to github user @stvnwrgs for posting this fix.

internal/shell

The internal/shell step is pretty simple: it drops you into a shell as soon as the step is run.

You can use it much the same way as internal/watch. If you want to run a service, but be able to mess with different flags for testing, just CTRL-C the service and restart it however you want.

You can also use it as a way to inspect the state of the system between steps, drop into a shell, and look around.

Here’s a use case where you’re checking some log entries:

box: nodesource/trusty
dev:
  - npm-install
  - internal/shell:
      cmd: /bin/sh  #defaults to /bin/bash
      code: |
        # some code to automatically run in your shell session
        # before you start interacting
        cd /var/log