GuidesAPI Reference
Log In
Guides

Configuring a Custom Runtime

Suggest Edits

To define a Custom Runtime, create a Prodvana config file. You will need a Docker image that does the actual apply in the Runtime and a Kubernetes Runtime already linked where your job will be running. See Configuring a Runtime for how to link a Kubernetes runtime.

runtime:
  name: my-custom-runtime
  custom:
    proxyRuntime:
      runtime: my-kubernetes-runtime  # pick a kubernetes runtime already linked with Prodvana
    apply:
      taskConfig:
        program:
          cmd:
          - apply-changes
          - --my-change-id={{.Params.changeId}}
          imageTag: my-tag
          imageRegistryInfo:
            containerRegistry: my-registry-name  # pick any registry already linked with Prodvana
            imageRepository: my-repository
    parameters:
    - name: changeId  # parameters can be whatever makes sense for your runtime
      description: the ID to deploy
      string:
        defaultValue: default-id

In the example above, the Custom Runtime is defined with the apply command apply-changes --my-change={{.Params.changeId}}, running in the my-kubernetes-runtime using the image my-repository:my-tag in the registry my-registry-name. Further more, changeId is defined as a parameter. This is useful as it will allow Services deployed to the Runtime to customize its behavior, e.g. picking a different VM image or specifying the replica count.

Apply the config with pvnctl configs apply.

pvnctl configs apply my-custom-runtime.pvn.yaml

To use the runtime, reference it in your Application config in each of your Release Channel. It is also possible to have different Custom Runtimes be used for different Release Channels, e.g. to have production use a different set of commands than staging.

application:
  name: my-app # replace with your desired app name
  releaseChannels:
  - name: staging
    runtimes:
    - runtime: my-custom-runtime
  - name: production
    runtimes:
    - runtime: my-other-custom-runtime  # can be the same runtime as the previous step
    preconditions:
    - releaseChannelStable:
        releaseChannel: staging
    - manualApproval: {}

Next, define a Service that uses the Custom Runtime.

service:
  name: my-service
  application: my-application
  customRuntime:
    parameterValues:
    - name: changeId
      string: my-change-id-for-this-service

It is also possible to have the service itself define parameters which are then passed through to the Runtime, allowing this parameter to be set from the deploy UI.

service:
  name: my-service
  application: my-application
  customRuntime:
    parameterValues:
    - name: changeId
      string: "{{.Params.changeId}}"
  parameters:
  - name: changeId
    string:
      defaultValue: my-default-change-id

Common Examples

Using an Existing Namespace

By default, Custom Runtime commands will run in an automatically generated namespace, pvn-proxy-runtime-{runtime-name}. However, it can be useful to run commands in an existing namespace created outside of Prodvana, e.g. to reference an existing Kubernetes secret.

To run commands in an existing namespace, modify the runtime config:

runtime:
  name: my-custom-runtime
  custom:
    proxyRuntime:
      runtime: my-kubernetes-runtime  # pick a kubernetes runtime already linked with Prodvana
      containerOrchestration:
        k8s:
          namespace: my-existing-namespace
    ... # the rest of your config here

You are responsible for making sure the namespace exists.

Using Multiple Custom Runtimes

It is possible to have multiple types of Custom Runtimes configured in a release channel. For example, you may have one kind of Runtime for deploying VMs and another for pushing static assets. To do so, define a name for the Runtime:

application:
  name: my-app # replace with your desired app name
  releaseChannels:
  - name: staging
    runtimes:
    - runtime: my-vm-runtime
      name: vm
    - runtime: my-static-assets-runtime
      name: static
  - name: production
    runtimes:
    - runtime: my-prod-vm-runtime  # can be the same or different from staging
      name: vm  # this name must match the one defined in staging
    - runtime: my-prod-static-assets-runtime  # can be the same or different from staging
      name: static
    preconditions:
    - releaseChannelStable:
        releaseChannel: staging
    - manualApproval: {}

Then, in your service config, reference the name to decide which runtime to use.

service:
  name: my-service
  application: my-application
  customRuntime: ...  # rest of your config here
  runtimeConnection: vm

Using Kubernetes Configurations

To define commands via Kubernetes configurations, use kubernetesConfig instead of taskConfig in your runtime config file.

runtime:
  name: my-custom-runtime
  custom:
    proxyRuntime:
      runtime: my-kubernetes-runtime  # pick a kubernetes runtime already linked with Prodvana
    apply:
      kubernetesConfig:
        type: KUBERNETES
        local:
          path: path/to/kubernetes/job.yaml
    parameters:
    - name: changeId  # parameters can be whatever makes sense for your runtime
      description: the ID to deploy
      string:
        defaultValue: default-id

🚧

Define Exactly One job and Nothing Else

Defining any other Kuberntes resources here (e.g. configmap) will result in undefined behaviors due to conflicts when multiple services use this Custom Runtime.

Parameters can still be used, directly inside the Kubernetes configs themselves. The order of operation with parametrization is the same as with Service configurations, see Configuring Services.

Defining a Fetch Command

Defining a fetch function allows Prodvana to continuously converge Services in your Runtime as well as skip the apply command if the Service is already at the correct state. You should define a fetch function if you have the ability to query your Runtime to figure out the state of your Service.

A fetch command is just a run-to-completion job with special meaning behind its exit codes:

  • exit code 0 - The Service is already at the desired state. Prodvana will not run the apply command.
  • exit code 2 - The Service is not at desired state. Prodvana will run the apply command, pending any preconditions.
  • any other exit codes - Unhandled error, Prodvana will not take any actions.

If defined, Prodvana will run the fetch command periodically as part of its convergence loop. If the fetch command exits 0, Prodvana does nothing and marked the Service as converged. This is the equivalent of pushing a Kubernetes Service where the Service is already at the desired version. If the fetch command exits 2, Prodvana will run the apply command after preconditions allow it to, potentially retrying until the fetch command returns 0. If the fetch command exits with any other exit code, Prodvana will take no action.

runtime:
  name: my-custom-runtime
  custom:
    fetch:
      kubernetesConfig:
        type: KUBERNETES
        local:
          path: path/to/kubernetes/job.yaml
    ... # rest of config here

runtime:
  name: my-custom-runtime
  extension:
    fetch:
      taskConfig:
        program:
          cmd:
          - fetch-state
          - --my-change-id={{.Params.changeId}}
          imageTag: my-tag
          imageRegistryInfo:
            containerRegistry: my-registry-name  # pick any registry already linked with Prodvana
            imageRepository: my-repository
    # rest of config here

Customizing Fetch Interval

By default, fetch commands run every 30s when not converged and every 5 minutes when converged. This can be customized in the fetch command definition:

runtime:
  name: my-custom-runtime
  custom:
    fetch:
      ... # rest of fetch definition here
      pollInterval: 300s  # poll interval when not converged
      steadyStatePollInterval: 1500s  # poll interval when converged, to check for drifts

Defining a getInfo Command

Defining a getInfo command is a way to help Prodvana display contextual information for owners of Services running on Custom Runtimes, without needing to understand how Custom Runtimes are implemented. You should define a getInfo command if displaying additional data specific to your Custom Runtime will improve the developer experience.

A getInfo command is a run-to-completion job that outputs structured data. If defined, Prodvana will run the getInfo command periodically as part of its convergence loop. The output of the command, whether it succeeds or fails, has no impact on the convergence behavior of the Service.

To define a getInfo command, first create a Docker image with pvn-wrapper installed. pvn-wrapper is required in order for the process to return structured output to Prodvana, as Kubernetes does not separate stdout and stderr.

Then, create a program that writes json of the following schema to stdout:

{
  "outputs": [
    {
      "name": "Some User Readable Name, used when displaying",
      "text": "free-form text used to display"
    }
  ]
}

Then, create a Kubernetes Job configuration and reference it in your Custom Runtime config file.

apiVersion: batch/v1
kind: Job
metadata:
  generateName: get-info-
spec:
  ttlSecondsAfterFinished: 600
  template:
    spec:
      containers:
        - name: get-info
          image: your-docker-image
          command: ["pvn-wrapper", "exec", "--"]  # This is the important part. The rest of the file can be adapted as you'd like.
          args: ["your-binary-here", ...]
      restartPolicy: Never
  backoffLimit: 0


runtime:
  name: my-custom-runtime
  extension:
  	getInfo:
      kubernetesConfig:
        type: KUBERNETES
        local:
          path: path/to/kubernetes/job.yaml
  ... # rest of config here

For very simple commands, it is also possible to use Prodvana's backend-agnostic config:

runtime:
  name: my-custom-runtime
  custom:
  	getInfo:
      taskConfig:
        program:
          entrypoint:
          - pvn-wrapper
          - exec
          - --
          cmd:
          - your-binary-here
          imageTag: my-tag
          imageRegistryInfo:
            containerRegistry: my-registry-name  # pick any registry already linked with Prodvana
            imageRepository: my-repository
    # rest of config here

The output of the getInfo command will be displayed in the convergence sidebar on the Prodvana UI.

Like fetch and apply, getInfo has access to any parameters you defined for your Custom Runtime.

Requiring approval before each change

For critical Custom Runtimes or Terraform modules, it is useful to require manual approval before any changes are applied. Every time drift is detected, Prodvana will wait for manual approval before applying any changes.

application:
  name: my-app # replace with your desired app name
  releaseChannels:
  - name: staging
    runtimes:
    - runtime: my-custom-runtime
  - name: production
    runtimes:
    - runtime: my-other-custom-runtime  # can be the same runtime as the previous step
    preconditions:
    - releaseChannelStable:
        releaseChannel: staging
    - manualApproval: 
        # Require approval before any changes are made.
        # Every time drift is detected, require approval before applying changes.
        everyAction: true

🚧

everyAction: true currently only works for Custom Runtimes and Terraform runners. It does nothing for Kubernetes services.

Customizing Convergence Grace Period

The convergence grace period is the time between when the apply command started running to when the fetch command returns a clean exit with no drift. If this time exceeds the configured limit, Prodvana will run the apply command again.

The default convergence period is 10m. To set it to something else, modify your Custom Runtime config.

runtime:
  name: my-custom-runtime
  custom:
    fetch: ...
    apply: ...
    convergenceGracePeriod: 86400s  # 1 day

Limiting Concurrency of Apply

You can limit the number of concurrent apply jobs that run for a Runtime by:

runtime:
  name: my-custom-runtime
  custom:
    apply:
      maxConcurrency: 10  # no more than 10 apply commands will run, everything else will wait until there is less than 10
     ...
   ...

Controlling When Apply Needs to Run Without Fetch

By default, without a fetch function, apply will always run when there is a new version to roll out. This can be needlessly expensive and can be controlled with applyId. For example, if your Custom Runtime takes a parameter that is the commit sha, and you only want to run apply when the commit sha changes, you can do so like this:

runtime:
  name: my-custom-runtime
  custom:
    applyId: "{{.Params.commit}}"
    apply:
      ...
    parameters:
    - name: commit
      required: true
      string: {}
   ...

Updated about 1 year ago