Plug

plug runs a developer's local process as a member of the cluster. Their service answers under its real name, and reaches the others by theirs: no port-forward, no code change. A deployed service of the same name is parked for the session and put back afterwards. What this is for, and how everyone looking at the application is told who is serving it, is on Dev mode. This page is how to open it and how to install it; everything else plug does - profiles, several clusters at once, volumes, environments - is in plug's documentation.

Enterprise edition The tunnel is built into the Enterprise image, and so is the plug client it hands out: plug 2.30.0.

Two conditions, two owners

The tunnel runs only when both of these are on:

switch where whose what it decides
Developer mode Application, General application admin what this installation offers its developers: the Developer menu, the API docs, the simulated sign-ins
Open the developer tunnel Infra, Plug infrastructure admin a port into the cluster, and the deployment's rights over it (Docker socket, Kubernetes role)

The tunnel's own switch ships off. These two switches decide, on a production gateway as anywhere else.

While the tunnel is closed, a developer is not offered a key to deposit: the page does not exist, and the user menu does not show it. A key for a door that is not there is a key nobody can use.

The Infra, Plug page says what the tunnel is doing, not what was ticked: listening on its port, closed and by which switch, or switched on but unable to run here and why - a gateway that does not run in a container, for instance, cannot provision a name in the cluster.

The address developers use

The gateway cannot see what sits between it and a laptop: a NodePort, a LoadBalancer, a published Docker port. So Infra, Plug records the published host and port, and every command on that page and on the developer's profile page carries them, ready to copy.

The Plug screen: the tunnel switch, the published host and port, and who holds the developer capability

The tunnel listens on 22222 in the container (MEERKAT_PLUG_ADDR).

On a developer's machine

On Infra, Plug, the On a developer's machine button opens these steps in a drawer, for macOS, Linux or Windows, with the commands already carrying this gateway's address. It is help, not configuration: nothing in it is saved.

The drawer with the steps for a developer's machine: install, name the profile, the key pair, deposit it, plug a service in

A developer needs the developer capability on their account (Application, Users). Then, once:

1. Install plug. It installs from the gateway itself, not from a package manager, and prepares the machine once, so that later runs need no privilege.

macOS and Linux, from a terminal (it may ask for your password once):

ssh -p 22222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null get@<gateway-host> install | sh

Windows, from Git Bash (it comes with Git for Windows; it asks once to run as Administrator):

ssh -n -p 22222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null get@<gateway-host> install-windows | bash -s -- <gateway-host> 22222

The installer creates a profile named after the host, which is what -p names below. A name of your own reads better once a second cluster is installed, and the key pair moves with it:

plug rn <gateway-host> my-cluster

<profile> below is that name: my-cluster once renamed, <gateway-host> otherwise.

The commands on the Infra, Plug page follow the address and the profile name typed there.

2. Generate the key pair. plug keeps it under ~/.plug/keys, one pair per cluster, so a key can be withdrawn from one cluster without touching the others.

plug keygen -p <profile>
plug pubkey -p <profile> | pbcopy                        # macOS
plug pubkey -p <profile> | xclip -selection clipboard    # Linux
plug pubkey -p <profile> | clip                          # Windows, Git Bash

3. Deposit the public key on your profile, signed in to the applications: /profile/dev/key, or from any application, the user menu, Developer, plug key. The page shows the SHA256 fingerprint, to compare with what plug pubkey printed. It takes a public key, never a certificate.

One key per workstation: plug keeps a pair per profile, so a second machine runs plug keygen there and deposits a second key beside the first. Each is listed with its fingerprint and the comment it carries, and removing one closes that machine alone. A key belongs to one account: the same key deposited twice, or by somebody else, is refused.

4. Plug a service in, by prefixing the command that runs it:

plug -p <profile> -s my-service:8080:3000 npm run start

The cluster reaches it as my-service:8080, forwarded to your local 3000, for as long as the command runs.

Beside Argo CD, Flux and other GitOps controllers

A cluster driven by a continuous-deployment controller - Argo CD, Flux, Rancher Fleet, Crossplane - keeps every object equal to what a repository declares, and undoes whatever differs. -s does the opposite on purpose: for the session, it changes the Service's spec.selector and spec.ports so the name points at the developer's machine. The controller sees drift and puts it back, usually within minutes. Consuming the cluster's services is not affected: only -s is.

Nothing fails on either side, which is what makes it easy to miss. When the deployed workload is running, the name goes back to it: the application answers, the page loads, and it is simply not the developer's code answering. A session that works for a few minutes and then stops reflecting changes is the first sign. plug checks the path in the background and warns: the name is answered by something else, or is not reachable inside the cluster when nothing is deployed.

Who manages a Service is in its metadata (kubectl describe svc <name>):

Controller What to look for
Argo CD annotation argocd.argoproj.io/tracking-id
Flux kustomize.toolkit.fluxcd.io/name, helm.toolkit.fluxcd.io/name
Rancher Fleet fleet.cattle.io/bundle-name
Crossplane, operators crossplane.io/composite, or an ownerReferences entry with controller: true

app.kubernetes.io/managed-by: Helm alone means nothing here: Helm acts on an upgrade and does not reconcile, so a Service deployed by a chart can be taken.

Argo CD: tell the Application not to compare the two fields plug changes; everything else about the Service keeps being reconciled. This goes in the Application, not in the chart:

spec:
  ignoreDifferences:
    - group: ""
      kind: Service
      name: my-service
      namespace: my-namespace
      jsonPointers:
        - /spec/selector
        - /spec/ports

On a cluster where plug is used routinely, set it once for every Service in the argocd-cm ConfigMap:

data:
  resource.customizations.ignoreDifferences._Service: |
    jsonPointers:
      - /spec/selector
      - /spec/ports

Turning Self Heal off also works for a one-off, but it suspends drift correction for the whole Application, and it is easy to forget to turn back on.

Flux takes the instruction on the resource itself, so the chart is the place: the annotation kustomize.toolkit.fluxcd.io/reconcile: disabled on the Service. It stops reconciling the whole object, not two fields.

Rancher Fleet compares with diff.comparePatches in the bundle's fleet.yaml, removing /spec/selector and /spec/ports. Crossplane: an Object switched to managementPolicies: ["Observe"] is no longer re-applied. An in-house operator rarely offers an equivalent: suspend it for the session, or plug in a name it does not own.

End a session with Ctrl-C Ctrl-C is what puts the Service back, from a receipt plug stored on it. A session killed with kill -9 leaves it parked until the agent's sweep, once a minute, finds the session gone and restores it: the Helm chart grants the deployments right that sweep needs. Under a controller, a Service can still keep an extra app: plug key in its selector, which the controller does not remove since it only manages the fields it knows: the Service then selects no pod, and the name answers nobody. If it still does after a couple of minutes, remove that key by hand. The chart also grants endpoints, so the agent writes the Endpoints itself when a selector cannot be repointed, rather than falling back to that key.

The details, and the other symptoms, are in plug's documentation: Continuous deployment & GitOps.

Who is plugged in

Application, Plug lists who is plugged into the cluster now: one row per plug run (plug -s, or plug -c for a client that only consumes), with the developer, the address it came from, the plug version, the time it was plugged in, and the names it serves. Each name served carries a small grid, the cluster, with the run's square coloured. In the grid: the run has parked the deployed service, the cluster's own is set aside and the workstation answers in its place. Turned and set beside it: a name the cluster did not have, which the run adds. The audit trail says the same of each name served. From plug 2.30.0 a run also says what it is: a plug -s that has not served its name yet reads nothing served yet rather than client only, and a --dockerrun carries the Docker logo beside its plug version. An older plug says neither, and a run without a name reads as a client. A run whose connections landed on several replicas of the gateway is still one row.

End this session stops the developer's command as a Ctrl-C would, with the reason shown in their terminal, and what the run had set aside in the cluster is back at once, on every replica. It is not a ban: a new run is let in as long as the key is.

Taking an access back

Removing the key on the profile, the developer capability on the account, or disabling or deleting the account ends at once what that key has open, whichever screen did it, and the developer reads why. Turning either switch off closes the tunnel. Nothing was issued that would outlive it. Application, Plug lists who holds the capability, with the fingerprint of their key or the fact that they have none yet.

The people are the application's, the tunnel the infrastructure's. Application, Plug - who holds the capability, who is plugged in now, ending a run - is the application administrator's, the one who grants the capability on Application, Users; its entry exists in the Enterprise image while the tunnel is open. Infra, Plug keeps the switch, the address and the commands.

In the audit trail

The audit trail, under plug, says once each run that opened and how it ended (the developer said goodbye, dropped, ended by somebody and why, the gateway stopped), how long it lasted, and each name served and given back: who was serving checkout on Tuesday at three is a search. Depositing and removing a key are already there, with its fingerprint.