← Back to dashboard

Skytap

Audience: operator · Profile: pov · Read this when: you are pointing a POV instance at Skytap and want to know what it will and will not do with the account.

The first lab platform for POV environments. A POV is a Skytap template instantiated whole; the dashboard creates those environments and wires their VMs into that POV's PRA, Password Safe and Entitle tenants.

Available on a POV instance only — see POV profile. On an estate instance the integration is masked off and Settings refuses to enable it.

What the dashboard does with an environment. Create it from a template, power it on and off, and destroy it. Enrol an agent inside it — see The control node VM and the template contract. Install a BeyondTrust Gateway on that control node, registered into the POV's own PRA tenant (the POV Gateway), and a Password Safe Resource Broker on a Windows VM beside it (the Resource Broker). Then, per VM: one PRA jump item through that Gateway, Password Safe onboarding through that Resource Broker, and Entitle registration for the Linux guests (wiring the VMs). Publish a customer-facing share link for the whole environment (the share link) — Skytap calls it a publish set, and the dashboard always gives it a generated password and an expiry. And reap the environment on an auto-delete timer (the auto-delete timer), which warns on a ladder rather than once because an evaluation outlives whoever set it up.

Prerequisites

Getting the token

In Skytap, open your Account page. If an API security token is listed there, that is what you use; Skytap authenticates with HTTP Basic as username:api_token.

This is the single most common setup mistake, so it is worth stating plainly: the password field is not your password. A wrong value produces a 401, and the dashboard's error message says so rather than reporting a generic failure.

Configuring it

Settings → Integrations → Skytap.

The token is encrypted at rest with the same Fernet key as every other secret, and — like any secret in this dashboard — it can instead be a reference into an external vault.

Verifying the connection

The Skytap panel has a Test connection button. It reads the saved values, not what is currently in the form, so save first.

It issues one page of GET /v2/templates — the same read the POV page makes — scoped to the Project ID when one is set. That single call answers three questions at once: whether the token authenticates, what this account can actually see, and whether the Project ID is real.

It distinguishes five outcomes:

An account that authenticates but exposes no templates is deliberately red. A POV cannot be created from an empty catalogue, so reporting green for a connection that cannot do the one thing it is for would be the exact false positive a check like this exists to prevent.

What the dashboard does with the API

Three of Skytap's behaviours are easy to get wrong once and then wrong everywhere, so all three are handled in one place (services/skytap_client.py) rather than at each call site.

423 Locked is normal. It is not an error — it is Skytap saying "this resource is busy, or the account is being rate-limited", and it carries a Retry-After. A plain 429 is retried identically. Environments also expose a rate_limited boolean, which the POV page surfaces as a badge. The client retries, honouring Retry-After, bounded; only after that does it report a failure, and the message says it is a rate limit rather than a fault.

The retry budget is per client. Provisioning gets the generous default, because a job that has already waited minutes should keep waiting; Test connection asks for one retry, so a rate-limited account answers the button instead of hanging behind a spinner.

Every read carries keep_idle=true. Without it, reading an environment resets its idle timer. A dashboard that polls environments would hold every one of them awake and quietly defeat suspend_on_idle — the single biggest lever on Skytap spend. The only symptom would be the invoice, which is exactly why it is not left to the caller.

The dashboard's view of an environment is refreshed on a timer. A POV's runstate used to be written only when this dashboard changed it — at provision, or at an explicit Start or Suspend. Nothing asked Skytap again. So suspend_on_idle, the biggest lever on spend, would suspend an environment and the POV page would go on saying running indefinitely; worse, it gates the Start button on that value, so the environment you most needed to wake was the one whose Start button was hidden. A reconcile sweep (services/pov_reconcile) now reads every managed environment back every ten minutes — one paginated collection read — and the POV page shows when each row was last confirmed rather than implying it is live. Each pass is a pov_env_reconcile job; completed ones are hidden on Job History unless you tick Show routine sweeps, and are pruned on the same retention as the other sweeps.

Collections paginate by count/offset. A single GET returns a first page that looks exactly like a complete answer, so listings are walked to the end.

Listings are project-scoped when a Project ID is set — they read /v2/projects/{id}/templates and /v2/projects/{id}/configurations instead of the account-wide collections. A project id the token cannot see answers 404, and the dashboard turns that into a message naming the remedy rather than a bare error. That is why the sub-resource paths were chosen over a filter parameter: a filter an API does not implement is silently ignored, and an unscoped list looks exactly like a correct one.

The lifecycle

Three orderings are load-bearing, and each is wrong in a way that leaves a resource nobody can reclaim:

The platform id is persisted before anything else can fail. An environment that exists in Skytap and not in this database is the one failure mode nothing can clean up automatically. So the create call is followed immediately by a commit, and every later step is written to be re-runnable around it.

A failure after creation keeps the id. If the power-on fails, the row stays failed with its Skytap id and a message telling you to press Destroy. Failing without the id is how an orphan is made.

A failed destroy does not mark the row destroyed. Marking it would hide an environment that is still running and still billing. The row stays visible, keeps its id, and Destroy can be re-run once the cause is cleared.

Three smaller rules worth knowing:

• Destroy is allowed from failed, not only from active. A POV that broke halfway through provisioning is exactly the one that most needs reaping.
• An empty VM read never prunes. A transient error returning zero VMs would otherwise delete every VM row — later taking the PAM artifact columns with it — and record success.
• The control node install cannot fail the provision. An environment that is up with no agent in it is running, billing and reapable; failing it would trade a fixable gap for a destroyed environment. The reason lands on the row next to an Install control node button that re-runs it, and never in the row's error_message, which means "this environment is broken".

Pages

The larger parts have a page each:

The template contract

Skytap's bootstrap_injection mechanism is metadata: the platform stores the payload and the guest fetches it. There is no cloud-init datasource and nothing runs it for you, so the control node VM in your template must carry a small runner.

The builder writes this for you. Building a template generates the runner from the same marker constants the dashboard writes into the payload, installs it, and installs Docker beside it when the guest has no runtime. The shape below is what it generates, and what you need if you are baking a template by hand — POV → Templates will also just hand you the script to paste. That script is plain ASCII on purpose: the console you may have to type it into has no clipboard.

The runner has to do four things. Anything that does them is fine; this is the shape:

#!/bin/sh
# /usr/local/sbin/dashboard-bootstrap-runner - run at boot and keep running.
set -eu
MARKDIR=/var/lib/dashboard-bootstrap
mkdir -p "$MARKDIR"

while :; do
  body=$(curl -fsS http://169.254.169.254/skytap/vms/self/user_data 2>/dev/null || true)
  case "$body" in
    *BEGIN-DASHBOARD-AGENT-BOOTSTRAP*END-DASHBOARD-AGENT-BOOTSTRAP*)
      sum=$(printf '%s' "$body" | sha256sum | cut -d' ' -f1)
      if [ "$sum" != "$(cat "$MARKDIR/last" 2>/dev/null || true)" ]; then
        printf '%s' "$body" > /run/dashboard-bootstrap.sh
        sh /run/dashboard-bootstrap.sh && printf '%s' "$sum" > "$MARKDIR/last"
      fi
      ;;
  esac
  sleep 20
done

1. Poll, do not read once. The payload arrives after the VM is up, because of the ordering above. A runner that reads user_data once at boot finds it empty and stops.
2. Require both markers. BEGIN-DASHBOARD-AGENT-BOOTSTRAP and END-DASHBOARD-AGENT-BOOTSTRAP must both be present before anything runs. A truncated metadata read would otherwise execute half the script — and the half at the top is the half that deletes the running agent and its state volume.
3. Key the "already ran" marker on the payload's hash, not on a flag. A reboot with unchanged user_data must not re-run; a re-injection — a new enrolment code — must. A boolean gets exactly one of those right.
4. Run it as root. It writes to /etc/dashboard-agent and calls docker.

Install it as a service that starts at boot — a systemd unit with Restart=always is the usual form — and bake it into the template. It is idempotent and it costs one HTTP call to a link-local address every twenty seconds.

If your account's metadata service does not serve http://169.254.169.254/skytap/vms/self/user_data, read http://169.254.169.254/skytap instead and pull user_data out of the JSON document. The payload is what matters; the path is Skytap's.

What is deliberately not used

The Terraform provider. skytap/skytap last released v0.15.1 in November 2022, and its own documentation says it "doesn't enumerate the resources contained within that template, including VMs and networks" — which is the one thing a POV needs, because a POV is a template instantiated whole. The dashboard talks to the REST API directly, as the Portainer and Rancher integrations do. That also keeps the feature clear of the provider pre-cache coupling in the image build.

Published services for PAM wiring. Skytap can NAT a guest port to a public ip:port, and the dashboard displays those, but the wire-up does not use them: a published address changes per environment and per power cycle. POV wiring reaches VMs on their private IPs through a Gateway inside the environment.

Capabilities

The registry (services/lab_platforms.py) records what each platform can do, so a feature a platform lacks degrades visibly instead of failing late:

The two project paths are not yet confirmed against a live account. Three things are assumed: that they return the same object shape as the account-wide collections, that they honour count/offset pagination, and that a project id the token cannot see answers 404 rather than an empty list. They are flagged the same way in the code, above the one place the paths are built. Nothing changes for an install that leaves the Project ID blank, and Test connection is the quickest way to check one that does not.

bootstrap_injection is one intent with more than one mechanism. Skytap hands data to the guest and the guest fetches it; another platform might run a script on the guest instead. The distinction matters because Skytap's metadata service works only on VMs attached to automatic networks, and nothing executes user_data for you — there is no cloud-init datasource, so the guest must fetch it. That is what the template contract above specifies, and it is why the control node's agent install refuses outright on a platform whose mechanism is anything other than metadata rather than failing somewhere inside a job.

Troubleshooting