diff --git a/.cspell/vantage-dictionary.txt b/.cspell/vantage-dictionary.txt index a9154ce..469562b 100644 --- a/.cspell/vantage-dictionary.txt +++ b/.cspell/vantage-dictionary.txt @@ -33,3 +33,6 @@ juju slurm-juju-localhost slurm-multipass-localhost slurm-microk8s-localhost +noVNC +NCCL +vdeployer diff --git a/docs/how-to-guides/workbench/compute/create-pool.mdx b/docs/how-to-guides/workbench/compute/create-pool.mdx index 24dd4db..50219ef 100644 --- a/docs/how-to-guides/workbench/compute/create-pool.mdx +++ b/docs/how-to-guides/workbench/compute/create-pool.mdx @@ -1,16 +1,16 @@ --- title: Create a Workbench compute pool -description: Add a compute pool to a Kubernetes cluster for Workbench sessions and training jobs. +description: Add a compute pool to a Kubernetes cluster for Workbench sessions, services, training jobs, and endpoints. sidebar_label: Create a compute pool --- # Create a Workbench compute pool -Compute pools provide the infrastructure that Workbench sessions, training jobs, and endpoints run on. This guide walks through creating a new pool from the Workbench interface. +Compute pools provide the Kubernetes node capacity that Workbench sessions, services, training jobs, and endpoints run on. This guide walks through creating a new pool from the Workbench interface.

- Add a compute pool to a Kubernetes cluster for Workbench sessions and training jobs. + Add a compute pool to a Kubernetes cluster for Workbench workloads.

@@ -24,7 +24,7 @@ Compute pools provide the infrastructure that Workbench sessions, training jobs,
Outcome - A compute pool ready for sessions and jobs + A compute pool ready for Workbench workloads
@@ -32,23 +32,16 @@ Compute pools provide the infrastructure that Workbench sessions, training jobs,
-## Open the Pools page +## Open Pools -Click **Workbench** in the left sidebar, then click **Pools**. +Click **Workbench** in the left sidebar, then click **Compute > Pools**.
## Start pool creation -Click `New Pool`. - -
-
- -## Select the cluster - -Select the **Kubernetes cluster** the pool will be created on. +Click **New Compute Pool**.
@@ -57,26 +50,23 @@ Select the **Kubernetes cluster** the pool will be created on. Set the following fields: - - **Name** -- a descriptive identifier (for example, `gpu-training` or `cpu-general`) - - **Role** -- `control` (cluster management) or `worker` (compute workloads) - - **Instance types** (AWS) -- click to open the instance browser and select one or more EC2 types - - **Profile** (non-AWS) -- choose Small (4 vCPU, 8 GiB), Medium (8 vCPU, 16 GiB), or Large (16 vCPU, 32 GiB) - - **Min size** -- minimum number of nodes - - **Max size** -- maximum number of nodes for autoscaling - -
-
- -## Add labels and taints (optional) - -Optionally add **Labels** (key-value pairs for scheduling) and **Taints** (to restrict which workloads run on this pool). +| Field | Required | What it does | +|---|---|---| +| **Name** | required | Unique identifier for the pool, for example `workers-cpu-small`. | +| **Min Size** | required | Minimum number of nodes. Set to `0` to allow scale-to-zero. | +| **Max Size** | required | Maximum number of nodes for autoscaling. | +| **Instance Type** | required | Cloud machine type selected from the instance browser. | +| **Workload** | required | Workload tag such as workspace, training, inference, cloud-shell, pvc-viewer, or remote-desktop. | +| **Root disk size (GB)** | required | Boot disk size in GiB. Default is `30`. | +| **Secondary Networks** | optional | Additional networks available to the pool. | +| **GPU compute pool** | optional | Enables GPU fields for GPU type, count, and vendor. |
## Create -Click `Create`. +Click **Create**.
@@ -85,7 +75,7 @@ Success looks like this: the compute pool is created and enters a ready state, a
-Set `min=0` for scale-to-zero behaviour. Nodes are provisioned on demand and terminated when idle. This reduces costs for bursty workloads but adds 2--5 minutes of startup latency. +Set **Min Size** to `0` for scale-to-zero behavior. Nodes are provisioned on demand and terminated when idle. This reduces costs for bursty workloads but adds startup latency.
@@ -97,7 +87,7 @@ The new pool appears in the Pools list with a node count and status. Sessions an ## What to do next -- [Manage compute profiles](/how-to-guides/workbench/compute/managing-profiles) -- configure named sizing options -- [Manage compute pools](/how-to-guides/clusters/kubernetes/manage-compute-pools) -- edit and delete pools from the cluster detail page +- [Manage compute pools in Workbench](/how-to-guides/workbench/compute/managing-profiles) -- edit, delete, and inspect pools +- [Manage compute pools from the cluster detail page](/how-to-guides/clusters/kubernetes/manage-compute-pools) -- edit and delete pools alongside other cluster settings diff --git a/docs/how-to-guides/workbench/compute/managing-profiles.mdx b/docs/how-to-guides/workbench/compute/managing-profiles.mdx index 6cbb975..f7de34f 100644 --- a/docs/how-to-guides/workbench/compute/managing-profiles.mdx +++ b/docs/how-to-guides/workbench/compute/managing-profiles.mdx @@ -35,7 +35,7 @@ This guide covers the day-to-day tasks for managing compute pools in Workbench: ### Open the Compute Pools page -Navigate to **Workbench > Compute Pools**. +Navigate to **Workbench > Compute > Pools**.
@@ -68,7 +68,7 @@ Click **Create**.
-Success looks like this: the profile is saved and appears in the profile list, ready to be used for new sessions. +Success looks like this: the pool is saved and appears in the pool list, ready to be used for new Workbench workloads. The new pool appears in the list and is immediately available for selection when creating sessions, training jobs, or endpoints. @@ -79,7 +79,7 @@ The new pool appears in the list and is immediately available for selection when ### Open the pool detail page -Navigate to **Workbench > Compute Pools** and click the pool row to open its detail page. +Navigate to **Workbench > Compute > Pools** and click the pool row to open its detail page.
@@ -111,7 +111,7 @@ Editing a pool does not affect resources that are already running on it. Changes ### Navigate to Compute Pools -Navigate to **Workbench > Compute Pools**. +Navigate to **Workbench > Compute > Pools**.
@@ -155,7 +155,7 @@ If you do not select a pool, the workspace's default pool is used. ### Open the pool detail page -Navigate to **Workbench > Compute Pools** and click a pool row to open its detail page. +Navigate to **Workbench > Compute > Pools** and click a pool row to open its detail page.
diff --git a/docs/how-to-guides/workbench/develop/cloud-shell.mdx b/docs/how-to-guides/workbench/develop/cloud-shell.mdx index 3985878..3b8b01a 100644 --- a/docs/how-to-guides/workbench/develop/cloud-shell.mdx +++ b/docs/how-to-guides/workbench/develop/cloud-shell.mdx @@ -1,11 +1,11 @@ --- title: Use Cloud Shell -description: Launch a browser-based terminal on cluster compute +description: Launch a browser-based terminal on a Workbench compute profile. --- # Use Cloud Shell -Cloud Shell gives you a browser-based terminal on cluster compute without SSH keys or a local client. It is the fastest way to get a shell on a GPU node. +Cloud Shell gives you a browser-based terminal without SSH keys or a local client. Use it for quick file checks, package inspection, Slurm commands, and lightweight cluster operations from your browser.

@@ -19,7 +19,7 @@ Cloud Shell gives you a browser-based terminal on cluster compute without SSH ke

You will need - A Kubernetes cluster with a compute pool tagged for Cloud Shell + A Kubernetes cluster with a compute profile available for Cloud Shell
Outcome @@ -35,14 +35,14 @@ Cloud Shell gives you a browser-based terminal on cluster compute without SSH ke ### Open the Cloud Shell page -Click **Workbench** in the left sidebar, then click **Cloud Shell** under the Develop section. +Click **Workbench** in the left sidebar, then click **Develop > Cloud Shell**.
### Start creation -Click **+ New Cloud Shell** in the top-right corner. +Click **New cloud shell**.
@@ -51,9 +51,9 @@ Click **+ New Cloud Shell** in the top-right corner. | Field | Required | Notes | |---|---|---| -| Name | No | Auto-generated if omitted | -| Compute pool | Yes | Only pools tagged for Cloud Shell appear | -| Slurm cluster | No | Enables Slurm job submission from the terminal | +| Name | No | Auto-generated if omitted. If provided, it must be a valid Kubernetes name. | +| Compute Profile | Yes | The compute profile to run the shell on. Only compatible profiles appear. | +| Slurm Cluster | No | Enables Slurm job submission from the terminal. Select **None** if you do not need Slurm access. |
@@ -64,9 +64,9 @@ Expand **Advanced Options** for additional settings: | Field | Notes | |---|---| -| Image version | Override the default ttyd image | -| CPU request / limit | Custom CPU allocation | -| Memory request / limit | Custom memory allocation | +| Image Version | Override the default ttyd image. Leave blank to use the cluster default. | +| CPU | Custom CPU request and limit, for example `500m`. | +| Memory | Custom memory request and limit, for example `512Mi`. |
@@ -99,15 +99,15 @@ Click **Open** to launch the embedded terminal in your browser.
-The terminal provides a full Linux environment with: +The terminal provides a browser-accessible Linux environment with: - Workspace storage mounted automatically -- Persistent home directory across sessions +- A persistent home directory for the service - Slurm CLI tools available if a Slurm cluster is attached ## Delete a Cloud Shell -Cloud Shell does not support pause/resume. When you are finished, click the **...** menu and select **Delete** to free compute resources. +Cloud Shell does not support pause or resume. When you are finished, click the row action menu and select **Delete** to free compute resources.
diff --git a/docs/how-to-guides/workbench/develop/manage-services.mdx b/docs/how-to-guides/workbench/develop/manage-services.mdx index 5d62d94..f447298 100644 --- a/docs/how-to-guides/workbench/develop/manage-services.mdx +++ b/docs/how-to-guides/workbench/develop/manage-services.mdx @@ -1,127 +1,48 @@ --- -title: Manage Workbench services -description: Create, monitor, and delete shared services in a Workbench workspace. -sidebar_label: Manage services +title: Manage user services +description: "Create, open, and delete Workbench user services: Cloud Shell, Remote Desktop, and PVC Viewer." +sidebar_label: Manage user services --- -# Manage Workbench services +# Manage user services -Services are shared deployments within a Workbench workspace, such as proxies, databases, or custom tooling that sessions and jobs can access over the cluster network. This guide covers creating, monitoring, and deleting services. +Workbench user services are lightweight browser-accessible tools that run on cluster compute. Unlike sessions, they do not support pause and resume, but they are faster to provision and ideal for quick tasks. -
- -Services are a platform feature that may vary by workspace configuration. Contact your workspace administrator if the Services page is not visible. - -
- -
-

- Create, monitor, and delete shared services in a Workbench workspace. -

- -
-
- Time - About 5 minutes -
-
- You will need - A Kubernetes cluster with compute pool, admin permissions on the workspace -
-
- Outcome - A shared service deployed in the workspace -
-
-
+| Service | What it does | +|---|---| +| [Cloud Shell](/how-to-guides/workbench/develop/cloud-shell) | Browser-based terminal with optional Slurm access | +| [Remote Desktop](/how-to-guides/workbench/develop/remote-desktop) | Full graphical environment via noVNC | +| [PVC Viewer](/how-to-guides/workbench/develop/pvc-viewer) | Web-based file browser for persistent volumes | ## Create a service -
-
- -### Open the Services page - -Click **Workbench** in the left sidebar, then click **Services**. - -
-
- -### Start creation - -Click `New Service`. - -
-
- -### Configure the service +Each service has its own creation page. Navigate to the service under **Workbench > Develop**, then click the **New** button. The creation dialog asks for: -Set the following fields: +- **Name** (optional) -- auto-generated if omitted +- **Compute Profile** -- required for Cloud Shell and Remote Desktop, optional for PVC Viewer +- **Slurm Cluster** (optional) -- available on Cloud Shell and Remote Desktop +- **Advanced options** -- image version, CPU, memory, and additional service-specific settings - - **Name** -- a unique identifier for the service - - **Image** -- the container image to run - - **Compute pool** -- the pool to deploy on - - **Port** -- the port the service listens on +## Open a service -
-
- -### Add optional settings - -Optionally add environment variables and resource limits (CPU, memory). - -
-
- -### Create - -Click `Create`. - -
-
- -Success looks like this: the service is started or updated, and the endpoint URL is accessible. - -The service transitions from **Pending** to **Running** once the container is healthy. - -## Check service status - -The Services list shows each service with its name, image, status, and internal endpoint. Click a service name to view its detail page with logs and connection information. - -## Connect to a service - -Running services are accessible from within the cluster network. Use the internal endpoint shown on the service detail page to connect from sessions or jobs. For example: - -``` -http://..svc.cluster.local: -``` +When the service status changes to **Running**, click **Open** on the list row or detail page to launch the service in your browser. ## Delete a service -
-
- -### Open the action menu +User services do not support pause or resume. When you are finished, click the row action menu and select **Delete** to free compute resources. -Click the **...** menu on the service row. - -
-
- -### Confirm deletion +
-Select **Delete** and confirm. +User services consume compute from your workspace's quota. GPU-tagged services left running idle are the most common source of preventable spend. Delete services you are not actively using.
-
- -Deleting a service terminates the container and frees compute resources.
## What to do next -- [Create a session](/how-to-guides/workbench/develop/creating-a-session) - [Use Cloud Shell](/how-to-guides/workbench/develop/cloud-shell) +- [Use the Remote Desktop](/how-to-guides/workbench/develop/remote-desktop) +- [Browse PVC data](/how-to-guides/workbench/develop/pvc-viewer)
diff --git a/docs/how-to-guides/workbench/develop/pvc-viewer.mdx b/docs/how-to-guides/workbench/develop/pvc-viewer.mdx index 18902ab..05f62df 100644 --- a/docs/how-to-guides/workbench/develop/pvc-viewer.mdx +++ b/docs/how-to-guides/workbench/develop/pvc-viewer.mdx @@ -1,15 +1,15 @@ --- -title: Browse PVC data -description: Browse, upload, and download files on a PersistentVolumeClaim without opening a session +title: Use PVC Viewer +description: Launch a browser-based file browser for workspace persistent volumes. --- -# Browse PVC data +# Use PVC Viewer -The PVC Viewer is a web-based file browser for PersistentVolumeClaims. Use it to browse directories, upload files, download data, and quick-edit text files without mounting the PVC locally or launching a full Workbench session. +PVC Viewer launches a lightweight browser-based file service for persistent-volume data in your Workbench workspace. Use it when you need quick file access without opening a full notebook or desktop session.

- Browse, upload, and download files on a PersistentVolumeClaim without opening a session. + Launch a browser-based file viewer without opening a session.

@@ -19,30 +19,30 @@ The PVC Viewer is a web-based file browser for PersistentVolumeClaims. Use it to
You will need - A Kubernetes cluster and at least one PVC + A Kubernetes cluster with a compute profile available for PVC Viewer
Outcome - PVC data browsable via the web viewer + A running PVC Viewer service you can open in your browser
-## Create a PVC Viewer +## Create a PVC Viewer service
### Open the PVC Viewer page -Click **Workbench** in the left sidebar, then click **PVC Viewer** under the Develop section. +Click **Workbench** in the left sidebar, then click **Develop > PVC Viewer**.
### Start creation -Click **+ New PVC Viewer** in the top-right corner. +Click **New PVC Viewer**.
@@ -51,16 +51,15 @@ Click **+ New PVC Viewer** in the top-right corner. | Field | Required | Notes | |---|---|---| -| Name | No | Auto-generated if omitted | -| PVC | Yes | The PersistentVolumeClaim to browse | -| Compute pool | No | Only pools tagged for PVC Viewer appear | +| Name | No | Auto-generated if omitted. If provided, it must be a valid Kubernetes name. | +| Compute Profile | No | The compute profile to run the file browser on. Leave blank to use the default compatible profile. |
### Configure advanced options (optional) -Expand **Advanced Options** to override the default filebrowser image version if needed. +Expand **Advanced Options** to override the default filebrowser image version. Leave it blank to use the cluster default.
@@ -72,7 +71,7 @@ Click **Create**.
-Success looks like this: the PVC viewer opens, and you can browse and edit files in the persistent volume through the web interface. +Success looks like this: the PVC Viewer service is created and moves to **Running**. ## Browse and manage files @@ -100,9 +99,9 @@ From the file browser you can: - **Download** files to your local machine - **Edit** text files directly in the browser -## Delete a PVC Viewer +## Delete a PVC Viewer service -Click the **...** menu and select **Delete**. This tears down the viewer only; your PVC and its data are not affected. +Click the row action menu and select **Delete**. This tears down the viewer service only; your persistent-volume data is not deleted. The PVC Viewer uses minimal compute. Delete it when you are finished to free the pod. diff --git a/docs/how-to-guides/workbench/develop/remote-desktop.mdx b/docs/how-to-guides/workbench/develop/remote-desktop.mdx index 6077a5f..e1d33ac 100644 --- a/docs/how-to-guides/workbench/develop/remote-desktop.mdx +++ b/docs/how-to-guides/workbench/develop/remote-desktop.mdx @@ -1,11 +1,11 @@ --- title: Use the Remote Desktop -description: Launch a full graphical environment on GPU compute via noVNC +description: Launch a full graphical environment on a Workbench compute profile via noVNC. --- # Use the Remote Desktop -Remote Desktop provides a full graphical Linux environment on GPU compute via noVNC in your browser. Use it for visualization tools, desktop IDEs, simulation software, or any workload that requires a GUI. +Remote Desktop provides a full graphical Linux environment through noVNC in your browser. Use it for visualization tools, desktop IDEs, simulation software, and other workloads that require a GUI.

@@ -19,7 +19,7 @@ Remote Desktop provides a full graphical Linux environment on GPU compute via no

You will need - A Kubernetes cluster with a compute pool tagged for Remote Desktop + A Kubernetes cluster with a compute profile available for Remote Desktop
Outcome @@ -35,14 +35,14 @@ Remote Desktop provides a full graphical Linux environment on GPU compute via no ### Open the Remote Desktop page -Click **Workbench** in the left sidebar, then click **Remote Desktop** under the Develop section. +Click **Workbench** in the left sidebar, then click **Develop > Remote Desktop**.
### Start creation -Click **+ New Remote Desktop** in the top-right corner. +Click **New remote desktop**.
@@ -51,10 +51,10 @@ Click **+ New Remote Desktop** in the top-right corner. | Field | Required | Notes | |---|---|---| -| Name | No | Auto-generated if omitted | -| Compute pool | Yes | Only pools tagged for Remote Desktop appear | +| Name | No | Auto-generated if omitted. If provided, it must be a valid Kubernetes name. | +| Compute Profile | Yes | The compute profile to run the desktop on. Only compatible profiles appear. | | Resolution | No | Default: 1920x1080. Options: 1024x768, 1280x1024, 1920x1080, 2560x1440, 3840x2160 | -| Slurm cluster | No | Enables Slurm job submission from the desktop | +| Slurm Cluster | No | Enables Slurm job submission from the desktop. Select **None** if you do not need Slurm access. |
@@ -65,10 +65,10 @@ Expand **Advanced Options** for additional settings: | Field | Notes | |---|---| -| Image version | Override the default TurboVNC image | -| CPU request / limit | Custom CPU allocation | -| Memory request / limit | Custom memory allocation | -| Extra labels | Key-value labels for the deployment | +| Image Version | Override the default TurboVNC image. Leave blank to use the cluster default. | +| CPU Request / CPU Limit | Custom CPU allocation. | +| Memory Request / Memory Limit | Custom memory allocation. | +| Extra Labels | Key-value labels for the deployment. |
@@ -110,7 +110,7 @@ The desktop provides: ## Delete a Remote Desktop -Remote Desktop does not support pause/resume. When you are finished, click the **...** menu and select **Delete** to free resources. GPU time is expensive, so delete desktops you are not actively using. +Remote Desktop does not support pause or resume. When you are finished, click the row action menu and select **Delete** to free resources. GPU time is expensive, so delete desktops you are not actively using.
diff --git a/docs/how-to-guides/workbench/get-started.mdx b/docs/how-to-guides/workbench/get-started.mdx index 38e51da..b658398 100644 --- a/docs/how-to-guides/workbench/get-started.mdx +++ b/docs/how-to-guides/workbench/get-started.mdx @@ -1,13 +1,13 @@ --- title: Get started -description: From a fresh workspace to a running notebook in under five minutes. +description: From a Workbench-enabled cluster to a running notebook in under five minutes. sidebar_position: 1 --- # Quickstart
-

Go from a fresh workspace to a running notebook in under five minutes.

+

Go from a Workbench-enabled cluster to a running notebook session in under five minutes.

Time @@ -15,7 +15,7 @@ sidebar_position: 1
You will need - A Workbench workspace with at least one compute pool + A Kubernetes cluster with Workbench enabled and at least one compute profile
Outcome @@ -29,33 +29,37 @@ sidebar_position: 1 ## Open Workbench -From the left sidebar, click **Workbench**. The secondary sidebar lists every tool in this guide. The status pill in the upper-left of that sidebar shows your active cluster; switch clusters there if your team works across regions. +From the left sidebar, click **Workbench**. The secondary sidebar opens with a Kubernetes cluster selector at the top and sections for **Develop**, **Compute**, **Train**, **Catalog**, and **Serve**. + +If your team uses more than one Kubernetes cluster, choose the cluster you want from the selector before creating resources.
-## Pick a compute pool (or accept the default) +## Check compute -Visit [Pools](/reference/workbench/compute-profiles) to see what's available. The default pool is fine for most starter notebooks; bigger pools (multi-GPU, A100/H100 nodes) require quota. +In **Compute > Pools**, confirm that at least one pool is available for session workloads. Pools define the node type, autoscaling range, GPU availability, and workload tags that Workbench uses when it schedules sessions and jobs.
## Create a session -On the [Sessions](/reference/workbench/sessions) tab, click New session. Pick a [preset](/reference/workbench/presets) (Jupyter, VS Code, or RStudio), choose an image and pod size, configure storage, and submit. +Go to **Develop > Sessions** and click New session. The wizard has three steps: **Preset**, **Compute**, and **Storage**. + +Pick a [preset](/reference/workbench/presets) such as JupyterLab, VS Code, or RStudio, choose an image, choose a pod size, then configure optional storage and Slurm access.
## Open it -When status flips from Pending to Running, click the row's **Open** button to launch the notebook in a new tab. +When the session status changes from **Pending** to **Running**, click **Open** on the row or detail page to launch the notebook or IDE in a new browser tab.
-Success looks like this: the workspace or session is ready, and you can start developing or running experiments. +Success looks like this: the session is running, the IDE opens in your browser, and your selected home and data volumes are mounted.
diff --git a/docs/how-to-guides/workbench/observability.mdx b/docs/how-to-guides/workbench/observability.mdx index 26767dd..ce23048 100644 --- a/docs/how-to-guides/workbench/observability.mdx +++ b/docs/how-to-guides/workbench/observability.mdx @@ -7,21 +7,30 @@ description: Monitor GPU utilization, spend, and active alerts across your Workb The Observability page gives you cluster-wide rollups of utilization, spend, idle GPU hours, and live alerts. It is the first page to check when reviewing resource usage or diagnosing cost spikes. -## Prerequisites - -- A Kubernetes cluster in **ready** status with Prometheus and Grafana enabled (default for Kubernetes clusters) - -
-
- -## Open the Observability dashboard - -Click **Workbench** in the left sidebar, then click **Observability** at the bottom of the Workbench sidebar. - -
+
+

+ Monitor GPU utilization, spend, and active alerts across your Workbench. +

+ +
+
+ Time + About 2 minutes +
+
+ You will need + A Kubernetes cluster with Prometheus and Grafana enabled +
+
+ Outcome + Metrics visible in the observability dashboard +
+
-Success looks like this: metrics and logs are visible in the observability dashboard, and you can monitor session and job performance. +Click **Workbench** in the left sidebar, then click **Observability** near the top of the Workbench sidebar. + +Metrics and logs are visible in the observability dashboard, and you can monitor session and job performance. ## KPI tiles @@ -34,7 +43,7 @@ The top of the page shows six tiles: | Running training jobs | Count plus total GPU-hours consumed today | | Spend today | Current burn rate compared to the daily ceiling | | Spend MTD | Month-to-date spend with an end-of-month forecast | -| Idle GPU hours | Hours where GPU utilization was below 5%; the target to drive down | +| Idle GPU hours | GPU hours where utilization was below 5%; the metric to drive down | If **Idle GPU hours** exceeds 20% of **Spend today**, sessions are likely being left open without active use. @@ -50,7 +59,11 @@ Three Grafana panels provide detailed breakdowns: Use the time range dropdown to switch between **1 hour**, **24 hours**, **7 days**, and **30 days**. The KPI deltas re-baseline against the matching prior window. -## Related +
+ +## What to do next - [Manage sessions](/how-to-guides/workbench/develop/managing-sessions) to pause idle sessions - [Configure autoscaling](/how-to-guides/workbench/serve/autoscaling) for endpoint cost control + +
diff --git a/docs/how-to-guides/workbench/serve/autoscaling.mdx b/docs/how-to-guides/workbench/serve/autoscaling.mdx index 03f02a5..1ceee05 100644 --- a/docs/how-to-guides/workbench/serve/autoscaling.mdx +++ b/docs/how-to-guides/workbench/serve/autoscaling.mdx @@ -1,18 +1,98 @@ --- -title: "Autoscaling" -description: "Replica bounds, autoscaling metrics, and scale-to-zero." +title: Configure endpoint autoscaling +description: Use inference presets and endpoint sizing to control replica bounds. --- -# Autoscaling +# Configure endpoint autoscaling -Replica bounds, autoscaling metrics, and scale-to-zero. +Endpoint scaling is controlled by the sizing configuration sent when an endpoint is deployed. In the current Workbench UI, endpoint presets populate the CPU, memory, GPU count, compute pool, configurations, and replica bounds used by the deployment. -Endpoints scale between `min_replicas` and `max_replicas` based on a metric: +
+

+ Review and apply endpoint replica bounds through inference presets. +

-- **concurrency:** number of in-flight requests per replica (good default for synchronous APIs). -- **rps:** requests-per-second per replica. -- **cpu:** CPU saturation as a percentage. +
+
+ Time + About 3 minutes +
+
+ You will need + A Workbench endpoint or access to inference presets +
+
+ Outcome + Replica bounds understood before deployment +
+
+
-Set `min_replicas` to 0 and turn on **scale to zero** for a development endpoint that costs nothing when idle. Cold-start latency goes up; the trade is yours. +## Review preset sizing -Success looks like this: the autoscaling policy is applied, and the endpoint scales up or down based on traffic. +
+
+ +### Open inference presets + +Click **Workbench** in the left sidebar, then click **Compute > Presets** and select the **Inference** tab. + +
+
+ +### Open a preset + +Click the preset you plan to use for the endpoint. + +
+
+ +### Check replica bounds + +Review the preset's CPU, memory, GPU count, compute pool, configurations, minimum replicas, and maximum replicas. + +
+
+ +## Apply autoscaling during deployment + +
+
+ +### Deploy an endpoint + +Start the [Deploy endpoint](/how-to-guides/workbench/serve/deploying) workflow. + +
+
+ +### Select a preset + +On the **Tuning** step, select the inference preset that has the replica bounds you want. The wizard uses the preset to populate the endpoint sizing payload. + +
+
+ +### Confirm the deployed endpoint + +After deployment, open the endpoint detail page. The **Related components** panel shows the autoscaler when min or max replicas are configured, and the **Status** panel shows ready, available, and target replica counts. + +
+
+ +Success looks like this: the endpoint detail page shows the expected target replica count and autoscaler-related component information. + +
+ +Scale-to-zero requires a preset with a minimum replica count of `0`. Expect cold-start latency when traffic resumes after the endpoint has scaled down. + +
+ +
+ +## What to do next + +- [Deploy an endpoint](/how-to-guides/workbench/serve/deploying) +- [Endpoints reference](/reference/workbench/endpoints) + +
diff --git a/docs/how-to-guides/workbench/serve/canary-rollouts.mdx b/docs/how-to-guides/workbench/serve/canary-rollouts.mdx index 46ec1c3..ec2c993 100644 --- a/docs/how-to-guides/workbench/serve/canary-rollouts.mdx +++ b/docs/how-to-guides/workbench/serve/canary-rollouts.mdx @@ -1,21 +1,71 @@ --- -title: "Canary rollouts" -description: "Split traffic across endpoint versions; promote or rollback." +title: Monitor canary rollouts +description: Check whether an endpoint is splitting traffic to a candidate version. --- -# Canary rollouts +# Monitor canary rollouts -Split traffic across endpoint versions; promote or rollback. +Workbench surfaces canary traffic information on an endpoint detail page when the backend reports a candidate receiving traffic. -When you push a new version of an endpoint, Workbench can split traffic, say, 90% to the existing version, 10% to the new one. The detail page shows a live canary % and gives you two primary actions: +
+

+ Check whether an endpoint is serving canary traffic and inspect its current traffic percentage. +

-- Promote, shift 100% to the new version. Old version drains and is removed. -- Rollback, shift 100% back to the old version. New version is removed. +
+
+ Time + About 1 minute +
+
+ You will need + An endpoint with canary status reported by the backend +
+
+ Outcome + The current canary traffic percentage +
+
+
+ +
+
+ +## Open endpoints + +Click **Workbench** in the left sidebar, then click **Serve > KServe > Inferences**. + +
+
+ +## Open the endpoint + +Click the endpoint row to open its detail page. + +
+
+ +## Check status + +In the **Status** panel, look for **Canary traffic**. When present, it shows the percentage routed to the candidate version. + +
+
+ +Success looks like this: the endpoint detail page shows the current canary percentage, for example `10% to candidate`.
-Treat Promote as the only "deploy succeeded" signal. A canary that's been stable at 10% for an hour with no error spikes is your green light. +The current Workbench UI displays canary traffic state but does not expose promote or rollback buttons on the endpoint detail page. Use your team's release process or API workflow to change rollout state.
-Success looks like this: the canary deployment is active, and traffic is split between the stable and canary versions as configured. +
+ +## What to do next + +- [Deploy an endpoint](/how-to-guides/workbench/serve/deploying) +- [Use Workbench observability](/how-to-guides/workbench/observability) +- [Endpoints reference](/reference/workbench/endpoints) + +
diff --git a/docs/how-to-guides/workbench/serve/deploying.mdx b/docs/how-to-guides/workbench/serve/deploying.mdx index 64d7df7..634cd2e 100644 --- a/docs/how-to-guides/workbench/serve/deploying.mdx +++ b/docs/how-to-guides/workbench/serve/deploying.mdx @@ -1,62 +1,109 @@ --- -title: "Deploying an endpoint" -description: "Seven-step wizard from model selection to live URL." +title: Deploy an endpoint +description: Deploy a model as a Workbench inference endpoint. --- -# Deploying an endpoint - -Seven-step wizard from model selection to live URL. +# Deploy an endpoint + +Deploy an endpoint when you want to serve a model behind an authenticated HTTP URL. Workbench supports LLM and predictive endpoints, multiple model sources, and runtime tuning through inference presets. + +
+

+ Deploy a model as a Workbench inference endpoint using the three-step endpoint wizard. +

+ +
+
+ Time + About 5 minutes, plus model startup time +
+
+ You will need + A model source and at least one inference preset/runtime +
+
+ Outcome + A running inference endpoint with a request URL +
+
+
-## Pick a kind +## Open KServe endpoints -Predictive or LLM. +Click **Workbench** in the left sidebar, then click **Serve > KServe > Inferences**.
-## Pick a model source +## Start the endpoint wizard -From your registry, from HuggingFace, from S3, from a PVC, or a custom container image. +Click **Deploy endpoint**. The **Deploy Inference Endpoint** modal opens.
-## Pick a runtime +## Complete Identity -Workbench filters runtimes by your kind and whether they need a GPU. +Enter an **Endpoint name** and optional **Description**.
-## Set sizing +## Complete Model source + +Select a source type and fill in the matching fields. + +| Source type | What you provide | +|---|---| +| **S3** | Bucket and optional prefix. | +| **HuggingFace** | Repository, optional revision, and optional HuggingFace secret for gated models. | +| **PVC** | Claim name and optional path. | +| **Model Registry** | Registered model and version. | +| **Custom URI** | Container or model URI. | -CPU, memory, GPU, replica bounds (min/max), and the autoscaling metric (concurrency / RPS / CPU). +For non-HuggingFace sources, also choose the endpoint **Kind**: **LLM** or **Predictive**. HuggingFace sources use runtime suggestions to infer the kind when possible.
-## Pick a compute pool +## Complete Tuning -Usually a GPU pool for LLMs and a smaller predictive box otherwise. +Select an inference **Preset**, **Runtime**, and **Framework**. Presets fill in sizing values such as CPU, memory, GPU count, compute pool, configuration defaults, and replica bounds.
-## Configure parallelism (LLM only) +## Deploy -Tensor / pipeline / data sharding to fit the model on your GPUs. +Click **Deploy endpoint**.
-## Submit +## Open endpoint details -The endpoint goes Pending while images pull and pods schedule, then Running once at least one replica passes its readiness check. +The endpoint appears in the Inferences list. Click the row to open the detail page, view status, copy the request URL, and inspect related components.
-Success looks like this: the model is deployed, the endpoint becomes available, and inference requests return predictions. +Success looks like this: the endpoint reaches **Running**, at least one predictor replica is ready, and requests to the displayed URL return responses. + +
+ +LLMs can report an active Kubernetes service before weights are fully loaded into GPU memory. If requests time out immediately after deployment, check the endpoint detail page and logs before changing the configuration. + +
+ +
+ +## What to do next + +- [Configure autoscaling](/how-to-guides/workbench/serve/autoscaling) +- [Register a model](/how-to-guides/workbench/train/registering-a-model) +- [Endpoints reference](/reference/workbench/endpoints) + +
diff --git a/docs/how-to-guides/workbench/train/configure-runtime.mdx b/docs/how-to-guides/workbench/train/configure-runtime.mdx index 97d48b3..585242f 100644 --- a/docs/how-to-guides/workbench/train/configure-runtime.mdx +++ b/docs/how-to-guides/workbench/train/configure-runtime.mdx @@ -29,17 +29,6 @@ A training runtime is a pre-built environment that combines a deep learning fram
-## Available runtimes - -Vantage ships with several built-in runtimes: - -| Runtime | Framework | Use case | -|---|---|---| -| `torch-distributed` | PyTorch + torchrun | General multi-GPU training | -| `deepspeed-zero3` | DeepSpeed | Memory-bound LLMs with ZeRO-3 sharding | -| `mlx-distributed` | MLX | Apple-silicon training | -| `huggingface-trainer` | Transformers | Standard HF Trainer workflows | - ## View runtimes
@@ -54,14 +43,14 @@ Click **Workbench** in the left sidebar. ### Navigate to Runtimes -Navigate to **Training Jobs > Runtimes** (or find runtimes in the Training section). +Navigate to **Train > Training > Runtimes**.
### Browse the list -The list shows each runtime with its framework, image, and status. +The list shows each runtime with its name, framework, description, base image, and status.
@@ -73,7 +62,7 @@ The list shows each runtime with its framework, image, and status. ### Open the creation form -On the Runtimes page, click `Create Runtime`. +On the Runtimes page, click **Create Runtime**.
@@ -82,17 +71,23 @@ On the Runtimes page, click `Create Runtime`. Set the following fields: - - **Name** -- a short identifier (for example, `jax-tpu` or `custom-pytorch`) - - **Framework** -- the ML framework (PyTorch, DeepSpeed, JAX, or custom) - - **Container image** -- the Docker image URI (for example, `pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime`) - - **Default entrypoint** -- the command or script that runs when a job starts (for example, `torchrun`) +| Field | Required | What it does | +|---|---|---| +| **Runtime name** | required | A short identifier, for example `pytorch-distributed`. | +| **Framework** | required | The ML framework: `torch`, `deepspeed`, `mlx`, `huggingface`, or `custom`. Determines which ML policy fields are shown. | +| **Base image** | required | The container image URI, for example `docker.io/pytorch/pytorch:2.1.0-cuda11.8-cudnn8-runtime`. | +| **Number of nodes** | optional | Number of nodes for multi-node training. Shown when a framework is selected. | +| **Processes per node** | optional | Enter `auto` or a positive integer. Available for torch, deepspeed, and huggingface frameworks. | +| **NCCL config** | optional | DeepSpeed only. `KEY=VALUE` pairs for NCCL configuration. | +| **Compute pool** | optional | Optional compute pool to constrain where the runtime runs. | +| **Description** | optional | Short description of this runtime. |
### Create -Click `Create`. +Click **Create runtime**.
@@ -113,18 +108,18 @@ Click the runtime name to open the detail page. ### Edit and save -Click `Edit` and modify the image, entrypoint, or other settings. Click `Save`. +Click **Edit** and modify the image, policy, or other settings. Click **Save**.
## Set a default runtime -Click the **Set as default** action on a runtime row. The default runtime is pre-selected when creating new training jobs. +On the Runtimes page, click the **Set as default** action on a runtime row. The default runtime is pre-selected when creating new training jobs.
-Runtimes are workspace-scoped. Admins can publish runtimes that are available to all users in the workspace. +Runtimes are workspace-scoped. Administrators can publish runtimes that are available to all users in the workspace.
@@ -132,7 +127,6 @@ Runtimes are workspace-scoped. Admins can publish runtimes that are available to ## What to do next -- [Create a training preset](/how-to-guides/workbench/train/create-preset) - [Submit a training job](/how-to-guides/workbench/train/submitting-a-job) - [Runtimes reference](/reference/workbench/training-jobs/runtimes) diff --git a/docs/how-to-guides/workbench/train/create-preset.mdx b/docs/how-to-guides/workbench/train/create-preset.mdx index 5019552..a02e1de 100644 --- a/docs/how-to-guides/workbench/train/create-preset.mdx +++ b/docs/how-to-guides/workbench/train/create-preset.mdx @@ -43,94 +43,49 @@ Click **Workbench** in the left sidebar. ### Navigate to Presets -Navigate to **Training Jobs > Presets**. +Navigate to **Compute > Presets**, then select the **Training** tab.
### Browse the sections -The page shows three sections: - - **Named presets** -- full configurations (runtime + sizing + defaults) - - **Sizing presets** -- compute-only configurations (nodes, CPU, memory, GPU) - - **TTL and defaults** -- default time-to-live and resource limits +The Training tab lists named presets with their key, runtime, sizing preset, and description. Click a row to open the detail drawer.
-## Create a named preset +## Review a training preset
-### Open the creation form +### Open a preset -On the Presets page, click `Create Preset`. +Click a preset row.
-### Configure the preset +### Review the configuration -Set the following fields: - - - **Key** -- a unique identifier (for example, `pytorch-single-gpu` or `deepspeed-multi-node`) - - **Runtime** -- select a training runtime - - **Sizing** -- select a sizing preset or configure custom compute resources - - **Description** -- a short summary of the intended use case +The detail drawer shows the runtime, sizing preset, and resolved resource settings. Use this to confirm which preset to pick when launching a training job.
-### Create +### Launch from the preset -Click `Create`. +If you want to use the preset, go to **Train > Training > Training Jobs**, click **New Training Job**, choose **Preset** mode, and select the preset key.
-Success looks like this: the training preset is created and appears in the preset list, ready to be used for training jobs. - -## Create a sizing preset - -Sizing presets define compute resources independently of a runtime: - -
-
- -### Navigate to Sizing presets +Success looks like this: you know which preset key to use when submitting a training job. -Navigate to the **Sizing presets** section. +## Create or modify training presets -
-
- -### Open the creation form - -Click `Create Sizing Preset`. - -
-
- -### Configure the sizing preset - -Set the following fields: - - - **Key** -- a unique identifier (for example, `small-1x-gpu`) - - **Display name** -- a human-readable label - - **Nodes** -- number of nodes - - **CPU / Memory** -- per-node resources - - **GPU** -- GPU type and count per node - -
-
- -### Create - -Click `Create`. - -
-
+Training presets are reported by the cluster and managed by workspace administrators. The current UI lets users browse and select them, but creation and modification are not exposed from the Training tab. ## Use a preset when submitting a job @@ -139,21 +94,21 @@ Click `Create`. ### Open the job form -Navigate to **Training Jobs** and click `Submit Training Job`. +Navigate to **Train > Training > Training Jobs** and click **New Training Job**.
### Select a preset -In the configuration form, click `Use Preset` and select a named preset. +In the configuration form, choose **Preset** mode and select a named preset.
### Review and submit -The runtime, sizing, and defaults auto-fill from the preset. Override any fields as needed, then submit. +The runtime and sizing are derived from the preset. Add a job name, optional compute pool, TTL, and output override if needed, then click **Launch job**.
diff --git a/docs/how-to-guides/workbench/train/creating-a-sweep.mdx b/docs/how-to-guides/workbench/train/creating-a-sweep.mdx index 3d56cc0..756db18 100644 --- a/docs/how-to-guides/workbench/train/creating-a-sweep.mdx +++ b/docs/how-to-guides/workbench/train/creating-a-sweep.mdx @@ -9,7 +9,7 @@ A sweep runs many trials, each with a different combination of hyperparameters.

- Run a systematic hyperparameter search — grid, random, or Bayesian — and find the best configuration. + Run a systematic hyperparameter search — grid, random, or Bayesian — and find the best configuration using the Workbench UI or the Vantage SDK.

@@ -19,7 +19,7 @@ A sweep runs many trials, each with a different combination of hyperparameters.
You will need - Vantage SDK installed, a Kubernetes cluster with compute pool + A Kubernetes cluster, algorithms configured, and a training runtime
Outcome @@ -38,6 +38,74 @@ Vantage supports three sweep algorithms: | **Random** | Large spaces, quick exploration | Samples random combinations from defined ranges | | **Bayesian** | Expensive training, adaptive search | Uses prior trial results to pick the next combination | +## Create a sweep in the UI + +
+
+ +### Open Sweeps + +Click **Workbench** in the left sidebar, then click **Train > Sweeps**. + +
+
+ +### Start creation + +Click **New sweep**. The **New Hyperparameter Sweep** dialog opens. + +
+
+ +### Configure the sweep + +Set the following fields: + +| Field | What it does | +|---|---| +| **Sweep name** | A unique name for the sweep. | +| **Algorithm** | The search algorithm: grid, random, or the configured Bayesian algorithm. | +| **Objective type** | `maximize` or `minimize` the objective metric. | +| **Objective metric** | The metric to optimize, for example `val_auc`. | +| **Goal** | Optional target value. | +| **Parameters (JSON)** | The parameter search space as a JSON array of `{ name, type, feasible_space }` objects. | +| **Parallel trials, Max trials, Max failed** | Trial count limits. | +| **Runtime name, Runtime kind** | The runtime to execute each trial. | +| **Compute pool** | Where to run the trials. | + +
+
+ +### Advanced configuration (optional) + +Expand **Advanced Options** to set: + +| Field | What it does | +|---|---| +| **Trainer image** | Override the runtime container image. | +| **Trainer command** | JSON array overriding the container entrypoint. | +| **Trainer args** | JSON array of argument strings. | +| **Trainer env** | `KEY=VALUE` environment variables. | +| **Assignment strategy** | How to pass parameters to the trial: `args` or `env`. | +| **Parameters per node** | Nodes, CPU, memory, GPU count, and GPU type. | +| **Metrics collector kind** | How trial metrics are collected. | + +
+
+ +### Launch + +Click **Launch sweep**. + +
+
+ +
+ +Sweeps are in preview. The detail view shows the best trial, a sortable trial table, and accumulated cost. See [Sweep algorithms](/explanation/workbench/sweep-algorithms) for details on each strategy. + +
+ ## Create a sweep with the SDK ```python @@ -66,41 +134,31 @@ sweep = client.sweeps.create( print(f"Sweep started: {sweep.id}") ``` -Success looks like this: the hyperparameter sweep is created, and the individual runs begin executing on the compute cluster. +Success looks like this: the sweep is launched, trials begin executing on the cluster, and the sweep detail page shows progress. ## Monitor a sweep in the UI
-### Open the Sweeps page - -Click **Workbench** in the left sidebar, then click **Sweeps** under the Train section. - -
-
- -### Find your sweep +### Open the Sweep detail page -The sweep list shows each sweep with its name, algorithm, objective metric, number of trials, and best metric value so far. +Click a sweep row in the list to open its detail page.
-### View sweep details +### Review trial progress -Click a sweep to view: - - **Trials** -- each trial's parameters and resulting metric, sorted by objective - - **Best trial** -- the parameter combination that produced the best result - - **Presets** -- saved configurations that can seed new sweeps +The detail page shows: + - **Best trial** -- the current leader, with its parameters and metric + - **Trial table** -- every trial, sortable by status, parameter, or objective + - **Parallel-coordinates chart** -- every parameter on its own axis, lines colored by metric + - **Spend so far** -- accumulated cost across all trials
-## Use a sweep preset - -Presets are pre-defined configurations for common search patterns. Select an existing preset when creating a sweep to start from a known-good configuration instead of defining every parameter from scratch. -
## What to do next diff --git a/docs/how-to-guides/workbench/train/registering-a-model.mdx b/docs/how-to-guides/workbench/train/registering-a-model.mdx index fda7d87..58dc5d0 100644 --- a/docs/how-to-guides/workbench/train/registering-a-model.mdx +++ b/docs/how-to-guides/workbench/train/registering-a-model.mdx @@ -1,28 +1,99 @@ --- -title: "Registering a model" -description: "Pull a model from HuggingFace, a local path, or a URL into the Vantage registry." +title: Register a model +description: Add a model from HuggingFace, S3, a URL, or a local path to the Workbench model registry. --- -# Registering a model +# Register a model -Pull a model from HuggingFace, a local path, or a URL into the Vantage registry. +The Workbench model registry stores versioned model records that can be deployed to inference endpoints. Register a model when you want a reusable handoff between training, evaluation, and serving. -Click Register model. Pick a source, set a name and version, optionally tag with labels and properties. +
+

+ Add a model to the registry from HuggingFace, S3, a URL, or a local path. +

-## Source types +
+
+ Time + About 3 minutes, plus transfer time +
+
+ You will need + Model source details and access to the Workbench model registry +
+
+ Outcome + A versioned model available in the catalog +
+
+
+ +
+
+ +## Open Models + +Click **Workbench** in the left sidebar, then click **Catalog > Local**. + +
+
+ +## Start registration + +Click **Register Model**. The **Register Model** modal opens. + +
+
+ +## Set model identity + +Enter a **Model name** and **Version**. The default version is `v0.1`. + +
+
+ +## Choose a source type + +Pick the source that matches where the model artifacts already live. | Source type | What you provide | |---|---| -| **HuggingFace** | Repo ID (e.g. `mistralai/Mistral-7B-v0.3`), revision (default `main`), optional include/exclude glob patterns, optional auth token for gated models. | -| **Local path** | An absolute path on the Vantage backend pod, typically used by automation, not humans. | -| **URL** | A single HTTP(S) URL. Good for one-off weights or custom safetensors. | +| **HuggingFace** | Repo ID, optional revision, optional allow patterns, and optional ignore patterns. | +| **S3** | S3 URI and optional model-format hint. This is useful for artifacts created by training jobs or sweeps. | +| **URL** | HTTP(S) URL and optional destination filename. | +| **Local** | Absolute path on the `vdeployer-web` pod. This is mostly for automation and platform administrators. | -Registration runs as a background job. You'll see a progress bar in the dialog, and the model lands in the list with status Active when complete. +
+
+ +## Add metadata + +Select **Framework** and **Task**. Expand **Advanced Options** if you want to add a description, base model, destination bucket, destination prefix, or overwrite behavior. + +
+
+ +## Register -Success looks like this: the model is registered in the model catalog, and it can be referenced for deployment or further training. +Click **Register**. + +
+
+ +Success looks like this: Workbench submits the model onboarding request and the model appears in the registry with the version you provided.
-Storage destination defaults to your workspace's S3 bucket and the `models/{name}/{version}/` prefix. Override only if you need a custom layout. +For gated HuggingFace models, make sure your workspace has the required credential secret configured before deployment. Model registration captures the model record; endpoint deployment still needs access to fetch and serve the weights. + +
+ +
+ +## What to do next + +- [Deploy an endpoint](/how-to-guides/workbench/serve/deploying) +- [Submit a training job](/how-to-guides/workbench/train/submitting-a-job) +- [Models reference](/reference/workbench/models)
diff --git a/docs/how-to-guides/workbench/train/submitting-a-job.mdx b/docs/how-to-guides/workbench/train/submitting-a-job.mdx index 8b88aea..ee0d44a 100644 --- a/docs/how-to-guides/workbench/train/submitting-a-job.mdx +++ b/docs/how-to-guides/workbench/train/submitting-a-job.mdx @@ -1,13 +1,13 @@ --- -title: "Submitting a job" -description: "Six-step wizard: runtime, sizing, initializers, overrides, output, TTL." +title: Submit a training job +description: Launch a training job from a preset or from a custom runtime and sizing configuration. --- -# Submitting a job +# Submit a training job

- Walk through the six-step wizard to configure and submit a training job. + Launch a Workbench training job from a preset or from a custom runtime, compute, and output configuration.

@@ -17,63 +17,84 @@ description: "Six-step wizard: runtime, sizing, initializers, overrides, output,
You will need - A Kubernetes cluster with compute pool, a runtime configured + A Kubernetes cluster with Workbench enabled and either a training preset or a training runtime
Outcome - A training job submitted and running + A training job queued or running in Workbench
-Six-step wizard: runtime, sizing, initializers, overrides, output, TTL. +Training jobs run finite ML workloads on Kubernetes compute. The **New Training Job** modal supports two creation modes: + +| Mode | Use when | +|---|---| +| **Preset** | Your team already has a saved training configuration. This is the fastest and safest path. | +| **Custom** | You need to choose the runtime, sizing, output destination, trainer overrides, or labels manually. |
-## Pick a runtime +## Open Training Jobs -The wizard filters compute pools to those compatible with the runtime's framework. +Click **Workbench** in the left sidebar, then click **Train > Training > Training Jobs**.
-## Set sizing +## Start a new job -Number of nodes, CPU / memory / GPU per node, and processes-per-node (usually `auto` = 1 per GPU). +Click **New Training Job**. The modal opens with a **Create mode** toggle.
-## Initializers (optional) +## Choose Preset or Custom mode + +For **Preset** mode, select a preset, optional compute pool, job name, and optional TTL. Preset mode derives the runtime and sizing from the selected preset. -Tell Workbench to fetch a dataset and a base model *before* training starts, from S3, HuggingFace, a PVC, or your model registry. Cuts startup time and avoids cold pulls inside the training loop. +For **Custom** mode, enter a job name, runtime kind, runtime name, optional compute pool, node count, CPU per node, memory per node, GPUs per node, and optional GPU vendor.
-## Trainer overrides (optional) +## Configure optional output and overrides -Custom command, args, env. Useful when the runtime's default entrypoint isn't quite right. +Use **Override output destination** in Preset mode, or **Trainer overrides** in Custom mode, when you need to change where outputs are written or how the runtime starts your workload. + +| Option | What it does | +|---|---| +| **Output type** | Use the cluster default, an S3 URI, or a PVC destination. | +| **Trainer command** | JSON array that overrides the runtime container entrypoint. | +| **Trainer args** | JSON array of argument strings passed to the trainer. | +| **Environment** | `KEY=VALUE` pairs passed to the trainer. | +| **Labels** | `KEY=VALUE` pairs attached to the job for filtering and ownership. |
-## Output destination +## Launch the job -Where final checkpoints go: S3 bucket+prefix or a PVC. Defaults to `s3://{workspace}/trainjobs/{name}/`. +Click **Launch job**.
-## TTL +## Monitor the job -How long the completed pods stick around for log retrieval. Defaults: 1d on success, 7d on failure. +The job appears in the Training Jobs list. Click the row to inspect status, runtime, sizing, logs, output destination, and related Kubernetes resources.
-Success looks like this: the training job is queued, and the job status moves from pending to running on the compute cluster. +Success looks like this: the training job is created, appears in the list, and moves from pending to running when the requested compute is available. + +
+ +Use **Preset** mode unless you need to override the runtime or resource sizing. Presets reduce repeated configuration and help teams submit jobs with known-good settings. + +
diff --git a/docs/how-to-guides/workbench/troubleshooting.mdx b/docs/how-to-guides/workbench/troubleshooting.mdx index c460748..f5cb38a 100644 --- a/docs/how-to-guides/workbench/troubleshooting.mdx +++ b/docs/how-to-guides/workbench/troubleshooting.mdx @@ -9,7 +9,7 @@ description: Common Workbench issues and how to resolve them. Three usual causes: -1. **No node has the requested GPU.** Check the compute pool's max, if you're at the autoscaler ceiling, the request will queue. The session's **Activity** tab will say "FailedScheduling" if so. +1. **No node has the requested GPU.** Check the compute pool's max size. If the pool is at the autoscaler ceiling, the request will queue. The session's **Activity** tab will say `FailedScheduling` if so. 1. **Image pull is slow.** First-time pulls of large CUDA images can take 5+ minutes. Subsequent starts on the same node are instant. 1. **PVC binding pending.** If you mounted a brand-new persistent volume, it has to be provisioned. Wait, or check storage status. @@ -19,22 +19,22 @@ Confirm three things: 1. You're hitting the right URL (the detail page header is canonical). 1. Your Vantage API key is in `Authorization: Bearer …`, not as a query param. -1. The model finished loading. The endpoint reports Running as soon as one replica is healthy, but big LLMs take a few minutes after that to load weights into GPU memory. The Logs tab will show "model ready" when the inference engine is actually serving. +1. The model finished loading. The endpoint reports Running as soon as one replica is healthy, but large models take a few minutes after that to load weights into GPU memory. The Logs tab will show `model ready` when the inference engine is actually serving. ## Training job logs are empty By default the Logs tab streams the trainer master pod. If your master finished early but workers are still going, the master log will be quiet. Use the container/rank dropdown to switch. -## Pipeline / Sweep limitations +## Pipeline and sweep limitations Both tabs are in **Preview**. Today: -- Pipeline authoring is API-only (Vantage SDK). The UI shows runs and DAGs but doesn't let you author or upload definitions. -- Sweep creation is API-only. The list and detail views work end-to-end. +- Pipeline authoring is API-first. The UI shows experiments, pipelines, runs, recurring runs, artifacts, and executions. +- Sweep creation is available from the UI when algorithms, runtimes, and compute pools are configured. The list and detail views show trials and sweep progress. - Cost tiles on these tabs may be missing or stale; we're wiring billing into them. ## Cost tiles read $0 or look wrong Workbench computes spend from compute pool rates × elapsed runtime. If your admin hasn't configured a rate for a pool, that pool's resources will read $0. This is a configuration issue, not a measurement bug, open an admin ticket. -Success looks like this: the diagnostic identifies the root cause, and applying the recommended fix restores normal workbench operation. + diff --git a/sidebars-main.js b/sidebars-main.js index 40e545c..25bfb8f 100644 --- a/sidebars-main.js +++ b/sidebars-main.js @@ -92,12 +92,12 @@ module.exports = { doc('how-to-guides/workbench/develop/creating-a-preset', 'Create a session preset'), doc('how-to-guides/workbench/develop/cloud-shell', 'Use Cloud Shell'), doc('how-to-guides/workbench/develop/remote-desktop', 'Use the Remote Desktop'), - doc('how-to-guides/workbench/develop/pvc-viewer', 'Browse PVC data'), - doc('how-to-guides/workbench/develop/manage-services', 'Manage services'), + doc('how-to-guides/workbench/develop/pvc-viewer', 'Use PVC Viewer'), + doc('how-to-guides/workbench/develop/manage-services', 'Manage user services'), ]), category('Compute', null, [ doc('how-to-guides/workbench/compute/create-pool', 'Create a compute pool'), - doc('how-to-guides/workbench/compute/managing-profiles', 'Manage compute profiles'), + doc('how-to-guides/workbench/compute/managing-profiles', 'Manage compute pools'), ]), category('Train', null, [ doc('how-to-guides/workbench/train/configure-runtime', 'Configure a training runtime'), @@ -114,7 +114,7 @@ module.exports = { category('Serve', null, [ doc('how-to-guides/workbench/serve/deploying', 'Deploy an endpoint'), doc('how-to-guides/workbench/serve/autoscaling', 'Configure autoscaling'), - doc('how-to-guides/workbench/serve/canary-rollouts', 'Run a canary rollout'), + doc('how-to-guides/workbench/serve/canary-rollouts', 'Monitor canary rollouts'), ]), doc('how-to-guides/workbench/observability', 'Use observability'), doc('how-to-guides/workbench/troubleshooting', 'Troubleshoot Workbench'),