# DataHub product guide Turn images into labeled datasets, then train and inspect supported models in the same project. DataHub connects data browsing, annotation, dataset versions, and model runs so you can follow an experiment back to the examples it used. ## What can I do with DataHub? | Your goal | Start here | What you get | | --- | --- | --- | | Find and organize useful examples | Project image grid and filters | A focused collection for labeling or review | | Teach a model what to recognize | Annotation editor | Classes, shapes, text, masks, or ordered landmarks | | Make a training-ready dataset | Export Data | A version with selected data, processing, splits, and downloadable formats | | Train on your own hardware | Model Training and Workers | A tracked run with logs, metrics, and output files | | Compare and use a model | Evaluate, Benchmark, Convert, and inference actions | Reports, converted files, and saved predictions | | Repeat a workflow programmatically | Python SDK | Scripts using the same projects and permissions | ## Follow a complete computer vision workflow Imagine you want to detect damaged packaging. Create a project, upload representative photos, and draw boxes or masks around the damage. Review examples with a teammate, then build a version that resizes the images and adds suitable training variation. Train a model on a connected worker, inspect its held-out results, and try predictions on an image it has not seen. The same sequence applies to other tasks. For a sports project, you might label player joints or court landmarks. For OCR, you might label text regions and transcripts. The annotation type and chosen trainer define the details. ### Pick the output your application needs | Example task | Label the data | Prepare the model input | Inspect the result | | --- | --- | --- | --- | | Detect damaged packaging | Draw a box around each damaged area; use one consistent class name | Build a reviewed version and generate a detection export accepted by your trainer | Check missed damage and false positives on held-out images | | Read text on labels | Mark text regions and enter their transcripts | Choose a detection export for locating text, or recognition crops and transcripts for reading it | Inspect region placement separately from the recognized words | | Locate joints or court landmarks | Choose a skeleton, place ordered keypoints, and mark visibility | Keep landmark order and flip pairs consistent; use a compatible keypoint export | Inspect point positions and connections, including occluded or unlabelled points | Start with a small representative batch and inspect one exported sample before building a larger dataset. DataHub also supports scientific raster and medical-image workflows where enabled; check [supported source formats](/docs/upload-data) before importing those files. ## Start with a project 1. Open [Workspaces](/workspaces), choose a workspace, and create or open a project. 2. [Upload data](/docs/upload-data) and review the imported images and labels. 3. [Annotate and verify](/docs/annotations) the data. 4. [Build a dataset version and export it](/docs/dataset-versions-exports). 5. Download the dataset or [train a supported model](/docs/model-training) using a registered worker. Projects can contain different annotation types. The export format and model recipe determine which types a particular workflow can use. The actions available in your deployment depend on its enabled capabilities, your permissions, and the workspace plan. ## Choose the right place for your data | Item | Purpose | | --- | --- | | Workspace | Membership, plan, storage, and usage scope | | Project | Images, annotations, classes, dataset versions, and project files | | Dataset version | A generated dataset with selected data, processing steps, and splits | | Artifact | A file such as model weights, an export from a run, or a reference file | | Worker | A machine you register to execute compatible jobs | ## Before starting compute A worker must be connected, authorized, and compatible with the selected environment. Resource assessment helps identify likely constraints; it does not reserve resources or guarantee that a job will start immediately. Runtime checks still apply. Check [Workers](/docs/workers) for installation and connectivity, and [Account, billing, and administration](/docs/account-billing-admin) for allowances. ## Find your way around Use **Workspaces** to reach your projects. Inside a project, the image grid is where you browse and open data. Project settings bring together **Upload**, **Export Data**, **Model Training**, and **Files & models**. Account **Settings** contains your workers, artifact library, and workspace billing controls. For a first session, aim to upload a small batch, label a few examples, and download a version. You can add worker-based training after you are comfortable with the dataset. ## Automate a workflow Use the [Python SDK](/sdk-docs) for uploads, annotations, dataset downloads, training, and supported model operations. See [SDK automation](/docs/sdk-automation) for a safe starting workflow. --- # Workspaces and projects A workspace groups projects and provides a shared membership, billing, and storage scope. A project holds the images, annotations, classes, versions, and files for a dataset. ## Create and open a project 1. Open [Workspaces](/workspaces). 2. Select a workspace you can access. 3. Create a project, or open an existing project card. 4. Use the project's settings to upload data, create dataset versions, and inspect files or model runs. The available actions depend on your workspace role, project role, additional permissions, and image assignments. Seeing a project does not necessarily grant permission to change or delete all its data. ## Organize a review queue ![Demo data · Project image grid with generated sample images](/docs-assets/dataset-grid.jpg) Demo data · The real project interface with sample records. Think of the image grid as your team's workbench. Open an image to annotate it, or narrow the grid to a specific question: Which images still need review? Where does a particular class appear? Which batch did a teammate upload? Use the project's available filters to narrow the data by class, tag, split, filename, verification, or assignment. Confirm the current filter before selecting images for an operation or building a version. Keep class names consistent across the project. Rename or remove a class through class management rather than editing individual labels merely to simulate a project-wide rename. ### A practical project structure Create separate projects when the data has a different purpose, ownership, or annotation policy. Within one project, use tags, batches, assignments, and splits to organize related examples instead of creating a new project for every small upload. For example, an inspection project can contain several camera batches and defect classes. A reviewer can filter one batch, check its annotations, and then create a dataset version from the intended selection. ### Expected result After setup, your team should have one agreed project, recognizable class names, and a clear way to find the next images to label. The project can grow without requiring you to finish every annotation before starting a review. ## Work with a team Workspace administrators can manage membership through the workspace controls. Project and image assignments determine which work a collaborator can access. A shared link does not replace membership or permission checks. Use assignments to divide work, then verification filters to review it. When someone else changes an image or its labels, refresh before replacing your local work. Keep a copy of a draft if a conflict requires you to reload. ## Clone or remove a project Cloning is a background workflow. Wait for the target to become ready before using it; a timeout while waiting does not mean you should submit another clone. Project deletion is destructive and may require background cleanup. Protected artifacts can survive project deletion in the account library. Check the files you need before deleting a project, and inspect any cleanup error rather than repeatedly issuing delete requests. Continue with [Upload data](/docs/upload-data) or [Artifacts and inference](/docs/artifacts-inference). --- # Upload data Use project upload for data you want to annotate or include in a dataset. Use the artifact library for files that should remain files, such as weights or reference material. ## Upload images or video 1. Open your project and its **Upload Data** section. 2. Choose files or drop them into the upload area. 3. Review the recognized media and available import options. 4. Start the upload and wait for its reported completion. 5. Open the project data to check image orientation, labels, and metadata. Image and video workflows share the upload entry point. Video processing can produce project frames; it is not the same operation as uploading unrelated still images. Use the controls shown for the recognized media. Do not close the page merely because file transfer has finished if the interface still reports processing. A failed or interrupted upload should be inspected before starting a duplicate batch. ### Check your first batch Open a few examples after upload. Confirm the orientation and dimensions, look for unexpected duplicates, and check that any imported labels sit on the correct objects. A quick check now is easier than discovering an import mismatch after labeling hundreds of images. For a video, review frames across the selected range rather than only its first frame. Choose a frame selection appropriate to the change in the scene; many near-identical frames can make a dataset larger without adding much useful variation. ## Import existing labels The [SDK importer](/sdk-docs) supports common dataset layouts, including YOLO, COCO, Pascal VOC, LabelMe, classification folders, and several CSV formats. Format support is specific: matching a filename extension does not establish that every annotation type in the file is supported. After importing a labeled dataset, inspect several images with different shapes and classes. Check normalized geometry, image dimensions, split names, and class names before training or exporting. ## Upload a file without adding dataset images Open the project's files section or the account artifact library and choose its upload action. Uploading an image as an artifact does not automatically add it to the annotation dataset. TIFF and medical-source actions are separate import workflows when available. Inspect their options and permissions before promoting a file into project data. ## Work with large TIFF and GeoTIFF images Large rasters can be viewed through a tiled image layer, so you can move around a detailed image without treating it as a normal small photograph. The original TIFF remains the scientific source; the displayed view is for navigation and annotation. 1. Upload the TIFF through the supported upload workflow, or upload it in **Files & models** and choose **Import as raster** when offered. 2. Review the image and its display settings in the annotator. 3. Annotate the areas of interest on the source image. 4. When creating training data, choose explicit raster chip settings: chip size, overlap, regions, and display bands or ranges as needed. 5. Preview the chips and then build the version. Training chips are RGB PNG images derived from the selected source regions. The original raster is retained. Annotation currently applies to the first TIFF page; other pages remain in the original file. ## View and annotate supported DICOM frames 1. Upload the DICOM as a project artifact in **Files & models**. 2. Choose **DICOM** to open the viewer. Navigate frames and adjust window center, width, and function to inspect the image. 3. To annotate it, use the available medical import action and select the frame range. 4. Open the resulting project frames in the annotator and label them as you would other project images. The source file is retained once, while selected frames become project annotation records. The medical source controls let you review the source and add frames later. Windowed previews do not modify the stored medical pixels. ### Medical-image scope The current viewer supports a defined subset of monochrome DICOM layouts. Unsupported compressed or color layouts return a message. This is a frame annotation workflow, not a complete clinical study-management system. Removing patient tags from displayed technical metadata does not de-identify the original file or text burned into its pixels. ## Resolve a failure - Check the message for unsupported media, size limits, storage allowance, or permission restrictions. - Keep the original files until the uploaded data has been verified. - For an uncertain network result, inspect the existing upload or target before submitting it again. Next: [Annotations](/docs/annotations). --- # Annotate and review images Annotations describe geometry relative to the image. DataHub supports multiple annotation types within a project; the available editor tools and downstream format determine how each type is used. ## Choose a label type | What you want to describe | Useful annotation | | --- | --- | | Whether an image belongs to a category | Image-level classification | | Where an object is | Bounding box, including rotation where needed | | An object's outline | Polygon or mask | | A landmark or path | Point or line | | Text and its location | OCR region with its transcript | | An articulated object or landmark layout | Pose instance with a skeleton | Use a box when location is enough. Use a polygon or mask when the boundary matters, such as a defect, organ region, or irregular object. Multiple annotation types can coexist; select an export that fits the labels you intend to train with. ## Label an image ![Demo data · Annotation editor showing a box and polygon on a generated sample image](/docs-assets/annotation-editor.jpg?v=2fef398e628c) Demo data · Choose a class in the editor, then use a box for location or a polygon for the object's outline. Check the saved annotation before moving to the next image. 1. Open a project image in the annotator. 2. Select a class and an appropriate tool, such as a rectangle, polygon, point, line, or mask. 3. Draw and adjust the object within the image boundary. 4. Check its class and any text or other properties. 5. Save your changes and confirm the save state before moving on. Use undo and redo to review edits. Avoid relying on an unsaved draft as your only copy when closing a session or resolving a conflict. ### Review a small batch before scaling up Label a few varied examples first: a clear object, a partial object, and a difficult background. Agree on when to draw an object and how much of its boundary to include. Review the examples together, then use the same class names and geometry rules for the rest of the batch. A consistent label policy is more useful than adding many loosely defined classes. Keep a separate held-out set for checking whether a trained model works on examples it did not learn from. ## Draw boxes, outlines, and masks Select a class first so new objects receive a meaningful label. Draw the shape, then use its handles to refine the fit. Zoom in for fine boundaries and zoom back out to check that the overall object still makes sense. For masks and multipart geometry, review separated regions and holes as well as the outer contour. A source annotation can retain this detail even when a later box-only export uses a simpler representation. ## Label text for OCR Mark the text region and enter the transcript that belongs to that region. Keep line or word labels aligned with the crop you expect a recognition model to read. A page-level transcript and a set of region-level transcripts serve different purposes. Recognition exports can produce rectified text crops and transcripts. Detection exports describe text-region geometry. Choose the workflow that matches your model's task. ## Create pose and landmark annotations The **Pose** panel provides reusable ordered skeletons. Built-in templates include Human · COCO 17, Animal · AP-10K 17, hand and face landmarks, and football, basketball, badminton, and tennis court layouts. 1. Select the class for the instance, such as person, animal, or court. 2. Open **Pose** and choose a skeleton template. 3. Choose **Draw instance**, then drag an instance area on the image. 4. Move each landmark handle to its actual position. 5. Select the pose and set each landmark's visibility: **Visible**, **Occluded**, or **Unlabelled**. 6. Save and inspect the complete instance. ### Customize a skeleton Choose **Customize** to name the skeleton, enter one ordered landmark name per line, define connecting edges, and add left/right flip pairs. In this panel, pairs use the displayed numbering—for example, `1-2, 2-3`. Choose **Save new version** to create a reusable project template. Keep the landmark order consistent across examples. Flip pairs help preserve left/right meaning when generating mirrored training data. Sports-court presets are convenient landmark layouts, not automatic camera calibration. ### Expected result A saved pose contains the instance class, ordered landmarks, visibility, and skeleton definition. COCO Keypoints and YOLO Pose exports carry the corresponding training structure, subject to each format's skeleton requirements. ## Manage classes Use class management for project-wide class names and colors. A rename should be treated as a dataset change, especially when comparing older versions or reviewing already downloaded exports. Use the latest visible class mapping when checking labels. An existing generated dataset is a separate object from the current editable project. ## Verify and compare labels Human verification and model agreement are different signals. Verification records review status; agreement records whether a model's suggestion agrees with an annotation. Filter the review queue using the signal relevant to your task. ## Image changes and save conflicts Geometry must match the image revision that you edited. If another operation changes the source image, a revision conflict may require a reload. Do not retry by attaching a newer revision to an older annotation draft. For programmatic edits, use the revision-aware read and save workflow described in the [SDK documentation](/sdk-docs). Keep the annotations and their captured source identity together. ## Check the target format An annotation can be valid in the editor while requiring a projection or being unsupported in a particular export. Inspect the [export summary](/docs/dataset-versions-exports), especially for rotated boxes, multipart shapes, holes, OCR, and keypoints. --- # Dataset versions and exports A dataset version prepares selected project data for a repeatable download or model workflow. An export then represents that version in a chosen dataset format. ## Build a version ![Demo data · Dataset version with sample images and training and validation splits](/docs-assets/dataset-version.jpg) Demo data · Inspect the version's images and split allocation before generating the format your training tool expects. Give versions names that describe their purpose: a reviewed batch, a different image size, or a new augmentation setup. The editable project remains your working collection; versions are prepared inputs for training and download. 1. Open the project's **Export Data** section and create a version. 2. Review the selected data and filters. 3. Choose the split strategy and train, validation, and test allocation. 4. Add preprocessing steps in the order you need them. 5. Add augmentations and review their settings and generation factor. 6. Preview representative images, then submit the version build. 7. Wait for the version's terminal status and inspect warnings or failures. Preprocessing prepares the input; augmentation creates training variation. Changing their order can change both pixels and geometry. ## Understand previews Choose variation that reflects situations your model should handle. Lighting changes may help camera inspection; flipping text may make OCR examples less realistic. Pose datasets need meaningful left/right pairs. Start simple, then add variation and compare on the same held-out data. A composed preview processes the example through the selected step. Editing a step includes its preceding steps, not later steps. Augmentation previews also include the preprocessing prefix. The preview is a seeded example, not a promise that every full-resolution generated image will have identical random pixels. Filtering can remove examples, and expansion steps can produce several examples. Observe any truncation or resource-limit message instead of assuming all outputs are shown. ## Generate an export ### Match a format to your task | Task | Relevant format examples | | --- | --- | | Detection | COCO, Pascal VOC, YOLO detection variants | | Segmentation | COCO RLE, YOLO Segmentation, PNG masks, Segment Anything 2 | | Classification | ImageFolder Classification | | Pose and landmarks | COCO Keypoints, YOLO Pose | | Text recognition | PaddleOCR Recognition crops and transcripts | | Labeling-tool exchange | LabelMe, VIA, and other formats in the menu | Choose the format your trainer or labeling tool expects. The format menu and summary show the exact choices for your data. ### Generate and inspect the files ![Demo data · COCO export summary for a sample dataset](/docs-assets/dataset-export.jpg) Demo data · The completed export card reports source records, image files, exported labels, and skipped kinds. Open its details before downloading or starting training. 1. Choose a completed version. 2. Select a format suitable for your trainer or labeling tool. 3. Review format-specific options, such as image conversion or box conversion. 4. Generate the export and wait for completion. 5. Inspect its summary and download the archive. Different formats preserve different geometry. A detection format may use boxes where the source contains polygons; a format without a particular annotation type may omit it. Review supported, exported, and skipped counts rather than assuming every source annotation is represented. Source records and image files are different counts: some formats can produce multiple crops from one input. Skipped reasons can overlap, so adding those counters does not give a count of unique failed images. Open the archive and check a representative image and label together when validating a new export setup. Classification exports require meaningful image-level classes. Mask exports and keypoint exports have their own class and geometry requirements. For custom metadata, verify the intended consumer understands the exported extension fields. ## Failed, queued, or recovery-required exports Use the existing export's status and controls. A recovery-required export needs status inspection or operator help; creating another generation is not a safe substitute for resolving an uncertain publication. Generation allowances can still apply to submitted attempts. Deleting an export is separate from deleting the source version or project. Keep a downloaded copy when needed, and wait for scoped cleanup to finish before assuming storage is released. Next: [Model training](/docs/model-training) or [SDK downloads](/sdk-docs). --- # Train and monitor models DataHub runs training recipes on your registered workers. Choose a recipe and worker that support your dataset task. ## Start a training run ### Choose a task and model | Goal | Catalog examples | Data to prepare | | --- | --- | --- | | Locate objects | Ultralytics detection, RF-DETR | Detection boxes | | Trace objects | Ultralytics segmentation, prompted SAM 2.1 | Compatible masks or segmentation labels | | Classify an image | Ultralytics classification, OpenCLIP or DINOv3 linear probes | Image-level classes | | Learn visual similarity | DINOv3 retrieval embeddings | At least two classes, with distinct examples in train and validation | | Predict rotated boxes | Ultralytics oriented detection | Oriented-box labels | | Predict landmarks | Ultralytics pose | Consistent skeleton and keypoints | | Locate or read text | PaddleOCR detection or recognition | Text regions or transcripts/crops | | Find visual anomalies | Bounded PatchCore recipe | The recipe's normal/anomaly dataset structure | Start with the output you want, then select a compatible recipe. The model card describes its training scope: a frozen linear probe learns a classifier rather than retraining a whole vision-language encoder. ### Train retrieval embeddings The DINOv3 retrieval recipe fine-tunes selected encoder blocks and produces normalized 384-dimensional image vectors. It uses your image-level class labels to define similar images. This is separate from a classifier or DreamSim's preference-triplet supervision. Use an **ImageFolder Classification** export with at least two classes. Each class needs at least two distinct training images and two validation images. The selected samples per class can raise the training minimum. The default batch has two classes and two images per class. The run reports validation retrieval Recall@1/5/10 and exact full mAP, excluding each query from its own gallery. These are retrieval scores, not classification accuracy. Evaluation uses disk-backed embeddings and an explicit validation-size budget; no validation samples are silently dropped. Choose a worker advertising this recipe, using worker 0.2.14 or later. Download the encoder checkpoints and model metadata from the run. The metadata records preprocessing, source weights, frozen and trainable checksums, and whether data-driven encoder updates occurred. Checkpoint pause/resume is not yet offered for this recipe. #### Export and try the trained embeddings 1. Open **Files & models** and choose **Convert** for the run's `best.pt` or `last.pt`. 2. Select a compatible CPU worker advertising the metric converter, using worker 0.2.15 or later. 3. Keep the default static batch of one, or enable a dynamic batch and choose a nominal batch from one to four. Image dimensions stay fixed at 256×256. 4. Review resource fit and start the conversion. The original checkpoint remains available beside the generated ONNX file and conversion metadata. 5. Open the ONNX graph to inspect its inputs and outputs, or open the model playground and upload a query image. The ONNX model produces normalized 384-dimensional vectors. Playground results show vector values, not classification probabilities. ImageNet normalization is embedded in the graph; use RGB/255 and the fixed stretch preprocessing. Browser and SDK inference each process one image per request, including dynamic-batch graphs. Conversion verifies numerical agreement with the native model; it does not establish accuracy on your application. TensorRT export for this recipe is not yet available. ### Queue the experiment ![Demo data · Model training interface with sample records](/docs-assets/model-training.jpg) Demo data · Choose the recipe, worker, and parameters for the dataset version you prepared. Review compatibility before submitting once. 1. Prepare a completed [dataset version and compatible export](/docs/dataset-versions-exports). 2. Open the training action for that version. 3. Choose an available trainer and its supported model options. 4. Select a worker and the required device, then review parameters. 5. Review the resource assessment and workspace admission limits. 6. Submit once and open the resulting run. The trainer catalog defines supported inputs and parameters for detection, segmentation, classification, pose, OCR, and other available tasks. Use the recipe's stated task and dataset format. ## Read resource assessment Reported free memory, disk, and GPU memory are observations. Estimates help screen a job but are not reservations or guarantees. Missing or stale evidence is unknown, not zero free capacity or proof of fit. A queued run is checked again before allocation. Dataset staging, environment preparation, compute, and output delivery are separate phases. Disk pressure or an unavailable connection can delay delivery after compute has stopped. Inspect the current run's recovery message before resubmitting. ### Resource checks for queued jobs Queued jobs receive an admission resource plan before a worker claims them. DataHub can refresh estimates when the pinned recipe and saved configuration can be reproduced. The original parameters, source references, and configuration checksums remain intact; the run details show the admission estimates separately. If a historical recipe or source cannot be verified, the job stays queued with a review message. Inspect its saved configuration, submit a new run using a reviewed recipe if needed, and cancel the old job when it is no longer wanted. DataHub does not silently replace a historical environment with the newest one. An updated plan may require a current worker. Update an idle worker when prompted, or wait for the selected machine's resources to become available. Active runs and retained-output delivery continue with their existing execution configuration. ## Follow a run ![Demo data · Training run console and metrics](/docs-assets/training-console.jpg) Demo data · Open the console for execution messages and the metrics view for the values the recipe reports. These example values illustrate the interface, not model quality. **Model Training** collects training and related model jobs. Open a row to inspect its console, charts, hardware observations, and output files. Metrics depend on the task. Detection may report precision, recall, and average precision; classification can report accuracy; segmentation and pose have their own measures. Compare validation results as well as training loss, using the same dataset and evaluation setup. Hardware charts describe resource use, not model quality. Use the run details to inspect status, console output, metrics, hardware samples, and retained artifacts. Recent event windows can be truncated; use the available log download or history controls for earlier output. A failed or cancelled terminal status is not a successful model result. ## Pause, resume, and cancel Pause is available only for supported checkpoint-capable jobs. Supported single-GPU Ultralytics training pauses asynchronously at an epoch checkpoint. It becomes paused after checkpoint delivery and resource release. Resume requires the same worker and retained working data. Cancellation is a request, not an immediate guarantee that the container has stopped. Wait for the reported outcome. Do not manually delete retained worker containers or working data to simulate cancellation. ## Compare configurations Reusable YAML or JSON configurations prepare settings for review. Importing a configuration in the browser does not itself submit a job. Sweeps submit several trials and consume pending-run capacity. Compare trials using the same dataset and relevant metric; a sweep is not distributed training of one model. Cancelling a sweep stops future dispatch and requests cancellation of its existing children. ## Evaluate and benchmark a model In **Model Training → New run**, choose **Evaluate model** or **Benchmark inference**. Eligible files also expose **Evaluate** and **Benchmark** in **Files & models**. 1. Select the model artifact and dataset version/export. 2. Review the model contract, worker, and device. 3. For evaluation, review the held-out split and task settings. 4. For benchmarking, choose the timing scope, warmup, and measured iterations. 5. Queue the run and open its report when complete. Evaluation compares predictions with ground truth. Benchmarking measures latency under a stated protocol. End-to-end timing includes work such as preprocessing; model-only timing uses prepared tensors. Compare matching protocols. ## Continue from a trained model Open its checkpoint in **Files & models** to download it, evaluate it, convert it, or try supported inference. See [Artifacts, conversion, and inference](/docs/artifacts-inference). ## Custom Python workflows Custom Python jobs require the worker owner's explicit opt-in, appropriate project access, and a compatible agent. They use selected reviewed environments and bounded resources; they are not a way to install arbitrary dependencies on the worker host. Custom evaluation and benchmark labels describe the script's own protocol and metrics. Record the dataset, preprocessing, and measurement settings when comparing results. Custom jobs support cancellation, but not checkpoint pause. See the [SDK documentation](/sdk-docs) for their separate API. Continue with [Workers](/docs/workers) or [Artifacts and inference](/docs/artifacts-inference). --- # Connect and operate a worker A worker is your own machine running the DataHub agent and Docker. It connects outbound to the DataHub gateway to receive authorized jobs and deliver results. ## Prepare the machine Connect a workstation or server when you want to run training and model operations on hardware you control. The app keeps the experiment and its outputs attached to the project; the worker supplies compatible compute. Several workers can run separate jobs or sweep trials. Connecting them does not automatically distribute a single training run across machines. - Use Python 3.10 or newer and a working Docker installation. - For CUDA jobs, install a compatible NVIDIA driver and NVIDIA Container Toolkit. - Keep enough persistent disk for datasets, environments, checkpoints, and pending outputs. - Allow outbound HTTPS to your DataHub gateway and the network access needed to obtain the selected container environment. Workers can connect from behind a firewall through outbound connections, without PortHub or an inbound public worker port. Configure restrictive firewalls or proxies to permit those connections. ## Install and connect 1. Open [Settings](/settings) and its Workers section. 2. Use the displayed install command for your current DataHub deployment. 3. Follow the credential prompt and registration instructions. Keep the worker credential private. 4. Confirm the worker appears and that its software and hardware report is current. 5. Resume job claiming when you are ready to use the machine. Use a distinct named installation when connecting the same machine to another deployment. Do not reuse credentials across deployments or overwrite an unrelated worker service. ## Select a compatible environment ### Check your connection After installation, the worker should appear in Settings with a recent heartbeat. Open its details to inspect its software and CPU, RAM, disk, and GPU report. An online worker can remain idle until a compatible job is assigned. The worker advertises supported environments and device capabilities. Reviewed recipes can use prebuilt images; an uncached environment may require a download and additional disk. A cached image does not eliminate space needed for writable output. CUDA support depends on the driver, architecture, device, container runtime, and recipe. CPU availability does not imply that a GPU recipe has a CPU fallback. Unsupported hardware or missing capabilities can keep a job ineligible. The reviewed CUDA 13.2 training images require NVIDIA compute capability 7.5 or newer. DataHub checks the selected GPU and the pinned environment recipe before admission. Worker 0.2.12 repeats the architecture check before staging or starting a container; an unknown capability waits for a verified report. Update an idle worker when the resource report requests this capability. Existing running containers are not restarted by the check. ## Read the resource report ![Demo data · Worker resource report for a sample GPU worker](/docs-assets/training-worker.jpg) Demo data · Sample worker records, not a measurement of your machine. CPU and RAM reservations describe admitted container limits, not current utilization. GPU memory and disk readings are observations with their own freshness and filesystem scope. Unknown values should remain unknown. Do not add free-space values from separate filesystems or assume a Docker observation describes the dataset working directory. Resource assessment is advisory; runtime admission and safety checks remain authoritative for starting work. ## Pause or remove a worker ### Daily operation Use the worker's claim-pause control when you want the machine to stop taking new jobs. Keep persistent working data available while a job is paused or delivering results. When updating an installation, select the same intended worker instance rather than registering duplicates. Pausing job claiming prevents new assignments; it does not promise to stop an existing run. Use the run's supported pause or cancellation action for active compute. Before uninstalling or removing persistent working data, resolve active and recovery-required runs and confirm their outputs are retained. Revoking a worker credential prevents future authorized worker access; it is not a substitute for verifying local compute has stopped. ## Troubleshooting | Symptom | Check | | --- | --- | | Worker is offline | Agent/service, outbound gateway connection, and credential | | Online worker does not take a run | Claim pause, sharing, recipe support, device fit, and concurrency | | Environment preparation waits | Image availability and space in the relevant storage domains | | Hardware value is unknown | Whether that filesystem or device can be observed | | Compute finished but delivery waits | The run's current storage/network recovery message | Keep the original run and working data while investigating so recovery can continue without duplicate compute. --- # Artifacts, conversion, and inference Artifacts are retained files such as model weights, converted models, run outputs, and reference files. Project files and the account artifact library have different scopes. ## Find and protect a file ![Demo data · Files and models listing with a sample artifact](/docs-assets/files-models.jpg) Demo data · The real project file interface with a sample record. **Files & models** is the bridge between dataset work and model use. Training checkpoints, conversion outputs, reports, and reference files can be inspected or downloaded without mixing them into your annotation image grid. | Action | Use it for | | --- | --- | | Upload | Keep a model or reference file with the project | | Download | Take the original retained file to another tool | | Protect | Keep an important project artifact across project removal | | Evaluate / Benchmark | Measure an eligible model using a stated dataset or timing protocol | | Convert | Produce a supported ONNX or TensorRT target | | Inference action | Try predictions and inspect a supported model's output | 1. Open a project's files section for project-specific artifacts, or [Settings](/settings) for the account library. 2. Filter or refresh the list and inspect the file's metadata. 3. Download the file you need. 4. Protect important artifacts before removing their source project. Open file lists update after SDK uploads, file changes, and generated outputs. Your search, page, and unsaved description remain in place; a new file outside the current search or page will not appear there. Returning to the tab or reconnecting also refreshes the list, and **Refresh** remains available if a network update is delayed. Protection prevents direct deletion and preserves a project artifact when the project is removed. Protected files still use storage. Unprotecting a file allows deletion; it does not delete it automatically. Account files do not need a project. ### Keep a metric checkpoint's metadata Protecting a completed, tracked DINOv3 metric **best.pt** or **last.pt** also retains its verified **model-metadata.json**. The metadata file shows **Required by protected model** and cannot be deleted or unlocked while a protected checkpoint depends on it. Its own protection setting remains separate; unlocking every dependent checkpoint releases the requirement without deleting the metadata. For a checkpoint protected before this support was added, deletion may ask its owner to refresh protection. Protect the checkpoint again before retrying project or workspace deletion. An explicit unlock can release a pending package after deletion failed, without moving storage billing. An uncertain storage copy still requires inspection before release. Retained files and their metadata remain downloadable in the account library. Model conversion and inference currently require the live source project; retention does not preserve its run screen or create a new execution destination. ### Keep files independently of a workspace The account library belongs to the file's original account owner. Account files and protected project files retained during workspace removal remain available there even if that was the owner's last workspace. The file details show **Account library** when no workspace is assigned for storage billing; DataHub does not choose another workspace on your behalf. You can continue to inspect, download, describe, protect, unprotect, or delete eligible account files without creating a replacement workspace. Protection still prevents deletion until you explicitly unprotect the file. Ordinary unprotected project files remain subject to project deletion policy. Protecting or unprotecting a file changes its protection only. **Move storage billing** is a separate explicit action offered when you have an eligible owned workspace. Uploading a new file also requires a valid active storage workspace; existing retained files remain browsable when none is available. Detached account-library bytes are separate from the deleted workspace's physical storage observation. They are accounted for by the applicable account-owner storage admission policy; this does not introduce a single global allowance pooled across all your workspaces. See [storage observations and plan usage](/docs/account-billing-admin). ## Convert a supported model Choose a worker to see a conversion resource assessment before submitting. It accounts for the checkpoint size, requested batch and image dimensions, dynamic profile maxima, and TensorRT workspace. GPU, host RAM, and disk estimates are planning heuristics; temporary memory pressure can leave an accepted job queued. Unsupported GPU architectures or environment recipes require another worker or a worker update. The SDK's `training.assess_conversion()` performs the same read-only check without creating a run or charging compute. Submission checks the current report again. 1. Choose a completed, tracked source artifact with a supported conversion action. 2. Select a compatible worker and target format. 3. Review input dimensions, precision, batch settings, and any dynamic profile. 4. Submit the conversion and inspect its result and retained output artifacts. Conversion retains the original weights. A filename alone does not establish valid model lineage or conversion support. Supported sources and settings are checked before execution. ONNX is a model exchange format, but an ONNX file still needs a compatible parser and runtime. TensorRT engines are tied to the target GPU and runtime; do not assume an engine built on one machine can run on another. FP16 conversion permits reduced-precision tactics where supported. It does not promise that every operation uses FP16. Conversion parity checks compare outputs under the stated test settings, not dataset accuracy. ## Run inference ### Try a model on an image 1. Open the eligible model's inference action in **Files & models**. 2. Select an image and inspect the task/parser defaults. 3. Choose the execution worker when required. 4. Adjust supported confidence or result limits for the task. 5. Submit once and inspect the prediction overlay and summary. For detection, look at the class and box placement; for segmentation, inspect the boundary; for pose, check landmark positions and skeleton connections. OCR region detection and text recognition answer different questions, so confirm which output the selected model produces. Use the model's available inference action and defaults. Select an input image, review parser and task settings, and choose a compatible worker when required. The capabilities and defaults determine whether that artifact is eligible. Supported ONNX models can use bounded DataHub CPU inference when available and within its allowance. Native checkpoints require an explicitly selected compatible worker. TensorRT inference requires the supported engine workflow and its matching target device and runtime; there is no CPU fallback for an engine. For pose models, coordinates-only keypoints do not acquire invented visibility or confidence values. OCR detection produces text regions; text recognition is a separate model task. ## Reopen a result ### A useful experiment loop Train a model, evaluate it on held-out data, and inspect a few representative predictions. If the results expose missing examples or inconsistent labels, return to the project, improve the dataset, and make another version. Keep the earlier model and reports so the comparison remains understandable. Use retained inference history to view the original input and result. Viewing a retained result does not submit another inference request. Availability depends on the retained run, artifact, and project authorization. See [Model training](/docs/model-training) and the [SDK model workflows](/sdk-docs) for automation. --- # Account, billing, and administration Workspace plans govern storage, features, and operational allowances. Account administration and plan entitlement are separate: being an administrator does not automatically make every workspace unlimited. Plans apply to each workspace. On a deployment licensed to create Starter, Team, or Enterprise workspaces, a new workspace uses that licensed default independently of any existing Free workspace. Ordinary additional Free workspaces require billing activation; they do not inherit another workspace's paid plan. Changing a plan does not remove an existing administrative billing lock. Ordinary Free admission allows one workspace at a time and up to two Free activations in a rolling 30-day window. Deleting a workspace does not undo an activation. Deletion in progress or failed cleanup continues to hold capacity until cleanup finishes. Additional workspaces can still be created in a locked state and activated on a paid plan. Explicit administrator overrides are recorded separately. New workspaces stay locked until admission and billing setup finish. If creation reports that setup is pending, retry with the same handle to resume it. A retry does not allocate another Free activation or reset existing credits. ## Check the current workspace Use this page to answer three different questions: Which features can this workspace use? How much of its allowance has it used? Is anything preventing new work? A plan, a usage observation, and an individual run's status each describe part of that picture. 1. Open [Settings](/settings) and select the intended workspace. 2. Review its plan, usage, and billing status. 3. Check storage and compute information before submitting a large batch. 4. Resolve a billing lock through the available account controls or your administrator. Do not infer available capacity from the plan name alone. Administrators can configure plan values, and different limits can apply to uploads, exports, indexing, pending jobs, and concurrent compute. ## Understand worker compute and platform CPU | Item | What to review | | --- | --- | | Dataset and artifact storage | Stored data, retained files, and the observation's freshness | | Dataset processing | The operation's displayed charge or allowance before submission | | Registered-worker jobs | Active and pending capacity, even when compute costs zero credits | | Hosted CPU inference | Its separate included-time allowance and runtime limits | Compute on registered workers costs zero platform credits under the current worker-compute policy. Storage still counts, and dataset processing or other billable platform operations remain subject to their own policy. DataHub-hosted CPU inference uses a separate included-time allowance. It has admission and runtime limits and is not an unlimited fallback for worker jobs. Enterprise can have uncapped commercial limits, represented as no commercial cap. Hardware fit, permissions, supported recipes, runtime safeguards, and available storage still apply. An uncapped allowance does not create resources. ## Read usage accurately Queued and paused jobs can count toward pending-job limits. Concurrency limits may hold an admitted job in the queue even when its worker is online. Storage observations can lag or be incomplete. Read the observation time and any partial or unavailable status. An unknown archive count is not zero, and a failed refresh does not prove that storage was deleted. Very large workspaces may require additional observation work before a complete breakdown is available. ### Follow a storage scan Storage is measured in the background so checking a large workspace does not hold up its credits and plan information. **Scanning storage** shows the current phase and available counts while the last successful measurement stays visible. These counts show work completed; they are not a percentage or a forecast of the final size. 1. Select the workspace in Settings and review **Observed storage**. 2. Check the last successful observation time, or the start-to-finish interval when shown. 3. Use **Refresh storage** or **Refresh scan** to request another observation. Refreshing while one is active checks that scan rather than starting a duplicate. 4. If a scan fails, keep the failure message separate from the last confirmed values. An explicit refresh can resume its retained progress. The observation covers an interval during which files may change; it is not an atomic snapshot. **Stored files** reports observed object bytes. **Search index (estimate)** is the estimated raw vector payload, not the search server's complete disk footprint. The combined observed total includes that estimate and is separate from plan admission accounting. If no scan has completed, physical storage is unknown rather than zero. Cancelling a scan leaves the last successful measurement intact and does not delete files, change credits, or refund allowances. SDK callers can pass the exact returned `storage_scan.id` to `workspace.cancel_storage_scan(scan_id)`; a normal status read does not restart a failed or cancelled scan. See the [SDK guide](/sdk-docs) for the storage observation methods. Compute duration excludes queue time and may be incomplete when an attempt could not be measured. Keep measured portions distinct from a verified total. ## Administer plans ### Review a plan change 1. Open the platform administration plan controls with an authorized admin account. 2. Select the plan and inspect its current feature and limit values. 3. Change the intended values and save. 4. Reopen the plan to confirm the saved result. Use an uncapped value only for a limit that supports it. Zero and disabled values have a different meaning from unlimited capacity. Workspace membership and hardware safety remain in effect regardless of plan size. Platform administrators can use the admin plan controls to manage supported limits and features. Preserve the distinction between zero, disabled, and uncapped values. Save deliberately and inspect the resulting plan. Resetting a plan is an explicit administrative operation. Do not use plan changes or account-wide cleanup as a workaround for an individual failed job. Review scoped recovery or deletion messages instead. ## Manage credentials Keep API keys and worker credentials private. Use the intended account and workspace scope, and revoke credentials that are no longer needed. Revocation can prevent future operations; already accepted work may still require a safe completion or cancellation path. --- # Automate with the Python SDK The AlgoFly Python SDK automates DataHub workflows while using the same authorization, plan limits, and data validation as the app. You do not need the SDK for everyday browsing and labeling. It becomes useful when the same steps need to run repeatedly: import a dataset, download each approved version, submit a planned experiment, or collect reports for comparison. | Repetitive task | SDK workflow | | --- | --- | | Bring labeled data from another tool | Folder/archive importer with a supported format | | Review or update labels in code | Image records and revision-bound annotation snapshots | | Supply training data to another environment | Version export and download | | Track several experiments | Training configurations, runs, and sweeps | | Collect model outputs | Artifact downloads and retained inference results | ## Install and authenticate Open the [SDK documentation](/sdk-docs) for the installation command and examples appropriate to this deployment. The Python import is: ```python from algofly import AlgoFly ``` Configure the DataHub URL and API key using the documented client options or environment variables. Do not put credentials in shared notebooks, screenshots, committed files, or job parameters. ## Build a small workflow first 1. Verify authentication and list your workspaces. 2. Select the intended project explicitly. 3. Upload a small labeled sample or read a few existing images. 4. Inspect dimensions, classes, splits, and annotations. 5. Create a dataset version, wait for its result, and generate a compatible export. 6. Download and inspect the archive before automating a larger batch. The SDK includes a command-line importer for images, folders, and supported dataset archives. Use an explicit format when automatic recognition cannot distinguish the layout reliably. ## Edit annotations safely When your deployment returns annotation revision metadata, keep it with the annotation snapshot and use the revision-aware save methods described in its SDK guide. Reload changed pixels before reapplying geometry from an older draft. For bulk operations, inspect each item's result. A partial failure does not mean every image was updated, and it should not erase your local failed draft. ## Handle long-running work Creation and execution are separate from waiting. After a timeout, inspect the existing version, export, or run instead of automatically submitting another operation. For operations with idempotency keys, retry with the same key and identical inputs. A terminal failed or cancelled run can be returned by a wait method; check its status before using outputs as a successful model. Recovery-required exports and retained worker warnings need inspection. Do not implement an unconditional regenerate, retrain, or delete loop around them. ## Choose the correct API surface Use project data methods for dataset images and labels, artifacts for files, and training or inference methods for supported compute workflows. An uploaded model file does not by itself authorize arbitrary code execution or establish recipe compatibility. See the [SDK reference and examples](/sdk-docs) for exact method signatures, accepted formats, resource assessment, training configurations, conversion, inference history, and sweeps.