# AlgoFly Python SDK Python client for automating AlgoFly DataHub workflows. Use it to: - list workspaces and projects - upload images with or without labels - upload folders and YOLO datasets - iterate project data - fetch and update labels - mark human verification and model agreement status - mark individual annotations as model-agree, model-disagree, or unset - filter review queues by verification and model agreement - create dataset versions - download exported datasets Projects are type-agnostic. You can keep classification, boxes, polygons, masks, points, lines, OCR text, and custom metadata in the same project. ## Install From this repository: ```bash pip install https://app.algofly.ai/api/install/sdk.tar.gz ``` Import the client: ```python from algofly import AlgoFly ``` ## CLI Upload The SDK installs an `algofly` command. Dataset upload requires API key, workspace id, and project id: ```bash algofly -m upload ./dataset.zip \ --api-key af_... \ --workspace-id WORKSPACE_ID \ --project-id PROJECT_ID ``` Equivalent subcommand form: ```bash algofly upload ./dataset-dir \ --api-key af_... \ --workspace-id WORKSPACE_ID \ --project-id PROJECT_ID \ --format auto \ --split-name auto \ --limit 1000 ``` `./dataset-dir` can be an image, directory, ZIP, TAR, TAR.GZ, or TGZ. The importer auto-discovers common CV layouts and labels, including YOLO, COCO, Pascal VOC, LabelMe, CreateML, TensorFlow CSV, RetinaNet CSV, KITTI, WIDERPerson-style text labels, classification folders/CSV, OCR metadata, and native AlgoFly annotation JSON. ## Authenticate ```python from algofly import AlgoFly af = AlgoFly( api_key="af_...", api_url="https://app.algofly.ai", ) print(af.verify()) ``` You can also use an environment variable: ```bash export ALGOFLY_API_KEY="af_..." export ALGOFLY_API_URL="https://app.algofly.ai" ``` ```python af = AlgoFly() ``` `api_url` overrides `ALGOFLY_API_URL`; otherwise the SDK defaults to `https://app.algofly.ai`. URLs may include or omit `/api`. ## Quickstart ```python from algofly import AlgoFly # Authenticate af = AlgoFly(api_key="af_...", api_url="https://app.algofly.ai") # Pick your workspace and project workspace = af.workspace() # first accessible workspace project = workspace.project("PROJECT_ID") # by id or name # Upload labeled images project.upload( "images/frame-001.jpg", labels=[{"type": "bbox", "label": "player", "left": 0.1, "top": 0.2, "width": 0.3, "height": 0.5}], batch_name="match-day-1", tags=["camera-1", "raw"], ) project.upload("images/frame-002.jpg") # image-only, no labels # Upload a folder (auto-discovers YOLO / COCO / VOC) project.upload_dataset("./dataset", batch_name="import-1", batch_size=10) # Iterate and inspect for image in project.live(page_size=50, include_labels=True): print(image.filename, image.verification_status, image.model_agreement) print([l.label for l in image.labels]) # Review model output image = project.first_image(verification_status="unverified", model_agreement="disagree") if image: image.mark_model_agree() image.mark_verified() # Create a dataset version and export version = project.create_version("v1", split=(80, 10, 10)) version.download("yolo", location="./exports") # exports a ready-to-train zip ``` Use `normalize_image=True` only when you want the SDK to re-encode the source image before upload, for example for MPO files saved with a `.JPG` extension. Install the optional image dependency first: ```bash pip install "algofly[image]" ``` ## Common Workflows ### Training data pipeline Collect images, upload with labels, create a version, and export for training: ```python project = af.workspace().project("PROJECT_ID") # Upload raw images with batch metadata project.upload_dataset("./field-images", batch_name="drone-run-1", tags=["drone"]) # Upload predictions from a model for image in project.live(page_size=100, tags=["drone"], exclude_tags=["labeled"]): predictions = my_model(image.download(as_bytes=True)) image.update_labels(predictions) project.update_images_metadata([image], add_tags=["model-labeled"]) # Freeze a dataset version and export version = project.create_version("drone-v1", split=(80, 10, 10)) path = version.download("yolo", location="./exports") print(f"Exported to {path}") ``` ### Model inference pipeline Fetch images from a version, run inference, upload results: ```python version = project.version("drone-v1") for item, _ in version.iter_labels(split_name="test", page_size=50): image_bytes = item.download(as_bytes=True) predictions = my_model.predict(image_bytes) labels = [ {"type": "bbox", "label": p["class"], **p["bbox"], "confidence": p["score"]} for p in predictions ] project.update_labels(item.id, labels, verification_status="unverified") ``` ### Label review and QC pipeline Iterate images, review labels, mark human verification and model agreement: ```python for image in project.live( page_size=50, verification_status="unverified", model_agreement="disagree", ): labels = image.refresh_labels() reviewed = [l for l in labels if l.get("confidence", 1) > 0.5] image.write_labels( reviewed, verification_status="verified", model_agreement="agree", ) print(f"Reviewed {image.filename}: {len(reviewed)} labels kept") ``` ### Bulk label rename across a project ```python registry = project.class_registry() receipt = project.rename_class( "person", "player", expected_class_revision=registry["class_revision"] ) ``` This changes the project class and its stored labels together. Keep any unsaved drafts; reload their canonical class names before deciding how to reapply edits. ## Workspaces List workspaces: ```python for workspace in af.workspaces(): print(workspace.id, workspace.slug, workspace.name, workspace.storage) ``` Select the first accessible workspace: ```python workspace = af.workspace() ``` Select by id or slug: ```python workspace = af.workspace("WORKSPACE_ID") workspace = af.workspace("my-workspace-slug") ``` Create a workspace: ```python workspace = af.create_workspace( name="Automation Lab", slug="automation-lab", description="Workspace created from Python", ) ``` ### Storage observations ```python usage = workspace.storage_summary() # Last confirmed values and current progress. scan = workspace.storage_summary(refresh=True) # Explicitly start or resume. scan_id = scan["storage_scan"]["id"] # Poll storage_summary() without refresh=True while the scan is active. # Cancel only the captured scan; this never deletes account files: workspace.cancel_storage_scan(scan_id) ``` Physical usage is observed over an interval while files may change. A pending or failed scan retains the previous confirmed measurement; missing values are not zero. Search-index byte counts are estimates. Refresh and cancellation are single-attempt requests: inspect the current status after an uncertain response. ## Projects The current SDK release is **0.3.13**. Install the matching SDK from your DataHub deployment for current project controls and retained export recovery errors. List projects: ```python for project in workspace.projects(): print(project.id, project.name, project.storage) ``` Select by id or name: ```python project = workspace.project("PROJECT_ID") project = workspace.project("Football Dataset") ``` Create a project: ```python project = workspace.create_project( name="SDK Experiments", description="Images and labels uploaded from Python", ) ``` ### Clone and monitor a project Cloning runs in the background and copies project data into a new project. Original video sources and registered frames retain their project-owned provenance; deleting the source project after a successful clone does not remove the clone's data. ```python copy = project.clone("Experiment copy") # submits once; returns a Project print(copy.id, copy.status, copy.clone_status) copy.refresh() # inexpensive operational status; does not load image previews/counts print(copy.data.get("clone_progress"), copy.data.get("clone_message")) copy.wait_for_clone(timeout=900, poll_interval=2) print(copy.data.get("data_available")) # Optional destination workspace; description=None copies the source description. copy = project.clone( "Another experiment", target_workspace_id="DESTINATION_WORKSPACE_ID", description="", wait=True, timeout=900, ) ``` The server checks access to both workspaces, project permissions, and quotas. Status and cancellation recheck current permissions. `refresh()` merges status into the existing metadata snapshot; it does not recalculate storage totals. An older server without lightweight status support returns its HTTP error; the SDK does not fall back to repeatedly listing every project and preview. ```python from algofly import AlgoflyHTTPError, ProjectCloneFailed, ProjectCloneTimeout try: copy = project.clone("Tracked experiment", wait=True, timeout=900) except ProjectCloneTimeout as error: copy = error.project # exact target is retained; timeout never cancels it print(copy.id, copy.clone_status) copy.wait_for_clone(timeout=900) # resume waiting, not another submission except ProjectCloneFailed as error: copy = error.project print(copy.id, copy.clone_status, error.message) except AlgoflyHTTPError as error: # A 503 can mean the target exists but delivery was not acknowledged. detail = error.message if isinstance(error.message, dict) else {} target_id = getattr(error, "project_id", None) or detail.get("project_id") print(error.status_code, detail.get("code"), target_id) # Inspect target_id in the destination workspace before taking another action. # error.response remains available; never blindly resubmit clone(). ``` To stop an inspected target explicitly, select its exact ID in the destination workspace and call `target.cancel_clone()`. This server-idempotent action does not start a replacement project. `timeout` is positive and bounded to 86,400 seconds; `poll_interval` is positive and at most 60 seconds. HTTP errors preserve their original response. A failed or cancelled clone raises `ProjectCloneFailed`; a wait timeout raises `ProjectCloneTimeout` with its inspectable `.project` and `.project_id`. Once a target is known, errors during waiting also retain `.project` and `.project_id`, including an unchanged `AlgoflyHTTPError` or a transport error such as `requests.Timeout`. Inspect that exact target or resume its `wait_for_clone()`; do not submit another clone. If a transport timeout returns after the polling deadline, `ProjectCloneTimeout` chains that original error. The polling deadline is checked between HTTP calls; transport retries may make one call exceed it, so it is not a strict end-to-end wall-clock limit. ## Labels Labels are plain dictionaries. Every label object must include `type` or `label_type` so the SDK can choose the right geometry and validate the required fields. For new labels, you can omit `id`; the SDK creates the correct internal id before sending data to the API. If you pass a custom `id`, the SDK rewrites its internal prefix from `type` while preserving your custom suffix. Coordinates should be normalized from `0.0` to `1.0`. ### Image Classification ```python image_classification = {"type": "classification", "label": "empty-court"} ``` Multiple image labels are supported: ```python multi_label_classification = [ {"type": "classification", "label": "indoor"}, {"type": "classification", "label": "broadcast-camera"}, {"type": "classification", "label": "night-game"}, ] ``` ### Box ```python box = { "type": "bbox", "label": "player", "left": 0.34, "top": 0.12, "width": 0.18, "height": 0.62, "confidence": 0.94, } ``` Equivalent normalized corner coordinates are also accepted. The SDK translates them to `left`/`top`/`width`/`height` before upload: ```python box = { "type": "bbox", "label": "player", "x1": 0.34, "y1": 0.12, "x2": 0.52, "y2": 0.74, } ``` Use one box coordinate style per label. Box values must be normalized between `0.0` and `1.0`, and `left + width` / `top + height` must stay within `1.0`. ### Rotated Box Rotated boxes are stored as rectangle annotations with `angle` in degrees. Use `type="rotated_bbox"` (aliases: `rbbox`, `obb`, `oriented_bbox`) when you want the SDK to make intent explicit. Center form is accepted: ```python rotated_box = { "type": "rotated_bbox", "label": "ship", "cx": 0.50, "cy": 0.44, "width": 0.28, "height": 0.10, "angle": -32, } ``` Four normalized corner points are also accepted. The SDK derives `left`/`top`/`width`/`height`, `cx`/`cy`, and `angle` before upload: ```python rotated_box = { "type": "obb", "label": "ship", "points": [ {"x": 0.36, "y": 0.31}, {"x": 0.61, "y": 0.45}, {"x": 0.56, "y": 0.54}, {"x": 0.31, "y": 0.40}, ], } ``` ### Polygon ```python polygon = { "type": "polygon", "label": "player", "points": [ {"x": 0.30, "y": 0.20}, {"x": 0.50, "y": 0.25}, {"x": 0.48, "y": 0.60}, {"x": 0.28, "y": 0.55}, ], } ``` ### Mask ```python mask = { "type": "mask", "label": "court", "rle": {"counts": "encoded-mask", "size": [720, 1280]}, } ``` ### Point ```python point = { "type": "point", "label": "left-ankle", "x": 0.42, "y": 0.37, } ``` ### Line ```python line = { "label": "baseline", "type": "line", "x1": 0.10, "y1": 0.80, "x2": 0.90, "y2": 0.80, } ``` ### OCR For The Whole Image ```python image_ocr = { "label": "document-text", "type": "image_ocr", "text": "Invoice #1007\nTotal: $42.00", "language": "en", } ``` ### Text In A Box ```python jersey_number = { "type": "bbox", "label": "jersey-number", "text": "23", "left": 0.44, "top": 0.20, "width": 0.08, "height": 0.12, } ``` ### Text In A Polygon ```python license_plate = { "type": "polygon", "label": "license-plate", "text": "MH12AB1234", "points": [ {"x": 0.61, "y": 0.71}, {"x": 0.78, "y": 0.70}, {"x": 0.79, "y": 0.76}, {"x": 0.62, "y": 0.77}, ], } ``` ### Custom Metadata ```python metadata = { "type": "classification", "label": "needs-review", "source": "my-model-v3", "reason": "low-light", "confidence": 0.91, } ``` ### Mixed Labels ```python labels = [ image_classification, image_ocr, box, polygon, jersey_number, metadata, ] ``` ## Upload Images Upload without labels: ```python project.upload( "images/frame-001.jpg", batch_name="camera-run-1", tags=["camera-1", "raw"], split_name="train", ) ``` Upload with labels: ```python project.upload( "images/frame-002.jpg", labels=[ {"type": "bbox", "label": "ball", "left": 0.42, "top": 0.51, "width": 0.06, "height": 0.06}, {"type": "classification", "label": "indoor"}, ], ) ``` Upload bytes or a file-like object without writing a temporary file: ```python success, encoded = cv2.imencode(".jpg", crop) if success: project.upload( encoded.tobytes(), batch_name="pipe crops", tags=image.data["tags"], annotations=[{"type": "classification", "label": str(count)}], ) ``` When uploading bytes without `filename`, the SDK generates a unique filename from the MD5 hash of the image bytes. `upload_bytes` is a convenience wrapper around `upload` for byte payloads: ```python project.upload_bytes( image_bytes, filename="frame-003.jpg", batch_name="pipe crops", tags=["crop"], labels=[{"type": "classification", "label": "12"}], ) ``` Upload a PIL image or an OpenCV image directly: ```python project.upload( pil_image, batch_name="pipe crops", tags=["crop"], annotations=[{"type": "classification", "label": str(count)}], ) project.upload( cv2_image, # NumPy array in OpenCV BGR format batch_name="pipe crops", tags=["crop"], annotations=[{"type": "classification", "label": str(count)}], ) ``` PIL/OpenCV uploads are encoded in memory. If `filename` is omitted, the SDK generates an MD5-based filename after encoding. Upload from an image URL: ```python project.upload( "https://example.com/images/frame-001.jpg", batch_name="url imports", tags=["url"], ) ``` For URL uploads, `filename` is optional. The SDK uses `filename` if provided, otherwise it uses the URL basename when it has an image extension, otherwise it generates an MD5-based filename from the downloaded bytes. Upload OCR data: ```python project.upload( "images/receipt.jpg", labels=[ image_ocr, {"type": "bbox", "label": "total", "text": "$42.00", "left": 0.60, "top": 0.74, "width": 0.22, "height": 0.05}, ], ) ``` ## Upload Folders Upload a folder or common dataset layout. `upload_dataset` auto-discovers labels for layouts such as YOLO, COCO, Pascal VOC, LabelMe, and sidecar JSON/TXT files: LabelMe imports read DataHub metadata from `algofly`, with per-field fallback to legacy `flags`. Multipart polygons, holes, masks, image classifications, and OCR remain associated with their image; ordinary boolean flags are not labels. ```python from datetime import datetime project.upload_dataset( "./images", batch_name="dataset-import", tags=["sdk", "unlabeled"], capture_date=datetime(2026, 5, 29, 14, 30), batch_size=25, show_progress=True, ) ``` `capture_date` accepts a `datetime` or ISO string. Omit it to use upload time. Folder and dataset uploads show a tqdm progress bar by default. Use `show_progress=False` to hide it. Uploads run concurrently using a thread pool. `batch_size` controls how many images are uploaded in parallel: set it higher (e.g. `batch_size=50`) for large datasets on fast connections, or lower (e.g. `batch_size=5`) to stay within API rate limits. Transient failures (429 rate limits, 5xx server errors) are retried automatically with exponential backoff. Upload a folder with labels keyed by filename: ```python labels_by_filename = { "frame-001.jpg": [ {"type": "classification", "label": "empty-court"}, ], "frame-002.jpg": [ {"type": "bbox", "label": "ball", "left": 0.1, "top": 0.2, "width": 0.1, "height": 0.1}, ], } project.upload_dataset( "./images", labels=labels_by_filename, batch_name="folder-import", ) ``` Load labels from sidecar JSON files: ```python import json def load_labels(image_path): sidecar = image_path.with_suffix(".json") if not sidecar.exists(): return None return json.loads(sidecar.read_text()) project.upload_dataset( "./images", label_loader=load_labels, batch_name="sidecar-import", ) ``` ## Upload YOLO Data For folders with `images/`, `labels/`, and `classes.txt` or `data.yaml`: ```python project.upload_yolo_dataset( "./dataset", dataset_format="yolov8", annotation_format="bbox", split_name="auto", tags=["training"], batch_size=25, ) ``` For YOLO segmentation: ```python project.upload_yolo_dataset( "./segmentation-dataset", dataset_format="yolov8-seg", annotation_format="polygon", split_name="auto", ) ``` For YOLO OBB/rotated boxes, use `annotation_format="obb"`: ```python project.upload_yolo_dataset( "./obb-dataset", dataset_format="yolov8", annotation_format="obb", split_name="auto", ) ``` CLI examples for detection and segmentation uploads are available in `sdk/examples`. They use argparse and accept `--api-url`, so they can target different AlgoFly deployments. ## Iterate Project Data ```python for image in project.live(page_size=100): print(image.id, image.filename) ``` If you need labels for every image, include them in each page instead of fetching labels image-by-image: ```python for image in project.live(page_size=100, include_labels=True): print(image.filename, image.labels) ``` SDK records support dict and dot access: ```python image = next(project.live(include_labels=True)) print(image.data.filename) print(image.labels[0]["label"], image.labels[0].label) print(image.labels[0].ocr.text) plain_labels = image.labels_dicts # JSON-safe list[dict] ``` Convert normalized labels to absolute pixel coordinates: ```python from algofly import label_to_absolute, labels_to_absolute for image in project.live(page_size=100, include_labels=True): for label in image.absolute_labels: if label.get("label") == "pipe": crop = cv2_image[label["y1"]:label["y2"], label["x1"]:label["x2"]] absolute_label = label_to_absolute(label, image_width=3840, image_height=2160) absolute_labels = labels_to_absolute(image.labels, image.width, image.height) ``` Bounding boxes include both corner and size forms in normalized and absolute helpers: `left`, `top`, `width`, `height`, `x1`, `y1`, `x2`, `y2`, `cx`, `cy`, `area`, and `aspect`. Rotated boxes preserve `angle`, `cx`/`cy`, and corner `points`. Label records also expose `rotation`, `is_rotated_bbox`, `rotated_points`, `obb`, and `enclosing_bbox`: ```python label = image.labels[0] if label.is_rotated_bbox: print(label.rotation, label.rotated_points) x1, y1, x2, y2 = label.enclosing_bbox ``` Label records also expose convenience geometry as properties: ```python label = image.absolute_labels[0] print(label.x1, label.y1, label.x2, label.y2) print(label.center, label.area, label.aspect) plain_label = label.to_dict() ``` Convert absolute pixel labels back to normalized labels for upload: ```python from algofly import label_to_normalized, labels_to_normalized pixel_labels = [ {"type": "bbox", "label": "pipe", "left": 120, "top": 80, "width": 420, "height": 360} ] project.upload( cv2_image, annotations=pixel_labels, normalize_labels_from_pixels=True, ) normalized = labels_to_normalized(pixel_labels, image_width=3840, image_height=2160) ``` With filters: ```python for image in project.live( page_size=50, split_name="train", classes=["ball"], exclude_classes=["ignore"], tags=["camera-1", "model-labeled"], exclude_tags=["rejected"], batch_names=["run-1", "run-2"], exclude_batch_names=["old-run"], ocr_values=["Invoice #1007"], exclude_roi_ocr_values=["REJECTED"], verification_status="unverified", model_agreement="disagree", capture_date_start="2026-05-01T00:00:00", capture_date_end="2026-05-31T23:59:59", filename_query="frame", sort_by="label_count_desc", ): print(image.filename, image.labels) ``` By default, `live()` and `list_images()` fetch metadata only. `live()` still fetches metadata in pages, then yields images one by one. Labels and image bytes are loaded lazily when accessed: ```python for image in project.live(page_size=100): print(image.filename) labels = image.labels # fetched on first access image_bytes = image.image # downloaded on first access ``` You can eager-load either when you need it. With `live()`, eager data is loaded for each page-sized batch before records from that batch are yielded: ```python for image in project.live(page_size=100, load_labels=True, load_image=True): print(image.filename, image.labels, len(image.image_bytes)) page = project.list_images(page_size=500, load_labels=True) ``` The same loading behavior applies when iterating page-sized batches: ```python for batch in project.live_batches(page_size=500): for image in batch: print(image.filename) for batch in project.live_batches(page_size=500, load_labels=True): print([image.labels for image in batch]) ``` Use `count()` for progress bars without loading labels or image bytes: ```python images = project.live( page_size=100, tags=["reviewed"], exclude_tags=["rejected"], ) for image in tqdm(images, total=images.count()): print(image.filename) ``` `tqdm` can also infer the total automatically: ```python for image in tqdm(project.live(page_size=100, tags=["reviewed"])): print(image.filename) ``` Common automation helpers: ```python total = project.count_images(tags=["reviewed"]) image = project.first_image(filename_query="frame-004") page = project.find_images(page_size=500, batch_names=["run-1"]) image.add_labels([{"type": "classification", "label": "reviewed"}]) image.set_labels([{"type": "classification", "label": "approved"}]) image.mark_verified() image.mark_model_agree() # Flag only incorrect model annotations for review. The image can be marked # unverified separately; annotation agreement does not imply verification. incorrect = [label for label in image.labels if label.get("ocr", {}).get("text") == "2"] image.mark_annotations_model_disagree(incorrect) image.mark_unverified() # Alternatively, flag only the OCR transcription while leaving box geometry unset. image.clear_annotations_model_agreement(incorrect) image.mark_annotation_ocr_model_disagree(incorrect) # After correction, either record agreement or remove the annotation status. image.mark_annotations_model_agree(incorrect) image.clear_annotations_model_agreement(incorrect) image.clear_annotation_ocr_model_agreement(incorrect) image.reload() image.delete() ``` Bulk metadata updates and deletes can be previewed before applying: ```python preview = project.update_matching(tags=["raw"], add_tags=["reviewed"]) # dry-run emits a warning and returns matched_count, would_update_count, and next_step result = project.update_matching( tags=["raw"], add_tags=["reviewed"], model_agreement="agree", dry_run=False, ) preview = project.delete_matching(tags=["bad"]) # dry-run emits a warning and returns matched_count, would_delete_count, and next_step result = project.delete_matching(tags=["bad"], dry_run=False) ``` Structured filters support multiple tags/classes and validate before the first page request: ```python project.live(tags=["cvat", "reviewed"]) project.live(tags=["cvat"], exclude_tags=["rejected"]) project.live(batch_names=["run-1", "run-2"]) project.live(batch_names=["run-1"], exclude_batch_names=["old-run"]) project.live(classes=["pipe", "ring"]) project.live(exclude_classes=["ignore"]) project.live(ocr_values=["MH12AB1234"]) project.live(image_ocr_values=["Invoice #1007"]) project.live(roi_ocr_values=["23"], exclude_roi_ocr_values=["0"]) project.live(ocr_values=["invoice"], ocr_match="contains") project.live(ocr_values=["MH12AB1234"], ocr_match="fuzzy", ocr_fuzzy_threshold=0.8) project.live(no_annotations=True) project.live(has_image_classification=True) project.live(has_image_ocr=True) project.live(has_roi_ocr=True) project.live(verification_status="verified") project.live(verified=False) # alias for verification_status="unverified" project.live(model_agreement="agree") project.live(model_agreement="disagree") project.live(model_agreement="none") project.live(model_agree=True) # alias for model_agreement="agree" project.live(model_agree=False) # alias for model_agreement="disagree" project.live(resolution={"field": "width", "operator": ">", "value": 1920}) project.live( filters={ "tags": ["cvat", "reviewed"], "exclude_tags": ["rejected"], "batch_names": ["run-1"], "classes": ["pipe"], "exclude_classes": ["ignore"], "verification_status": "unverified", "model_agreement": "disagree", "resolution": [ {"field": "width", "operator": ">", "value": 1920}, {"field": "aspect", "operator": ">", "value": 1.3}, ], } ) ``` Unsupported filters fail immediately: ```python project.live(filters={"made_up_filter": "x"}) # ValueError: Unsupported image filter(s): made_up_filter ... ``` OCR value filters support `ocr_match="exact"` by default, `ocr_match="contains"` for partial text, and `ocr_match="fuzzy"` for approximate text matching. Exact matching trims surrounding whitespace. Contains and fuzzy matching are case-insensitive. Use `ocr_values` for either image-level or ROI OCR, or the scoped `image_ocr_values` / `roi_ocr_values` filters when you need a specific source. Each has a matching `exclude_*` form. OCR include groups are ANDed together, and exclude groups reject matching images: ```python project.live( image_ocr_values=["Invoice #1007"], # must exist at image level roi_ocr_values=["$42.00"], # must exist inside some ROI exclude_roi_ocr_values=["VOID"], # must not exist inside any ROI ) project.live( ocr_values=["MH12"], # image-level or ROI OCR may match exclude_image_ocr_values=["draft"], # but image-level OCR cannot be draft ocr_match="contains", ) ``` Fetch a single page. By default, `data` contains `ImageRecord` objects; pass `as_dict=True` for raw backend rows: ```python page = project.list_images(page_number=0, page_size=500, tags=["reviewed"]) for image in page["data"]: print(image.filename, image.batch_name, image.tags) raw_page = project.list_images(page_number=0, page_size=500, as_dict=True) ``` Iterate image and label pairs: ```python for image, labels in project.iter_labels(page_size=100): print(image.filename, labels) ``` Download images: ```python image = next(project.live()) image.download("./downloads") image.download("./thumbnails", thumbnail=True) image_bytes = image.download(as_bytes=True) pil_image = image.download(as_pil=True) cv2_image = image.download(as_cv2=True) data_uri = image.download(as_base64=True) # data:image/...;base64,... ``` Only one output mode can be enabled at once. Delete images: ```python project.delete_image(image) project.delete_image(image.id) project.delete_images([image_1.id, image_2.id]) ``` ## Update Labels `update_labels` merges labels by `id`. Existing labels with matching ids are replaced; labels with other ids stay untouched. Use `write_labels` or `overwrite=True` / `override=True` when you want to replace all labels. ```python image = next(project.live(filename_query="frame-004", page_size=1)) labels = image.labels.as_dict() for label in labels: if label.get("label") == "ball": label["label"] = "basketball" label["confidence"] = 0.97 image.update_labels(labels) image.write_labels(labels) # replaces all labels ``` SDK label writes preserve the existing verification status by default. Pass `verification_status` only when the write should explicitly mark images verified or unverified. SDK label writes also preserve model agreement status by default. Pass `model_agreement` or the boolean alias `model_agree` only when the write should explicitly change model agreement. Use `model_agreement="none"` to clear agree/disagree and return the image to the unset state: ```python project.upload("images/frame-001.jpg", verification_status="verified") image.update_labels(labels, verification_status="verified") project.update_labels("IMAGE_ID", labels, verification_status="unverified") project.update_image_verification(["IMAGE_ID"], "verified") project.mark_images_unverified(["IMAGE_ID_1", "IMAGE_ID_2"]) image.update_labels(labels, model_agreement="agree") image.write_labels(labels, model_agree=False) project.update_labels("IMAGE_ID", labels, model_agreement="disagree") project.update_image_model_agreement(["IMAGE_ID"], "agree") project.update_image_model_agreement(["IMAGE_ID"], "none") project.mark_images_model_agree(["IMAGE_ID_1", "IMAGE_ID_2"]) project.mark_images_model_disagree(["IMAGE_ID_1", "IMAGE_ID_2"]) project.clear_images_model_agreement(["IMAGE_ID_1", "IMAGE_ID_2"]) ``` Image records expose review status helpers: ```python print(image.verification_status) # verified or unverified print(image.is_verified) # bool print(image.model_agreement) # agree, disagree, or None print(image.model_agree) # True, False, or None image.set_verification_status("verified") image.mark_unverified() image.set_model_agreement(False) image.mark_model_agree() image.mark_model_disagree() image.clear_model_agreement() ``` Append or replace matching ids without fetching and resending existing labels: ```python image.update_labels([new_label]) ``` Bulk update labels: ```python project.update_labels_bulk( [ {"image_id": image_id_1, "labels": labels_1}, {"image_id": image_id_2, "labels": labels_2}, ], overwrite=True, ) ``` Bulk append labels: ```python project.update_labels_bulk( [ {"image_id": image_id_1, "labels": new_labels_1}, {"image_id": image_id_2, "labels": new_labels_2}, ], overwrite=False, ) ``` Add a new label without removing existing labels: ```python image = next(project.live(filename_query="frame-004", page_size=1)) labels = image.refresh_labels() labels.append({ "type": "classification", "label": "reviewed", "source": "human-qc", }) image.update_labels(labels) ``` Replace all labels: ```python project.update_labels( "IMAGE_ID", [ {"type": "classification", "label": "verified"}, {"type": "bbox", "label": "ball", "left": 0.1, "top": 0.2, "width": 0.2, "height": 0.2}, ], ) ``` Clear all labels: ```python project.update_labels("IMAGE_ID", []) ``` ## Classes ```python for class_item in project.classes(): print(class_item) ``` Read the full registry when you need colors and a revision for an edit: ```python registry = project.class_registry() print(registry["classes"], registry["class_colors"]) receipt = project.rename_class( "old label", "new label", expected_class_revision=registry["class_revision"] ) print(receipt["previous_class_revision"], receipt["class_revision"]) ``` Other class controls use the same explicit revision check: ```python registry = project.class_registry() receipt = project.update_classes( ["cat", {"label": "dog", "color": "#336699"}], expected_class_revision=registry["class_revision"], ) receipt = project.set_class_color( "dog", "#224466", expected_class_revision=receipt["class_revision"] ) receipt = project.remove_classes( ["cat"], expected_class_revision=receipt["class_revision"] ) ``` Class updates merge the supplied labels; `remove_classes` explicitly removes labels. Mutations return the server receipt and make one transport attempt. On HTTP 409 `class_revision_changed`, keep the draft and review a fresh registry before deciding what to change. An uncertain connection or server response must also be inspected before another mutation. These methods never fetch a newer revision automatically or advance the revision of existing annotation snapshots. Omitting `expected_class_revision` retains the server's unversioned compatibility behavior. `classes()` continues to return only the class list. ## Dataset Versions Version sources include images with annotations and images explicitly saved with an empty annotation set. Untouched uploads and annotation reads do not mark an image as a negative. Check the counts before creating a version: ```python readiness = project.version_source_readiness() selection_readiness = project.version_source_readiness(image_ids=[first_id, second_id]) # positive_image_count, explicit_negative_image_count, ready_image_count, # unreviewed_image_count, total_image_count project.update_annotations(image_id, [], overwrite=True) # explicit negative ``` An explicit negative has no objects; it is not an invented classification label. Supervised versions still require at least one positive source; inspect `can_create_version` and `requires_positive_image` in the readiness response. Version metadata records selected positive and explicit-negative source counts, separately from post-augmentation output image counts. Version image records expose `source_negative` and `source_annotation_count` from before class filtering. An empty exported annotation array alone does not establish that its original source was an explicit negative. Create a version: ```python version = project.create_version( name="baseline-v1", notes="Created from SDK", split=(70, 20, 10), ) ``` List versions: ```python for version in project.versions(): print(version.id, version.number, version.status) ``` Get a version by id or number: ```python version = project.version("VERSION_ID") version = project.version(1) ``` Refresh status: ```python version.refresh() print(version.status) ``` ## Iterate Version Data ```python for item in version.live(split_name="all", page_size=100): print(item.id, item.split_name, item.filename, item.labels) ``` ```python for item, labels in version.iter_labels(split_name="val", page_size=100): print(item.filename, len(labels)) ``` Download version images: ```python version_image = next(version.live(split_name="train")) version_image.download("./version-images") version_image.download("./version-thumbnails", thumbnail=True) version_image_bytes = version_image.download(as_bytes=True) ``` ## Download Exports If publication is uncertain, `download()` stops with the retained export's key and recovery details instead of continuing to poll or downloading incomplete data. `generate_export()` returns the server's status unchanged. `DatasetExportBlocked` stops waiting immediately for credit, permission, allowance, or uncertain-execution blocks. Its `export`, `key`, `code`, and `action` describe the retained attempt. Ordinary queued exports and scheduled dispatch retries keep polling. Resolve the reported cause before explicitly retrying; the SDK never resubmits automatically. Publication recovery takes precedence over these blocks. ```python from algofly import DatasetExportRecoveryRequired try: path = version.download("COCO", location="./exports") except DatasetExportRecoveryRequired as error: print(error.key, error.code, error.action) version.refresh() # Inspect the retained label_formats status. ``` Refresh status or contact the administrator to reconcile the retained attempt. The SDK does not automatically resubmit, cancel, delete, or refund it. To explicitly verify an existing retained publication, use its export key: ```python status = version.reconcile_export(error.key, wait=True, timeout=300) print(status["status"], status.get("verified_objects"), status.get("total_objects")) ``` This sends one verification request, then GET-only polls while prepared or verifying. It returns `committed` or `blocked` status unchanged. It never creates an export, charges credits, or consumes another generation allowance. A timeout leaves the verification request retained; inspect status rather than regenerating. To explicitly remove one export, use its existing key: ```python updated_version = version.delete_export(export_key) ``` The server fences active producers and requires sealed proof before deleting a retained publication. An unconfirmed producer may still refuse deletion. Completed processing charges and generation usage remain. This method sends one DELETE; it does not generate another export or retry automatically. ```python path = version.download("yolo", location="./exports") print(path) ``` Other export names depend on the backend export registry: ```python version.download("COCO", location="./exports") version.download("COCO RLE", location="./exports") version.download("LabelMe", location="./exports") version.download("florence2-od", location="./exports") version.download("CSV Classification Manifest", location="./exports") version.download("ImageFolder Classification", location="./exports") version.download("Multi-label Classification CSV", location="./exports") version.download("Multiclass Classification CSV", location="./exports") version.download("OCR Images + TXT Labels", location="./exports") version.download("Segment Anything 2", location="./exports") version.download("OpenImages CSV", location="./exports") version.download("VGG Image Annotator JSON", location="./exports") version.download("VoTT JSON", location="./exports") ``` ## Automation Examples Attach model predictions: ```python for image in project.live(page_size=100, split_name="test"): local_path = image.download("./tmp") predictions = my_model_predict(local_path) labels = [] for prediction in predictions: labels.append({ "type": "bbox", "label": prediction["class"], "left": prediction["left"], "top": prediction["top"], "width": prediction["width"], "height": prediction["height"], "confidence": prediction["confidence"], "source": "my-model", }) image.update_labels(labels) ``` Add review metadata: ```python for image in project.live(page_size=100): labels = image.refresh_labels() labels.append({ "type": "classification", "label": "needs-review", "reason": "automation-check", }) image.update_labels(labels) ``` ## Project Artifacts and Model Graphs Upload arbitrary reference files or model weights to a project. Downloads preserve the original bytes. Uploading a model does not load or execute it. ```python artifact = project.artifacts.upload("best.pt", description="Training weights") print(project.artifacts.list(search="best", limit=50)) print(project.artifacts.get(artifact["_id"])) project.artifacts.download(artifact["_id"], "downloads/best.pt") # Explicit permanent deletion: project.artifacts.delete(artifact["_id"]) ``` For an ONNX artifact, `project.artifacts.inspect(artifact_id)` returns graph nodes, tensor shapes (including symbolic dimensions), inputs, outputs and metadata. Inspection is bounded to 128 MiB models and does not execute inference. Files larger than 8 MiB use resumable, checksum-verified upload parts, including through public gateways with request-size limits. Supply the same explicit key to recover an upload across process restarts: ```python artifact = project.artifacts.upload( "large-model.pt", idempotency_key="experiment-42-final-weights" ) ``` The SDK reads bounded chunks from disk. It returns only after the server verifies the whole file. A deleted artifact's old key returns `410`; use a new key to upload it again. Account files use `client.artifacts.upload(..., billing_workspace_id=...)` and need no project. `client.artifacts.protect(artifact_id)` locks deletion and preserves a project file when its project is removed; `unprotect()` unlocks it. For a completed tracked DINOv3 metric `best.pt` or `last.pt`, protection also retains the verified `model-metadata.json`. A required companion reports `required_by_protected_model=True`; deletion and explicit unlock are blocked until every dependent checkpoint is unprotected. This does not change the companion's independent `protected` setting or charge its bytes twice. If deletion returns `model_protection_refresh_required`, the original owner should call `client.artifacts.protect(checkpoint_id)` again before retrying. An explicit `unprotect(checkpoint_id)` can release a quiescent pending package after deletion failed, without a billing move. Uncertain COPY receipts still require inspection. Retained downloads and metadata do not enable inference or conversion after the source project is deleted; those operations currently require that live project. Retained account files remain listable and downloadable after their last workspace is deleted. Their `workspace_id` and `billing_workspace_id` may be `None`; new uploads still require a valid chosen billing workspace. Protection alone does not move storage billing. Use `list()["account_storage_usage"]` for detached account-library bytes, including pending reservations. ```python files = af.artifacts.list(search="best") status = af.artifacts.retention_status("ARTIFACT_ID") ``` An `artifact_retention_recovery_required` error means a file transfer needs inspection. Retention status exposes receipt states without private object keys; it never retries an uncertain storage copy automatically. ## Workers and Training Runs Workers connect outbound to your DataHub URL over HTTPS. They need Docker and, for CUDA recipes, a compatible NVIDIA driver and NVIDIA Container Toolkit. Register a credential once and keep it on that worker; do not embed it in notebooks. New workers expose a shared allocation snapshot in `worker["capabilities"]["host_resource_admission"]`. CPU and RAM values describe reserved container limits within one Docker daemon, rather than current CPU use or resident memory. Check `status`, `stale` and `observed_at` before using its `budget_cpus`, `reserved_cpus`, `budget_memory_mb` and `reserved_memory_mb` values. An absent or null value is unknown, including on older workers. Separate Docker daemons are separate resource pools, even when they share a physical machine. ### Worker compute billing Compute on registered workers costs **0 platform credits**. Dataset and artifact storage still counts toward the workspace plan. Read the current workspace run limits before submitting a batch: ```python policy = client.training.billing(project.id) print(policy["backend"], policy["compute_charge_credits"]) print(policy["limits"]) # max_active_byo_runs / max_queued_byo_runs print(policy["usage"]) # active_byo_runs / queued_byo_runs (pending jobs) ``` Concurrency limits keep admitted jobs queued until a slot is available. `max_queued_byo_runs` and `queued_byo_runs` count preparing, queued and paused jobs, including planned sweep jobs. A full pending-job allowance rejects new submissions before compute starts. Enterprise has null commercial run limits; worker availability, GPU memory and other hardware safeguards still apply. Each new run records its own `billing` policy snapshot. Historical runs without a snapshot have no recorded billing policy; do not infer charges from the current plan or from the run's wall-clock duration. ```python usage = client.training.compute_usage(job_id) if usage["duration_status"] == "measured": print(usage["total_compute_seconds"]) else: print(usage["duration_status"], usage.get("known_compute_seconds")) ``` Compute receipts measure each worker container attempt and exclude queue time, dataset staging and artifact delivery. `total_compute_seconds` is null when any attempt is unknown; `known_compute_seconds` is only the measured portion. Recovery and delivery retries do not create another charge. The policy endpoint reports workspace admission counts but limits its measured usage aggregate to the selected project; read `compute_usage_scope` and `admission_usage_scope` explicitly. `unrecorded_observations=true` means some compute telemetry could not be recorded. Treat the total as incomplete and retain any known measured portion. The run may also include `compute_usage_warning` with code `invalid_compute_usage`; inspect it through `client.training.get(job_id)`. This warning does not invent a duration or charge and does not discard the run's other outputs. When original compute identity or resource admission cannot be verified, a running run can expose a recovery warning through both `training.get()` and `training.list()`: ```python run = client.training.get(job_id) progress = run.get("progress") or {} if progress.get("code") == "worker_compute_identity_unverified": print(progress["reason"]) # Inspect the worker; local state is retained. ``` This warning does not mean compute has stopped. Do not automatically resubmit the job or delete retained containers. Use the current run response for this hint; events can arrive out of order. The worker clears it after verifying the original compute and resource admission. A terminal run has no active recovery hint. ### Recent run events Use a bounded snapshot to inspect a large run without paging through its entire history first: ```python snapshot = client.training.event_snapshot(job_id) windows = snapshot["windows"] updated = client.training.event_snapshot(job_id, if_snapshot=snapshot["snapshot_id"]) if not updated["unchanged"]: windows = updated["windows"] # Replace all windows; do not append. ``` `truncated_by_kind` and `oversized_omitted_by_kind` disclose omitted history for each window. These are recent samples ordered by retained event ID, rather than a complete training curve or delivery-order stream. Use `training.events()` with pagination or log search for earlier events; full logs remain downloadable from the run page. Recovery warnings come from `training.get(job_id).get("progress")`. ### DataHub CPU fallback allowance CPU inference on DataHub itself uses a separate included-time allowance. It does not consume the zero-credit registered-worker quota and has no credit overage. Read the current policy instead of assuming a fixed subscription limit: ```python cpu = client.training.billing(project.id)["platform_cpu_compute"] print(cpu["allowance_seconds"], cpu["remaining_seconds"]) print(cpu["known_used_seconds"], cpu["reserved_seconds"]) print(cpu["unmeasured_committed_seconds"]) print(cpu["period_start"], cpu["period_end"]) ``` The default period allowance is 300 seconds for Free, 1,800 for Starter and 7,200 for Team. Enterprise has no commercial allowance cap (`null`), while memory, execution deadlines and operational queue limits remain enforced. Admin-configured limits may differ. DataHub reserves a bounded execution budget before dispatch. Measured execution consumes actual time and releases the unused reservation. An interrupted attempt without a trustworthy duration conservatively consumes its reservation, reported as `unmeasured_committed_seconds`; never present it as measured compute time. Queue wait, input staging and artifact delivery do not count as measured execution. If the allowance cannot cover a new reservation, choose a registered worker, wait for the next period or upgrade the workspace. The rejection does not deduct credits or silently switch the selected execution backend. For a smaller CPU request, set an explicit execution deadline without changing the model's input/output options: ```python job = client.inference.start( project_id=project.id, source_artifact_id=model_artifact_id, image="example.jpg", cpu_time_limit_seconds=5, ) ``` The advertised range is 1–60 seconds, with 60 seconds as the omitted default. The reservation adds a 6-second guard (the example reserves 11 seconds). A short deadline may fail if the model cannot load and execute in time; it never silently increases the limit. This field cannot be used with `worker_id` and is sent as a separate multipart field, not inside model `options`. When admission exceeds the remaining allowance, HTTP 402 reports `platform_cpu_allowance_exhausted`, `required_seconds`, and action `wait_choose_worker_or_upgrade`. Choose an explicitly registered worker, wait for the next period, or change the workspace plan; automatic retries cannot create more allowance. A released initial admission returns HTTP 409 `platform_cpu_admission_expired`; a deliberate resubmission needs a new `idempotency_key`. ### Register a worker ```python registration = client.workers.create("Luna", allowed_project_ids=[project.id]) worker_id = registration["worker"]["_id"] # registration["token"] is returned once. Store it securely on the worker. workers = client.workers.list(project_id=project.id) catalog = client.training.trainers() assessment = client.training.assess( "ultralytics-detect", worker_id, project_id=project.id, version_id=version_id, export_key=export_key, parameters={"family": "yolo11", "size": "n", "batch": 1, "imgsz": 640}, ) ``` Supply both selection fields to assess an existing completed export. Its `disk_assessment.dataset_size_bytes` is known when `dataset_estimate_kind` is `sealed_manifest`; legacy exports may report `unknown` with a null size. Disk reports and estimates are advisory, and the worker rechecks space before writing. Work-directory and Docker filesystem checks remain separate. The gateway also serves `/api/install/worker-install.py`. Its installer creates an isolated Python environment and a user systemd service with automatic restart. Use a persistent disk for `--prefix`; active datasets and checkpoints do not use RAM-backed storage. Reviewed environment recipes prefer available prebuilt images and can build a verified context on first use after checking disk capacity. Use catalog IDs, parameter schemas and compatible export formats rather than guessing model or environment names. Each trainer includes its official GitHub URL. Memory estimates are heuristics; a model copy must fit each selected GPU, and multiple GPUs do not combine their memory for data parallel training. ```python job = client.training.start( project_id=project.id, version_id=version_id, export_key=export_key, trainer_id="ultralytics-detect", worker_id=worker_id, parameters={"family": "yolo11", "size": "n", "epochs": 10, "batch": 1}, gpu_ids=[], name="detector-baseline", priority=0, idempotency_key="my-experiment-unique-submission-id", ) print(client.training.list(project.id, search="baseline")) page = client.training.events(job["_id"], kind="log", limit=200) next_page = client.training.events(job["_id"], after=page["cursor"]) result = client.training.wait(job["_id"], timeout=7200) outputs = project.artifacts.list(run_id=job["_id"]) ``` Pass `execution_device="cpu"` to `start()` or `version.train()` for a recipe that advertises CPU support. This reserves no GPU and explicitly hides GPUs from its container. CPU training can be substantially slower; host RAM and disk checks still apply. The default uses the recipe's preferred device. ### Available training recipes Query the live catalog for current parameter schemas and capabilities. The initial additional recipes have deliberately specific supervision contracts: | Recipe ID | Training | Dataset export | | --- | --- | --- | | `ultralytics-detect`, `-segment`, `-classify`, `-obb`, `-pose` | YOLO task-specific training; YOLO11 remains available | Catalog-declared YOLO, ImageFolder or pose format | | `rfdetr-detect` | RF-DETR detection from scratch | COCO | | `sam2-image` | Pretrained SAM 2.1 Tiny mask-decoder fine-tuning; image encoder frozen | Segment Anything 2 | | `openclip-linear-probe` | Pretrained SigLIP2 image encoder frozen; classification head trained | ImageFolder Classification | | `dinov3-linear-probe` | Pretrained DINOv3 ViT-S/16 encoder frozen; classification head trained | ImageFolder Classification | | `dinov3-metric` | Fine-tune selected DINOv3 encoder blocks for normalized retrieval embeddings | ImageFolder Classification | | `anomalib-patchcore` | Fit a bounded normal-feature memory bank; no gradient optimization | ImageFolder Classification with an explicit normal class | | `paddleocr-recognition` | Pretrained PP-OCRv6 Tiny recognizer with CTC supervision | PaddleOCR Recognition | | `paddleocr-detection` | Pretrained PP-OCRv6 Small text detector probability-head/backbone fine-tuning | PaddleOCR Detection | SAM evaluation uses ground-truth box prompts; it is not automatic segmentation AP. The OpenCLIP recipe does not train a text tower or contrastive encoder. The OCR recognition recipe handles crops; detection is a separate recipe, and document layout is not included. Native checkpoints, metrics and package/source provenance are retained as artifacts. Reuse the same idempotency key and identical inputs after a network error. `scheduled_at` accepts an ISO timestamp with a timezone. A busy or offline worker queues jobs; resource checks run again before allocation. `priority` ranges from 0 to 100. `training.update(run_id, priority=50)` changes a queued job's priority. `training.cancel(run_id)` requests cancellation. Single-GPU Ultralytics training can use `training.pause(run_id)` and `training.resume(run_id)` when the worker advertises checkpoint support. Pause is asynchronous at an epoch checkpoint; the status becomes `paused` after the checkpoint is uploaded and resources are released. Resume requires the same worker and retained working data. Run status, retained events, metrics and artifacts remain available after worker disconnection. Inspect `status` and `error` on the returned run; `wait()` returns terminal failed and cancelled runs as well as completed ones. ### Retrieval embedding training `dinov3-metric` trains image embeddings with class-based triplet supervision. Prepare at least two classes, with at least two distinct images per class in both train and validation. A batch contains `samples_per_class` distinct images from each chosen class; `batch` must be divisible by that number and include at least two classes. Increasing `samples_per_class` raises the training-data minimum. Class labels define which images should be similar. ```python job = version.train( "dinov3-metric", worker_id, export_key=imagefolder_export_key, parameters={"batch": 4, "samples_per_class": 2, "last_n_blocks": 1}, idempotency_key="retrieval-experiment-001", ) result = client.training.wait(job["_id"], timeout=7200) outputs = project.artifacts.list(run_id=job["_id"]) ``` The output is an encoder producing L2-normalized 384-dimensional CLS vectors. It is separate from frozen classifier probes and DreamSim preference training. Validation uses every validation image as a query, excludes itself from the gallery, and reports Recall@1/5/10 hit rates plus exact full mAP. Validation embeddings are stored on disk; distance ranking uses linear memory, while computation remains quadratic. The explicit `max_eval_samples` budget defaults to 5,000; larger versions require raising it rather than silently omitting data. Worker 0.2.14 or later must advertise the metric recipe. Checkpoints, metrics, preprocessing, source hashes, and frozen/trainable weight checksums are retained. A valid zero-loss run may need no data-driven updates; inspect `encoder_update_verified` and `optimization_outcome` in model metadata. Checkpoint optimizer/RNG state is saved; pause/resume is not yet exposed for this recipe. Classifier converters reject these checkpoints. The separate metric converter exports the trained encoder as normalized vectors. ### Convert retrieval embeddings to ONNX Choose a worker advertising `dinov3-metric-converter` (worker 0.2.15 or later). The converter uses CPU, fixed 256×256 RGB input, FP32 and ONNX opset 18. Its 6 GiB host-RAM estimate is a planning heuristic. Completed tracked `best.pt` and `last.pt` checkpoints up to 512 MiB are accepted; their original files stay intact. The selected checkpoint's own checksums are verified, including when `best.pt` precedes the final epoch. ```python weights = next(a for a in outputs["data"] if a["filename"] == "best.pt") parameters = { "format": "onnx", "imgsz": [256, 256], "opset": 18, "precision": "fp32", "dynamic": True, "batch": 4, } assessment = client.training.assess_conversion( project_id=project.id, source_artifact_id=weights["_id"], worker_id=conversion_worker_id, parameters=parameters, ) conversion = client.training.convert( project_id=project.id, source_artifact_id=weights["_id"], worker_id=conversion_worker_id, parameters=parameters, idempotency_key="retrieval-onnx-001", ) client.training.wait(conversion["_id"], timeout=600) converted = project.artifacts.list(run_id=conversion["_id"])["data"] model = next(a for a in converted if a["filename"] == "model.onnx") project.artifacts.download(model["_id"], "retrieval.onnx") graph = project.artifacts.inspect(model["_id"]) ``` Use `dynamic=False, batch=1` for a static graph. Dynamic exports have a symbolic batch axis, with nominal batches 1–4 reviewed by DataHub; height and width stay 256. A symbolic ONNX axis does not itself impose a maximum batch size. Conversion metadata retains source lineage, preprocessing, graph constraints and numerical parity results. These results verify conversion, not model accuracy. TensorRT conversion for this embedding adapter is not yet available. Browser and SDK inference process one image at a time, including dynamic-batch exports. They resolve `raw` output, RGB/255 and bicubic stretch to 256; ImageNet and L2 normalization are already in the graph. The output is a 384-value unit vector, not class probabilities. Explicitly incompatible preprocessing or confidence/OCR/pose controls are rejected. ```python inference = client.inference.start( project_id=project.id, source_artifact_id=model["_id"], image="query.jpg", idempotency_key="retrieval-query-001", ) client.inference.wait(inference["_id"]) vector = client.inference.result(inference["_id"])["tensors"][0]["values"] ``` Omitting a worker uses the platform's bounded CPU inference allowance. Supplying a compatible worker uses its CPU. Generated ONNX files remain in the platform after the conversion worker disconnects. ### Reusable YAML or JSON configurations In the dataset version's training dialog, open **Reusable configuration** to download YAML or import settings for review. Browser imports prepare a new run; they do not submit the job. The SDK uses the document's explicit idempotency key, so retrying the unchanged document returns the same run. ```python from pathlib import Path source = Path("datahub-training.yaml").read_text(encoding="utf-8") validated = client.training.validate_config(source) # no allocation or dispatch print(validated["assessment"]) job = client.training.start_config(source) ``` The document contains `schema_version: 1`, `kind: training`, and `run`, whose fields match `training.start()`. Include project/version/export/worker/trainer IDs and an `idempotency_key`. Change the key to intentionally create another run. Trainer defaults and parameter validation are shared with normal UI/SDK jobs; authorization, export readiness and resource checks run again on submission. Documents are limited to 64 KiB. Duplicate keys, YAML anchors/aliases, executable tags, unknown run fields and unsupported schema versions are rejected. This is declarative training configuration; custom Python execution uses its own job API. ### Custom Python, evaluation and benchmark jobs The worker owner enables **Allow custom Python jobs** in account worker settings, or calls `client.workers.update(worker_id, allow_custom_jobs=True)`. The connected agent must advertise custom protocol 1. Project members need dataset export access and the worker must belong to them or be explicitly shared with the project. ```python job = client.training.custom( project_id=project.id, version_id=version.id, export_key=export_key, worker_id=worker_id, environment_id="ultralytics", kind="custom", script='''import json, os from pathlib import Path config = json.loads(Path(os.environ["DATAHUB_CONFIG"]).read_text()) outputs = Path(os.environ["DATAHUB_OUTPUTS"]) outputs.joinpath("metrics.jsonl").write_text( json.dumps({"epoch": 1, "score": config["parameters"]["score"]}) + "\\n") outputs.joinpath("result.json").write_text(json.dumps({"dataset": config["dataset"]})) ''', parameters={"score": 0.7}, execution_device="cpu", estimated_host_ram_mb=2048, time_limit_seconds=300, idempotency_key="my-custom-experiment-001", ) ``` Query `GET /training/custom/environments?project_id=...` for reviewed environment IDs. Scripts run with the selected environment's installed Python dependencies, read-only `/dataset`, code and configuration, and writable `/outputs`. Optional `source_artifact_id` pins a project model by SHA; its path appears in the JSON configuration. Worker control files, credentials and the Docker socket are not mounted. Containers use a nonroot user, read-only root filesystem, disabled network, bounded scratch space and CPU/RAM/GPU/process limits. `kind="evaluation"` and `kind="benchmark"` organize user-defined protocols; their metrics are labelled **Custom script metrics**. Record dataset splits, preprocessing, warmup, iteration counts and task definitions in parameters and output reports for meaningful comparisons. Built-in canonical evaluation and benchmark adapters are separate work. Scripts are limited to 64 KiB, parameters to 32 KiB, and execution to 24 hours. Output file/byte limits and periodic disk checks apply; aggregate filesystem quotas are not assumed. Custom jobs support cancellation and retained logs/artifacts; checkpoint pause is unavailable. ### Parameter sweeps across workers ```python sweep = client.training.sweeps.create( project_id=project.id, version_id=version.id, export_key=export_key, trainer_id="ultralytics-detect", worker_ids=[worker_a_id, worker_b_id], base_parameters={"family": "yolo11", "size": "n", "epochs": 50}, space={"lr0": {"type": "choice", "values": [0.001, 0.005, 0.01]}}, method="grid", max_trials=3, max_parallelism=2, objective_metric="metrics/mAP50-95(B)", objective_mode="max", idempotency_key="detector-lr-comparison-001", ) print(client.training.sweeps.get(sweep["_id"])) print(client.training.sweeps.list(project.id, search="comparison")) # client.training.sweeps.cancel(sweep["_id"]) ``` Each trial is an ordinary, independently inspectable run with metrics, logs and model artifacts. Parameters are validated before materialization. Up to 128 trials and 16 workers are supported; parallelism is enforced across the parent sweep. The worker agent must advertise sweep claim protocol 1. Assignments are fixed among compatible selected workers. This is independent trial execution; gradient synchronization and adaptive worker reassignment are separate features. Seeded `method="random"` also accepts `uniform`, `loguniform` and inclusive `int` dimensions with `low`/`high`. `scheduled_at` schedules a sweep; per-trial wall-clock limits are not currently supported. Cancel stops future dispatch and requests cancellation of existing children. Comparison uses each trial's reported metric under the shared dataset/configuration; it is not a universal model ranking. ## Model Conversion Convert a completed, tracked Ultralytics checkpoint using an independently chosen worker. Original weights remain unchanged. ONNX and TensorRT outputs become downloadable project artifacts, with source checksums and environment provenance. Inspect a proposed conversion before queuing it: ```python settings = {"format": "onnx", "imgsz": [640, 640], "dynamic": True} preview = client.training.assess_conversion( project_id=project.id, source_artifact_id=weight_artifact_id, worker_id=conversion_worker_id, parameters=settings, # gpu_ids=["GPU-..."] # Optional explicit target device. ) print(preview["requirements"]) print(preview["assessment"]) ``` This read-only request creates no run or billing admission. A worker that cannot run the converter returns `assessment["compatible"] == False` with reasons. Review warnings about current resources or unverified capabilities as well. Shape, model, and workspace memory estimates are planning heuristics, not guaranteed minima. Submission and worker launch perform fresh checks. Submit the reviewed settings explicitly: ```python conversion = client.training.convert( project_id=project.id, source_artifact_id=weight_artifact_id, worker_id=conversion_worker_id, parameters=settings, ) converted = client.training.wait(conversion["_id"]) files = project.artifacts.list(run_id=converted["_id"]) ``` Native checkpoint conversion requires CUDA Linux x86_64. A completed verified `model.onnx` can instead target a separate Linux x86_64 worker or the validated ARM64 NVIDIA GB10/CDI cohort (driver 580+, TensorRT 10.14.1). Select its worker ID with the same method and `parameters={"format": "tensorrt", "imgsz": [64, 64], "batch": 1, "workspace_gib": 1, "time_limit_seconds": 360}`. Source dimensions must match the verified ONNX graph. Other ARM devices and Jetson/L4T remain unsupported. The ARM runtime uses a pinned official base plus reviewed offline dependencies; `trusted_python_overlay` (persistent) or `trusted_tmpfs` (ephemeral) and `recipe_verified=false` distinguish this from a verified derived image. Normal installed worker storage remains persistent; explicit ephemeral pilots warn that worker loss can lose unuploaded outputs. TensorRT engines are specific to the target GPU and runtime; choose the worker where you intend to use the engine. Unsupported hardware or conversion settings return an explicit error. Export parity checks compare tensor outputs, not dataset accuracy. Tracked PaddleOCR detector **selected** `model.safetensors` checkpoints use an independent Linux x86_64 CPU worker. They support static FP32 ONNX, batch 1, opset 18, with a square input side from 64 through 1024 in multiples of 32: ```python conversion = client.training.convert( project_id=project.id, source_artifact_id=detector_selected_weight_id, worker_id=conversion_worker_id, parameters={"format": "onnx", "precision": "fp32", "imgsz": [128, 128], "batch": 1, "dynamic": False, "opset": 18}, ) ``` The server verifies the selected checkpoint, configuration, processor and training lineage. `last.safetensors`, arbitrary uploads, TensorRT, FP16 and dynamic detector conversion are unsupported. Detection conversion preserves the original weights and uploads ONNX plus provenance/parity artifacts. The worker's `supported_environment_recipes`, when advertised, must include `paddleocr-detection-converter`. A legacy worker without that list can use an exact cached `datahub-converter-paddleocr-detection:0.1.0` alias or `datahub-recipe-paddleocr-detection-converter:<64-hex-recipe-hash>` image. Otherwise update the worker or choose one with the recipe already cached. This software check is separate from hardware fit and from the ONNX inference protocol: detector conversion does not require inference protocol 2. ## On-Demand Inference ```python capabilities = client.inference.capabilities(project.id) defaults = client.inference.defaults(project.id, onnx_artifact_id) # read-only job = client.inference.start( project_id=project.id, source_artifact_id=onnx_artifact_id, image="example.jpg", worker_id=None, parser="auto", normalization="0_1", input_size=640, ) finished = client.inference.wait(job["_id"]) predictions = client.inference.result(job["_id"]) original_image = client.inference.download_image(job["_id"], "./history/input.jpg") ``` For a historical run, `get()`, `result()`, and `download_image()` read retained records and bytes using GET requests; they never restart inference or queue compute. The worker and original model need not remain available. Project access and the retained input/result files are still required. `download_image()` streams the original uploaded image bytes, including EXIF, to the destination and returns its `Path`; it does not return a resized preview or re-encode the image. Without a worker, DataHub executes bounded ONNX CPU inference on demand. Query capabilities for supported shapes, limits, parsers and execution targets. Unknown output signatures return bounded raw tensors rather than guessed labels. Checkpoint inference requires an explicitly selected worker and a supported, tracked model origin. Ultralytics checkpoints support detection, classification, instance masks, 2D pose and oriented boxes. Pose returns normalized keypoints with confidence and trained skeleton metadata; masks retain holes through COCO RLE. These outputs have explicit image, prediction-count and result-size limits. The initial worker ONNX runtime supports CPU execution; ONNX GPU requests are rejected until a compatible CUDA runtime is available. Native pose inference uses the reviewed `inference-native-pose-v1` worker recipe when `capabilities.native_pose.enabled` is true. Position-only (XY) models return keypoint `confidence=None` and `visible=None`; keep these unavailable values distinct from zero confidence or certainty. Instance confidence remains separate. XY + visibility models retain reported keypoint probabilities. Their optional `pose_visibility_confidence` override defaults to the runtime threshold when omitted; position-only models reject an explicit threshold. Native checkpoints use scalar stride-32 `input_size`, not the ONNX/engine `pose_input_shape` or `pose_max_candidates` settings. Saved legacy results are not rewritten. Reviewed tracked pose ONNX conversions support CPU keypoint inference when the server advertises `inference_capabilities.pose_onnx`. The default `parser="auto"` resolves the tracked pose model; explicit `parser="ultralytics_pose"` also works. Use the exact `model.onnx` artifact from the reviewed pose converter. Older generic conversions and arbitrary uploaded pose graphs are not admitted by this adapter. ```python defaults = client.inference.defaults(project.id, pose_onnx_artifact_id) print(defaults["parameters"]["pose_input_shape"]) # [height, width] job = client.inference.start( project_id=project.id, source_artifact_id=pose_onnx_artifact_id, image="test.png", # DataHub CPU, or pass an eligible worker_id pose_max_candidates=1000, ) ``` Omit `input_size` to use source-resolved dimensions (generic models default to 640). An explicitly supplied scalar, including `640`, is rejected for pose; use `pose_input_shape=[height, width]` for dynamic models. Static model shapes cannot be changed. Pose uses RGB/255 and centered letterbox preprocessing; `confidence`, `iou`, and `max_predictions` control instance filtering. `pose_visibility_confidence` controls keypoint confidence filtering for models that report it. Position-only models return `confidence=None` and `visible=None` for keypoints; these values are unavailable, not zero confidence or certainty. Coordinates remain relative to the original image with tracked names and topology. Registered CPU workers need the `inference-pose-v1` recipe in their supported list; DataHub CPU remains subject to its included execution allowance. Deployments advertising `capabilities.pose_tensorrt.enabled` also accept reviewed, tracked pose `model.engine` artifacts. Read `defaults()` to inspect the saved profile and `model_metadata.source_target_gpu_uuid`. Select a registered Linux x86_64 worker supporting `inference-pose-tensorrt-v1` and that exact physical GPU: ```python defaults = client.inference.defaults(project.id, pose_engine_artifact_id) job = client.inference.start( project_id=project.id, source_artifact_id=pose_engine_artifact_id, image="test.png", worker_id=worker_id, gpu_ids=[defaults["model_metadata"]["source_target_gpu_uuid"]], ) ``` These engines have no CPU fallback. The worker verifies the engine checksum, GPU identity and reviewed runtime before loading. Re-registering the same GPU is supported; another GPU with the same model name is insufficient. Static dimensions remain fixed; dynamic `pose_input_shape` overrides must fit the recorded profile. Position-only keypoints retain null confidence and visibility. An engine artifact's presence alone does not establish inference compatibility; the server validates its completed conversion provenance. PaddleOCR detector ONNX inference returns text-region polygons and confidence, without recognizing or inventing a transcript. A tracked conversion resolves its static model side, graph-embedded normalization and saved detector defaults automatically. Its preprocessing preserves aspect ratio, rounds dimensions to stride 32 and pads at the bottom/right; incompatible transforms are rejected. `client.inference.defaults(project.id, artifact_id)` returns source identity, canonical parameters and environment without executing inference, reading model weights or consuming compute allowance. Use its resolved size and thresholds when building configurable clients. ```python job = client.inference.start( project_id=project.id, source_artifact_id=detector_onnx_artifact_id, image="page.png", worker_id=inference_worker_id, # omit for DataHub CPU parser="ocr_detection", normalization="0_1", resize_mode="ocr_stride32_top_left", input_size=128, det_threshold=0.3, box_threshold=0.6, unclip_ratio=1.5, max_candidates=1000, max_predictions=50, ) ``` The four detector settings are optional flat options and are omitted from the request when `None`, allowing tracked training defaults to resolve on the server. `max_candidates` is bounded at 10,000 and returned `max_predictions` at 100. OCR uses `det_threshold`/`box_threshold`; generic `confidence` and `iou` must retain their defaults rather than silently becoming ignored controls. Static tracked detector raw output uses the same correct preprocessing. OCR worker inference requires a Linux x86_64 Docker worker advertising `capabilities.onnx_cpu_inference_protocol == 2`. Update an older worker when the server reports an upgrade requirement; this is distinct from insufficient memory. The capability is reported by the worker, not supplied in an inference request. Current workers use the thin CPU ONNX environment for all new ONNX inference; older workers retain the legacy environment for non-OCR models. If `supported_environment_recipes` is advertised, it must also include `inference-ocr-v1`; an explicit empty list denies eligibility even with a cached image. A missing list remains compatible when protocol 2 is advertised. DataHub CPU fallback does not require a worker capability. ## SDK Function Index Use this as the compact map of the public SDK surface when writing automation or using this document as LLM context. ### Client ```python af.verify() af.workspaces() af.workspace() af.workspace("WORKSPACE_ID_OR_SLUG") af.create_workspace(name, slug, description="", logo=None) ``` ### Workspace ```python workspace.projects() workspace.project("PROJECT_ID_OR_NAME") workspace.create_project(name, description="") workspace.upload_dataset("./dataset", project_name="Project Name", **upload_options) ``` ### Project image lookup and iteration ```python project.list_images(page_number=0, page_size=100, include_labels=False, include_image=False, as_dict=False, **filters) project.count_images(**filters) project.find_images(page_size=500, **filters) project.first_image(**filters) project.live(page_size=100, include_labels=False, include_image=False, **filters) project.iter_images(...) # alias for live project.live_batches(page_size=500, **filters) project.iter_image_batches(...) # alias for live_batches project.matching_image_ids(**filters) project.image("IMAGE_ID") ``` ### Project upload, download, and delete ```python project.upload(image, labels=None, annotations=None, filename=None, batch_name=None, tags=None, split_name=None) project.upload_bytes(image_bytes, filename=None, **upload_options) project.upload_dataset("./dataset", batch_name=None, tags=None, batch_size=25, show_progress=True) project.upload_auto_dataset("./dataset", **upload_options) project.upload_yolo_dataset("./dataset", dataset_format="yolov8", annotation_format="bbox", split_name="auto") project.download_image("IMAGE_ID", location="./downloads") project.delete_image("IMAGE_ID") project.delete_images(["IMAGE_ID_1", "IMAGE_ID_2"]) project.delete_matching(dry_run=True, **filters) ``` ### Project labels and review status Annotation reads return a list that also carries `image_revision`, `source_write_sequence`, and `class_revision`. Keep that list when editing: the SDK sends the captured source and class identities on save and in each bulk item. Its `.copy()` and slices retain those identities. A concurrent image transform or class rename produces a conflict so you can review the draft. ```python labels = project.get_annotations("IMAGE_ID") pixels = project.download_image( "IMAGE_ID", as_bytes=True, image_revision=labels.image_revision ) # Edit labels using these pixels, then save the same snapshot. project.update_annotations("IMAGE_ID", labels, overwrite=True) ``` For an `ImageRecord`, `refresh_annotations()` captures the source and canonical dimensions together; `image.download()` pins that source. A source change clears its cached pixels. On a conflict, keep the draft and explicitly reload the image before deciding how to reapply edits. Do not fetch a newer revision solely to retry old coordinates or old class names. When storing labels as plain JSON, store all three revision fields too and pass them back explicitly: ```python project.update_annotations( "IMAGE_ID", stored_labels, overwrite=True, expected_image_revision=stored_image_revision, expected_source_write_sequence=stored_source_sequence, expected_class_revision=stored_class_revision, ) ``` ```python project.get_annotations("IMAGE_ID") project.update_annotations("IMAGE_ID", annotations, overwrite=False, verification_status=None, model_agreement=None) project.update_labels(...) # alias for update_annotations project.write_annotations("IMAGE_ID", annotations, verification_status=None, model_agreement=None) project.write_labels(...) # alias for write_annotations project.update_annotations_bulk([{"image_id": "IMAGE_ID", "labels": labels}], overwrite=False) project.update_labels_bulk(...) # alias for update_annotations_bulk project.update_image_verification(["IMAGE_ID"], "verified") project.set_image_verification(...) # alias project.mark_images_verified(["IMAGE_ID"]) project.mark_images_unverified(["IMAGE_ID"]) project.update_image_model_agreement(["IMAGE_ID"], "agree") project.set_image_model_agreement(...) # alias project.mark_images_model_agree(["IMAGE_ID"]) project.mark_images_model_disagree(["IMAGE_ID"]) project.clear_images_model_agreement(["IMAGE_ID"]) project.mark_model_agree(...) # alias for mark_images_model_agree project.mark_model_disagree(...) # alias for mark_images_model_disagree project.clear_model_agreement(...) # alias for clear_images_model_agreement project.update_images_metadata(["IMAGE_ID"], add_tags=["reviewed"], split_name="train", model_agreement="none") project.update_matching(dry_run=True, add_tags=["reviewed"], model_agreement="agree", **filters) project.iter_labels(page_size=100, **filters) ``` ### Project metadata and versions ```python project.classes() project.tags() project.versions() project.version("VERSION_ID_OR_NUMBER") project.create_version(name, notes="", split=(80, 10, 10)) project.version_source_readiness(image_ids=None) project.status project.clone_status project.clone(name, target_workspace_id=None, description=None, wait=False, timeout=600, poll_interval=1) project.refresh() project.wait_for_clone(timeout=600, poll_interval=1) project.cancel_clone() ``` ### ImageRecord ```python image.id image.filename image.tags image.batch_name image.split_name image.width image.height image.verification_status image.is_verified image.model_agreement image.model_agree image.labels image.labels_dicts image.absolute_labels image.refresh_labels() image.reload() image.update_labels(labels, overwrite=False, verification_status=None, model_agreement=None) image.write_labels(labels, verification_status=None, model_agreement=None) image.add_labels(labels, verification_status=None, model_agreement=None) image.set_labels(labels, verification_status=None, model_agreement=None) image.set_verification_status("verified") image.mark_verified() image.mark_unverified() image.set_model_agreement("agree") image.mark_model_agree() image.mark_model_disagree() image.clear_model_agreement() image.download("./downloads") image.download(as_bytes=True) image.download(as_pil=True) image.download(as_cv2=True) image.download(as_base64=True) image.delete() ``` ### Version ```python version.refresh() version.live(split_name="all", page_size=100) version.iter_images(...) # alias for live version.iter_labels(split_name="val", page_size=100) version.download_image("IMAGE_ID", split_name="train", location="./downloads") version.generate_export("yolo", export_options=None) version.download("yolo", location="./exports") ``` ### Label helpers ```python from algofly import ( LabelRecord, labels_to_dicts, label_to_absolute, labels_to_absolute, label_to_normalized, labels_to_normalized, ) ``` ## Practical Notes - Use `from algofly import AlgoFly`. - API URLs may include or omit `/api`; the SDK appends it when missing. - Omit `id` for new labels. The SDK generates a stable id from the image id and label data. - Custom ids must include an AlgoFly prefix, or include `type`/`label_type` so the SDK can add the prefix. - Preserve existing label ids when editing labels returned by the API. - `update_labels` merges labels by id; use `write_labels` or `overwrite=True` to replace all labels. - Box coordinates use `left`, `top`, `width`, and `height`. - Rotated boxes use bbox geometry plus `angle`; `cx`/`cy` center form and four-corner `points` are accepted. - Polygon coordinates use `points`. - Point coordinates use `x` and `y`. - Line coordinates use `x1`, `y1`, `x2`, and `y2`. - OCR with no ROI geometry is stored as whole-image OCR. - OCR with ROI geometry is stored on that annotation under `ocr.text`. - Multi-label image classification is supported for storage, web annotation, and export. Enable the Multi-label toggle in the classification label panel to select more than one class for an image. ### Read annotations by id or custom fields If you upload crops with custom annotation fields, you can fetch the crop annotation directly by id or search by one of those fields: ```python crop = pipes_counting_project.annotation(crop_id) print(crop.image_id, crop.annotation.label) matches = pipes_counting_project.annotations_by_metadata( "source_image_id", original_image.id, ) for match in matches: print(match.image_id, match.annotation.source_label_id, match.annotation.label) ``` ### Find an uploaded crop image from crop pixels Use an explicit match mode: ```python crop_image = pipes_counting_project.find_image_by_crop( crop, match="hash", load_labels=True, ) best_crop_image = pipes_counting_project.find_image_by_crop( crop, match="embedding", load_labels=True, ) print(best_crop_image.id, best_crop_image.match_score) ``` When uploading crops, put source mapping on the crop image too: ```python classification_project.upload( crop, tags=image.tags, metadata={ "source_image_id": image.id, "source_image_filename": image.filename, "source_label_id": label["id"], }, annotations=[ { "id": "image-classification", "type": "image_classification", "label": str(count), } ], ) ``` ### Pose and skeletons Discover built-in and immutable project templates with `project.skeletons.list()`. Create a custom version with `project.skeletons.create(name, keypoint_names, edges=[[0, 1]], flip_pairs=[[0, 1]], previous_id=template_id)`. Edge and flip indices are zero based; annotation snapshots retain their original name order. ```python skeleton = next(item for item in project.skeletons.list() if item["id"] == "coco17") pose = { "type": "pose", "label": "person", "instance_id": "person-1", "skeleton": skeleton, "keypoints": [{"x": 0.5, "y": 0.5, "visibility": 2} for _ in skeleton["keypoints"]], } project.update_annotations(image.id, [pose]) ``` Coordinates are normalized image coordinates. Visibility is `0` unlabelled, `1` occluded, or `2` visible; unlabelled coordinates become zero. Update each landmark to its actual position before exporting. `label_to_absolute` and `label_to_normalized` preserve visibility and skeleton metadata. Use `version.download("COCO Keypoints", location="./pose")` or `version.download("YOLO Pose", location="./pose")`. COCO requires a consistent skeleton within each class; YOLO requires one common skeleton across classes and a nonzero box derived from labelled landmarks. Exports reject incompatible orders instead of truncating keypoints. YOLO includes `kpt_shape`, `flip_idx`, `skeleton.json`, and instance metadata sidecars. The `ultralytics-pose` trainer defaults to `keypoint_dimensions=3` (XY + visibility). Choose the integer `2` for a trained XY-only head; every source keypoint must be labelled (visibility greater than zero). The adapter derives training labels without changing source annotations. `nbs` is the nominal batch size used for gradient accumulation and defaults to 64; change it deliberately for the desired training configuration. A completed short run establishes execution, not model accuracy. ### OCR recognition crops `version.download("PaddleOCR Recognition", location="./recognition")` downloads rectified word/line crops, exact UTF-8 transcripts and original ROI provenance. Each split has `crops/`, `metadata/` and `labels.txt` (`pathtranscription`). Unsupported geometry or multiline text is counted as skipped. This export does not turn image-level page transcripts into line supervision. The `paddleocr-recognition` trainer fine-tunes the pretrained PP-OCRv6 Tiny recognizer backbone and CTC head. It preserves the pretrained dictionary and rejects unknown characters. This trains recognition only; text detection, layout and end-to-end OCR training are separate capabilities. ### Raster training chips TIFF/GeoTIFF sources require explicit chip mode. The original scientific raster stays unchanged; training chips are RGB uint8 PNGs using selected native-resolution bands and one fixed range per output channel. Omitted bands/ranges are frozen from each source's display metadata. Only page zero is currently annotated. ```python chips = {"width": 512, "height": 512, "overlap_x": 0, "overlap_y": 0, "windows": [{"x": 4000, "y": 3600, "width": 1024, "height": 512}], "classification_policy": "drop", "nodata": "skip_all"} preview = project.preview_raster_chips(raster_a, chips) version = project.create_version("raster-chips", image_ids=[raster_a, raster_b], split=(50, 50, 0), raster_chips=chips) # Poll version.refresh().status until completed before generating an export. archive = version.download("yolo", location="./chips", export_options={"convert_to_bounding_boxes": True}) ``` Every chip from a source inherits the same split, including overlapping chips. One source cannot supply independent training and validation data. Source image classifications are dropped unless `classification_policy="inherit"` is chosen; global OCR transcripts are omitted. Partial OCR regions lose their transcript with an explicit warning. Regional geometry is cropped while source annotations remain unchanged. Supported exports include per-chip source SHA, native pixel window, selected bands/ranges, CRS and affine provenance sidecars; SDK version records expose `record.raster_chip`. Native-dtype scientific TIFF chips and additional-page annotation are separate pending capabilities. ### Derived storage accounting New dataset versions, generated format files and cached ZIP archives count toward workspace storage. Writes reserve bytes before upload; deletion releases bytes after storage confirms removal. Chip base images use their existing base ledger and are not charged again as ordinary version objects. Historical files can be measured and accounted one bounded page at a time. This requires the workspace billing owner and defaults to a read-only dry run: ```python audit = version.audit_storage(wait=True, limit=500) report = audit["result"] applied = version.audit_storage(apply=True, wait=True, limit=500) # Repeat with report["next_cursor"] until it is None; inspect blocked_objects. ``` Backfill records existing bytes without changing or deleting dataset files. It is not an automatic global migration. Existing data remains downloadable by normal streaming when storage is full; creating a new cached archive may return `402 workspace_storage_limit`. An interrupted deletion can return `version_object_delete_pending`; create a new version or ask an operator for explicit scoped reconciliation. Missing object HEAD alone does not clear that safety fence. ### DICOM artifact viewing Upload a DICOM file as a project artifact, or as an independent account artifact. The browser's **DICOM** action opens lazy frame navigation and window controls. The SDK exposes the same private metadata and frame access: ```python source = project.artifacts.upload("series.dcm") metadata = project.artifacts.medical.metadata(source["_id"]) print(metadata["frame_count"], metadata["series_digest"]) project.artifacts.medical.frame(source["_id"], 0, "first-frame.png") project.artifacts.medical.frame(source["_id"], 12, "windowed-frame.png", center=40, width=400, function="LINEAR") # For account artifacts, use client.artifacts.medical instead. ``` SDK frame indices start at zero; the browser shows frame numbers starting at one. Frames are display PNGs: modality rescale/LUT precedes VOI window/LUT, including MONOCHROME1 inversion and pixel padding. Original stored pixels remain unchanged. Technical metadata hashes study/series/instance identifiers and omits patient tags; this does not de-identify the original file or alter burned-in pixels. The current verified subset is native explicit/implicit little-endian and explicit big-endian monochrome 8/16-bit images. Reads use revision-pinned storage ranges, bounded native decoding and a small private browser cache. Unsupported compressed, color and floating-point layouts fail explicitly. Frame-scoped source imports and native version materialization are available below; medical segmentation formats and full study-aware workflows remain pending. ### DICOM sources and virtual annotation frames Move an owned, unlocked project artifact into a retained original source. Frame indices are zero-based and `stop_exclusive` is exclusive; an omitted stop selects through the last frame. Original bytes are stored once; registered virtual frames each count toward the image quota. Repeating the same selection reuses frame identities. Account Library artifacts cannot be moved directly. ```python uploaded = project.artifacts.upload("series.dcm") job = project.medical_sources.import_artifact( uploaded["_id"], start=0, stop_exclusive=2, wait=True) if job.get("recovery_required"): # Inspect job["error"] and retry the stored immutable selection explicitly. job = project.medical_sources.retry_import(job["_id"], wait=True) if job["status"] != "completed": raise RuntimeError(job.get("error") or job["status"]) source_id = job["source_id"] source = project.medical_sources.get(source_id) sources = project.medical_sources.list(search="series") remaining = project.medical_sources.add_frames(source_id, start=2, wait=True) project.medical_sources.download(source_id, "original-series.dcm") # Explicitly remove the source and its frames/annotations; snapshots stay separate. deleted = project.medical_sources.delete(source_id, wait=True) ``` The staging artifact entry is consumed after a successful move. The immutable original remains downloadable through `medical_sources.download`; its SHA is unchanged. Annotation and version preparation use project frame identities. Native version creation requires explicit `medical_frames` window and padding settings; browser display windows never silently set training pixels. Wait helpers return completed, failed, cancelled, or `recovery_required` jobs. They never automatically restart exhausted publication. Asynchronous calls return a durable job immediately; inspect it with `import_status(job["_id"])`. An SDK timeout leaves that server-side job running. Deletion is asynchronous too; use `delete(..., wait=True)` before assuming the source has disappeared. ### Native model evaluation and warm inference benchmarks `training.evaluate` and `training.benchmark` create reviewed worker runs with kinds `model_evaluation` and `inference_benchmark`. They are separate from `training.custom(kind="evaluation" | "benchmark")`, whose metrics follow user scripts. Native tasks cover single-label classification or axis-aligned bounding-box detection on a frozen version's validation or test split. CPU is the default; tracked Ultralytics PT also has the guarded CUDA path described below. A generated COCO/ImageFolder export is not required; `export_key` is optional provenance. No model fitting or data augmentation occurs. Discover the exact deployed schema with `client.training.evaluation_capabilities()`. Use `client.training.evaluation_defaults(project_id=..., version_id=..., source_artifact_id=...)` for verified tracked-model input/output metadata and unique exact-label mapping recommendations. Check `review_required` and `reasons` before using them. Uncertain, renamed or user-declared inputs require review; this read never starts a run or invents training lineage. Supply explicit input size, resize/interpolation, normalization, output parser, score semantics and a complete class mapping. Model output indices must match the evaluated frozen version classes. Unused version classes may remain unmapped; every selected ground-truth annotation must map, and ambiguous labels reject. Supported parser/layout pairs are `classification`/`NC`, `yolo_v8_detect`/`BCN`, and `end_to_end_detect`/`BNC`. Detection scores are probabilities. The server never guesses a parser from tensor dimensions. ```python # Example: a reviewed DINOv3-converted classification ONNX model. # Use the actual frozen labels and model output order, not alphabetic guesses. parameters = { "model_contract": { "input": {"size": [256, 256], "resize": "stretch", "interpolation": "bicubic", "normalization": "embedded"}, "output": {"parser": "classification", "layout": "NC", "score_semantics": "logits"}, }, "class_mapping": {"Actual first model class": 0, "Actual second model class": 1}, } evaluation = client.training.evaluate( project_id=project.id, version_id="FROZEN_VERSION_ID", source_artifact_id="MODEL_ARTIFACT_ID", worker_id="WORKER_ID", task="classification", split="test", parameters=parameters, idempotency_key="evaluate-model-version-test-v1", ) finished = client.training.wait(evaluation["_id"]) if finished["status"] != "completed": raise RuntimeError(finished.get("error") or finished["status"]) result = client.training.evaluation_results(evaluation["_id"]) print(result["summary"]["metrics"]) project.artifacts.download( result["artifacts"]["confusion-matrix.json"]["id"], "confusion-matrix.json") benchmark = client.training.benchmark( project_id=project.id, version_id="FROZEN_VERSION_ID", source_artifact_id="MODEL_ARTIFACT_ID", worker_id="WORKER_ID", task="classification", split="test", parameters={**parameters, "scope": "end_to_end", "batch": 1, "warmup_iterations": 10, "measured_iterations": 100, "sample_images": 32, "cpu_threads": 1, "max_duration_seconds": 300}, idempotency_key="benchmark-model-version-test-v1", ) ``` Reviewed tracked Ultralytics classification/detection checkpoints and supported ONNX models are accepted. DINOv3 conversion uses the exact fixed contract above; ImageNet normalization is already in its graph. A generic uploaded ONNX model also needs ordered `model_contract.class_names`; its contract is user-declared, with no implied training lineage. Arbitrary uploaded pickle checkpoints are not accepted. External tensors, custom operators or incompatible signatures fail rather than execute API-host code. Workers need native protocol support; custom-Python owner opt-in is not required for reviewed native adapters. Classification reports accuracy, confusion counts and per-class/aggregate precision, recall and F1. Detection reports the pinned COCO-style bbox protocol, not official leaderboard accuracy. Verified negative images participate; filtered-empty or unsupported ground truth fails closed. Results and prediction artifacts retain model SHA, version/split, source/GT and class-mapping provenance. Default evaluation processes the whole split up to the current 1,000-image capacity; larger splits require an explicit seeded `max_images` subset. Benchmarks load the model once, exclude warmups, and retain measured samples, p50/p90/p95/p99, throughput and runtime/hardware metadata. `model_only` excludes decode/preprocessing/postprocessing; `end_to_end` includes them. Cold loading is reported separately. Incomplete measurements do not become completed benchmarks. `parameters.time_limit_seconds` bounds the whole execution (default and maximum 3,600 seconds); `max_duration_seconds` separately limits measured benchmark work. Compare completed native runs with `client.training.compare(project.id, run_ids)`; incompatible cohorts include reasons and are never automatically ranked. Native CUDA is a separately gated path for tracked Ultralytics PT models only. CPU remains the default, and every ONNX model remains CPU-only. Check `evaluation_capabilities()["execution_devices"]` and `conditional_execution_devices["cuda"]` before exposing CUDA to users. Its `status="validated_subset"` and `validated_hardware` describe the tested matrix; `validation_pending` still means that deployment has no completed physical proof. Select `execution_device="cuda"` and exactly one canonical lowercase `GPU-...` UUID in `gpu_ids`; use the selected worker's reported UUID, never an automatic device index. The server validates tracked PT lineage, worker capability, fresh capacity and the GPU lease. Failure never silently switches the request to CPU. The native CUDA recipe preserves a 2,048 MiB free GPU reserve beyond its model estimate and monitors it during execution. Its PyTorch allocator limit and cooperative monitor are not a hard GPU memory quota. Physical classification and bounding-box detection pilots passed on Luna's Linux x86_64 NVIDIA RTX 3060 (compute capability 8.6), driver 595.99.02 and PyTorch CUDA 13.2. Their small test fixtures matched CPU predictions, including nonempty detector output, with FP32 inputs, synchronization and TF32 disabled. This verifies that tested combination, not every model, GPU, driver or CUDA version. Other supported combinations remain subject to the worker's device, compiled-architecture, runtime and resource guards and need their own validation. CUDA result provenance records actual provider/device, synchronization, precision, TF32 policy and GPU/runtime characteristics. CPU/CUDA or different runtime/hardware cohorts remain incompatible for automatic timing comparisons. These pilots do not establish model accuracy on production data or controlled hardware rankings. ## Original video sources and exact frames In the browser, open the project's **Videos** tab to upload an original, follow indexing, search sources and inspect or download exact frames. Project data managers can cancel, retry or delete sources there. Live updates refresh an open tab when another client registers or changes a source. Upload the original once as a project artifact, then register it for background indexing. Registration resolves the stored checksum and size on the server. Reusing an idempotency key preserves the original request; a retired key cannot recreate a deleted source. ```python artifact = project.artifacts.upload("clip.mp4") source = project.video_sources.register( artifact["_id"], title="Inspection clip", idempotency_key="inspection-clip-1" ) source = project.video_sources.get(source["_id"]) ``` Poll `get()` until `status` is `ready`, `failed` or `cancelled`. A polling timeout does not stop the server job. `indexed_frames` reports progress; errors include a code, message and retryability. Use `cancel()` to request a stop and `retry()` to retry a failed or cancelled source. Indexing uses bounded CPU, memory and disk; busy capacity stays queued. Unsupported codecs and resource limits are explicit. Once ready, retain the returned revision for every timeline and PNG request: ```python revision = source["index_revision"] timeline = project.video_sources.frames( source["_id"], index_revision=revision, offset=0, limit=100 ) project.video_sources.frame( source["_id"], 5, "frame-000005.png", index_revision=revision ) ``` Ordinals are zero-based presentation-frame identities, including variable-rate video. Original PTS values remain integer strings, with missing and best-effort timestamp evidence kept separate. The server rejects stale revisions. PNGs use square pixels and the supported display rotation; the original artifact remains unchanged. Initial decoding starts from the beginning, with an explicit ordinal and execution budget; certified random-access seeking is not yet available. `list(search="Inspection", status="ready")` returns bounded pages. `delete()` removes the source and its derived indexes after processing stops, preserving the original artifact. Artifact deletion is blocked while a source references it. Uncertain uploads retain their storage reservation until verified recovery. Register a bounded range as ordinary annotatable dataset images when `source["features"]["frame_registration"]["available"]` is true: ```python intent = project.video_sources.register_frames( source["_id"], index_revision=source["index_revision"], start=0, stop_exclusive=120, stride=10, wait=True, ) history = project.video_sources.registrations(source_id=source["_id"]) intent = project.video_sources.registration(intent["_id"]) ``` Ordinals are zero-based and the stop is exclusive. Follow the advertised `max_selected_frames` and `max_ordinal` limits. Registration consumes image slots, stores the original once, and keeps annotations image-relative. Registered frames support dataset augmentation and image-format exports with temporal provenance sidecars. Pixel transforms and full-image SAM are disabled for them. `cancel_registration(id)` requests cancellation before publication; `retry_registration(id, wait=True)` explicitly retries a clean failed/cancelled operation. Repeating a completed selection returns its historical operation. To restore logically deleted frames, send the same frozen selection with `restore_missing=True` and a new `idempotency_key`; reuse that key if delivery is uncertain. Existing published frames and their annotations remain unchanged. `current_visible_count` reports the frames currently present. Video tracks, native temporal annotation formats, and source-preserving project cloning remain separate capabilities and are not enabled by frame registration.