Skip to content

Comments

Add submit_external_metrics#2042

Open
kozikkamil wants to merge 1 commit intodevelopfrom
upsert-external-metrics
Open

Add submit_external_metrics#2042
kozikkamil wants to merge 1 commit intodevelopfrom
upsert-external-metrics

Conversation

@kozikkamil
Copy link
Contributor

@kozikkamil kozikkamil commented Feb 10, 2026

Description

Allow syncing external metrics to projects

Example usage:

project = client.get_project(PROJECT_ID)

# Build entries using the new schema# Build entries using the new schema
entries = [
    # Entry 1: full label + review with rating, done queue
    lb.ProjectSyncEntry(
        task_id="test-task-031",
        content_url="https://storage.example.com/data/test-task-031.json",
        label=lb.ProjectSyncLabel(
            submitted_by=lb.SubmittedBy(email="user@email.com"),
            auto_qa=lb.AutoQA(
                status=lb.AutoQaStatus.Approve,
                score=0.92,
                feedback="Looks good",
                custom_scores=[
                    lb.CustomScore(name="accuracy", value=0.95),
                    lb.CustomScore(name="relevance", value=0.88),
                ],
            ),
            seconds_to_completion=45.0,
            submitted_on="2025-06-15T12:00:00.000Z",
        ),
        review=lb.ProjectSyncReview(
            reviewed_by=lb.ReviewedBy(email="user@email.com"),
            rating=lb.GranularRating(score=5, comment="Excellent work"),
        ),
        queue_type="DONE_QUEUE",
    ),
    # Entry 2: label + review without rating, done queue
    lb.ProjectSyncEntry(
        task_id="test-task-032",
        label=lb.ProjectSyncLabel(
            submitted_by=lb.SubmittedBy(email="user@email.com"),
            seconds_to_completion=30.0,
        ),
        review=lb.ProjectSyncReview(
            reviewed_by=lb.ReviewedBy(email="user@email.com"),
        ),
        queue_type="DONE_QUEUE",
    ),
    # Entry 3: label with rejected autoQA, no review, no content_url
    lb.ProjectSyncEntry(
        task_id="test-task-033",
        label=lb.ProjectSyncLabel(
            submitted_by=lb.SubmittedBy(email="user@email.com"),
            auto_qa=lb.AutoQA(
                status=lb.AutoQaStatus.Reject,
                score=0.3,
                feedback="Multiple errors found",
            ),
            seconds_to_completion=120.5,
        ),
    ),
]

print(f"Syncing {len(entries)} entries...")
result = project.sync_external_project(entries)
print(f"  submissionId={result.submission_id}")
print("Done!")

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you updated any code comments, as applicable?

Note

Medium Risk
Adds a new GraphQL mutation path and input serialization for project data sync; risk is mainly schema mismatch/serialization edge cases (e.g., optional vs explicit null) causing failed or incorrect syncs.

Overview
Adds a new Project.sync_external_project() method that calls a syncExternalProject GraphQL mutation to asynchronously push external label/review metadata (including AutoQA status/scores and queue state) into a project and returns a submission_id for tracking.

Introduces a new project_sync pydantic schema (ProjectSyncEntry, ProjectSyncLabel/Review, AutoQA, etc.) plus a _to_gql_input serializer (including explicit null support), and re-exports these types from labelbox.__init__ for SDK users.

Written by Cursor Bugbot for commit f0b166d. This will update automatically on new commits. Configure here.

content_url: Optional[str] = None
label: Optional[ProjectSyncLabel] = None
review: Optional[ProjectSyncReview] = None
queue_type: Optional[str] = None
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing row_status field in ProjectSyncEntry model

High Severity

The ProjectSyncEntry model is missing a row_status field, even though the PR description's example usage shows row_status="PASS" being passed. Since Pydantic ignores extra fields by default, any row_status value provided by users will be silently dropped. The _to_gql_input function also lacks serialization logic for this field, so even if the model were fixed, the value would never reach the GraphQL API.

Additional Locations (1)

Fix in Cursor Fix in Web

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

{"name": cs.name, "value": cs.value}
for cs in entry.review.custom_scores
]
result["review"] = review
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing explicit-None handling for review field in serialization

Medium Severity

The _to_gql_input function handles explicitly-set None values via model_fields_set checks for label and queue_type, but the review field is missing this same elif branch. When a user explicitly passes review=None to a ProjectSyncEntry, the review key is silently omitted from the GQL input instead of being sent as None, which is inconsistent with how label and queue_type behave.

Fix in Cursor Fix in Web

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant