Skip to content

Backfill should handle rows with missing AnalysisDate gracefully #571

@kbighorse

Description

@kbighorse

Context

Rows with missing AnalysisDate were previously inserted with a 1970-01-01 epoch sentinel, which is indistinguishable from real data downstream. In #558 these rows are now skipped with an error logged to BackfillResult.errors.

Work Required

Decide on a longer-term strategy for date-less observations:

  • Option A: Keep skipping (current behavior) and report to data stewards for manual correction
  • Option B: Allow nullable observation_datetime (schema change) for legacy data
  • Option C: Derive date from a related field (e.g., sample collection date from NMA_Chemistry_SampleInfo)

Refs

Changed in #558

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions