Extras in unmerged copy the standard

[VisLab]

No description provided.

[Copilot]

Pull request overview

This PR simplifies the handling of schema extras sections (Sources, Prefixes, ExternalAnnotations) by removing the in_library tracking mechanism. Previously, the system tracked which extras entries came from which library using an internal in_library column, allowing it to filter entries when saving unmerged library schemas. The new approach is simpler: unmerged library schema files now contain full copies of the standard schema's extras sections, and both merged and unmerged saves output all entries identically.

Changes:

• Removed in_library column tracking from all schema I/O modules (XML, JSON, MediaWiki, TSV readers and writers)
• Added new merge_extras_dataframes utility function that combines extras by simple concatenation and deduplication
• Replaced three format-specific roundtrip tests with one comprehensive test covering all formats
• Expanded test data files to include full standard extras in library schema files (14 prefixes, 17 external annotations, 2 sources)

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/schema/test_schema_extras_xml_roundtrip.py	Deleted - replaced by comprehensive test
tests/schema/test_schema_extras_wiki_roundtrip.py	Deleted - replaced by comprehensive test
tests/schema/test_schema_extras_json_roundtrip.py	Deleted - replaced by comprehensive test
tests/schema/test_schema_extras_comprehensive.py	New comprehensive test covering all formats and roundtrip scenarios
tests/data/schema_tests/test_merge/HED_testlib_4.0.0.xml	Expanded with full standard extras (14 prefixes, 17 external annotations)
tests/data/schema_tests/test_merge/HED_testlib_4.0.0.json	New JSON format test data with full standard extras
tests/data/schema_tests/test_merge/HED_testlib_4.0.0.mediawiki	New MediaWiki format test data with full standard extras
tests/data/schema_tests/test_merge/HED_testlib_4.0.0/*.tsv	New TSV format test data files with full standard extras
hed/schema/schema_io/xml2schema.py	Removed in_library tracking on read, simplified merge logic
hed/schema/schema_io/wiki2schema.py	Removed in_library tracking on read, simplified merge logic
hed/schema/schema_io/json2schema.py	Removed in_library tracking on read, simplified merge logic
hed/schema/schema_io/schema2xml.py	Removed in_library filtering and attribute output on write
hed/schema/schema_io/schema2wiki.py	Removed in_library filtering and attribute output on write
hed/schema/schema_io/schema2json.py	Removed in_library filtering and attribute output on write
hed/schema/schema_io/df_util.py	Added merge_extras_dataframes function for simple concatenation and deduplication
hed/schema/schema_io/df2schema.py	Updated to use new merge_extras_dataframes function

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.