feat: update relationship schema

[omkar-ethz]

Description

Add fields to relationship schema used by relationships field in the dataset schema.

Testing:
PATCH /api/v4/datasets/:pid

{
  "relationships": [
    {
      "identifier": "https://scilog.development.psi.ch/logbooks/6895bea625f055bca783dfdd",
      "identifierType": "URL",
      "entityType": "Logbook",
      "externalId": "6895bea625f055bca783dfdd"
    },
    {
      "identifier": "10.1016/j.epsl.2011.11.037",
      "identifierType": "DOI",
      "entityType": "JournalArticle"
    }
  ]
}

Updated dataset contains:

"relationships": [
  {
    "identifier": "https://scilog.development.psi.ch/logbooks/6895bea625f055bca783dfdd",
    "identifierType": "URL",
    "relationship": "IsReferencedBy",
    "entityType": "Logbook",
    "externalId": "6895bea625f055bca783dfdd",
    "_id": "69d7b7e797c6654f48cab9df"
  },
  {
    "identifier": "10.1016/j.epsl.2011.11.037",
    "identifierType": "DOI",
    "relationship": "IsReferencedBy",
    "entityType": "JournalArticle",
    "_id": "69d7b7e797c6654f48cab9e0"
  }
]

A corresponding frontend feature (widget / tab on dataset detail page) will follow.

Motivation

Previously, RelationshipClass was dataset specific. However, there is a need to link to related entities outside of SciCat, e.g. a SciLog logbook.
So we add relatedEntityType (inspired by datacite's resourceTypeGeneral) . The existing relationship field is roughly relationType

Fixes

• NA

Changes:

• generalize description of pid and relationship fields from "dataset" to "entity"
• add additional fields to the schema as indicated above

Tests included

• Included for each change/fix?
• Passing?

Documentation

• swagger documentation updated (required for API changes)
• official documentation updated

official documentation info

[minottic]

LGTM, thanks!

I am wondering if it's relatedIdentifierType might be valuable? Apparently, e.g. for DOIs, that's useful later for datacite to count citations and might become valuable when we publish the dataset?

[omkar-ethz]

LGTM, thanks!

I am wondering if it's relatedIdentifierType might be valuable? Apparently, e.g. for DOIs, that's useful later for datacite to count citations and might become valuable when we publish the dataset?

Thanks for the review! I think this is a good idea.
Currently, the assumption was that relatedIdentifierType is always URL, so we had the url field in the model.

But we can indeed align it more closely with DataCite. So we can remove the url field, as @HayenNico suggests, rename pid to relatedIdentifer. This would contain the URL if relatedIdentifierType is URL, a DOI if relatedIdentifierType is doi and so on.

We could add an optional externalId field that stores the id of the external entity (e.g. logbookId), and have an index on both externalId and relatedIdentifier?

"relationships": [
  {
    "relationship": "IsReferencedBy",
    "relatedEntityType": "Logbook",
    "externalId": "6895bea625f055bca783dfdd",  // id of the related entity in the external system, opaque for SciCat
    "relatedIdentifier": "https://scilog.development.psi.ch/logbooks/6895bea625f055bca783dfdd",
    "relatedIdentifierType": "URL" 
  }
]

Question: As we are renaming the fields, do we need a migration script? Or we can do without, as it seems the class wasn't used before

[nitrosx]

It looks good to me.
Do we want to use the DataCite naming which is long?
Also does this cover if I want to express an arbitrary relationship between two different datasets within the same SciCat instance? If so, how would we do that? Do you already have something in mind?

[omkar-ethz]

It looks good to me. Do we want to use the DataCite naming which is long? Also does this cover if I want to express an arbitrary relationship between two different datasets within the same SciCat instance? If so, how would we do that? Do you already have something in mind?

Thanks a lot! Yes the DataCite names were quite verbose, I dropped the related- prefix from the field names.

For SciCat entities, I have proposed using identifierType: Local, and then if entityType is Dataset we can interpret the identifier as the PID of a SciCat dataset. Similarly we can use identifierType: Local, with entityType of Proposal / Instrument to link to other local entities.

So we could express a derived dataset relationship as:

{
    "identifier": "PID.SAMPLE.PREFIX/psi_ds3",
    "identifierType": "Local",
    "relationship": "IsDerivedFrom",
    "entityType": "Dataset"
}

I added this in the documentation of the identifierType field for now, but we can also enforce this in the validation in the future, or use this info in frontend to treat local identifiers specially if needed