Skip to content

chore: added missing endpoint, verifiedby and mv refresh job (CM-1030, CM-1038)#3907

Open
themarolt wants to merge 8 commits intomainfrom
feat/missing-endpoint-and-verifiedby-CM-1030-CM-1038
Open

chore: added missing endpoint, verifiedby and mv refresh job (CM-1030, CM-1038)#3907
themarolt wants to merge 8 commits intomainfrom
feat/missing-endpoint-and-verifiedby-CM-1030-CM-1038

Conversation

@themarolt
Copy link
Contributor

@themarolt themarolt commented Mar 10, 2026

Note

Medium Risk
Adds a new public API endpoint backed by new data-access queries and schema/index migrations, plus a new cron job that refreshes a materialized view; these changes can affect production DB performance and API responses. Risk is moderated by additive changes and use of CONCURRENTLY, but correctness depends on the new joins/refresh cadence and new verifiedBy field propagation.

Overview
Adds a new public endpoint GET /v1/members/:memberId/project-affiliations that returns per-project contribution counts, maintainer roles (including maintainerFile/repo file URL), and organization affiliations, preferring manual memberSegmentAffiliations overrides and falling back to work experiences.

Propagates verifiedBy throughout member identity/work-experience flows: public identity APIs now return verifiedBy, DAL bulk identity inserts persist verifiedBy, and multiple ingestion/enrichment pipelines set verifiedBy when creating verified identities.

Introduces DB support for the new affiliations flow by adding verified/verifiedBy columns to memberSegmentAffiliations, adding indexes for member lookups (including on mv_maintainer_roles), and adding a new cron_service job to REFRESH MATERIALIZED VIEW CONCURRENTLY for mv_maintainer_roles every 30 minutes with Slack alerts on timeouts.

Written by Cursor Bugbot for commit 51f4d3c. This will update automatically on new commits. Configure here.

Signed-off-by: Uroš Marolt <uros@marolt.me>
@github-actions
Copy link
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

1 similar comment
@github-actions
Copy link
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@themarolt themarolt changed the title chore: added missing endpoint, verifiedby and mv refresh job chore: added missing endpoint, verifiedby and mv refresh job (CM-1030, CM-1038) Mar 10, 2026
Signed-off-by: Uroš Marolt <uros@marolt.me>
@themarolt themarolt requested a review from joanagmaia March 10, 2026 13:29
organizationLogo: a.organizationLogo ?? null,
verified: a.verified,
verifiedBy: a.verifiedBy ?? null,
source: 'ui',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for the source here being hardcoded?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a source column in the db for memberSegmentAffiliations table. And since the only time this table is updated is through UI I just set it to ui for now but yea let me know what to set here instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean, didn't realise we didn't have it. I would say we can remove the source if it's hardcoded. I'll also update the specs document to remove it from the response.

Signed-off-by: Uroš Marolt <uros@marolt.me>
Copilot AI review requested due to automatic review settings March 10, 2026 19:36
@CLAassistant
Copy link

CLAassistant commented Mar 10, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for project-affiliation retrieval in the public members API, while standardizing verifiedBy across multiple ingestion/enrichment paths and improving query performance via indexes + a materialized-view refresh job.

Changes:

  • Add public endpoint GET /:memberId/project-affiliations that returns per-project contribution segments with maintainer roles and affiliations (manual overrides with work-experience fallback).
  • Extend integration/enrichment identity payloads and public API outputs to include verifiedBy (normalized to null when absent).
  • Add DB migration for memberSegmentAffiliations.verified/verifiedBy, missing indexes for project-affiliation lookups, and a cron job to refresh mv_maintainer_roles every 30 minutes with Slack timeout alerts.

Reviewed changes

Copilot reviewed 25 out of 27 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
services/libs/integrations/src/integrations/twitter/processData.ts Include verifiedBy on created Twitter identities.
services/libs/integrations/src/integrations/stackoverflow/processData.ts Include verifiedBy on created StackOverflow identities.
services/libs/integrations/src/integrations/slack/processData.ts Include verifiedBy on created Slack identities (username/email).
services/libs/integrations/src/integrations/reddit/processData.ts Include verifiedBy on created Reddit identities.
services/libs/integrations/src/integrations/linkedin/processData.ts Include verifiedBy on created LinkedIn identities.
services/libs/integrations/src/integrations/hackernews/processData.ts Include verifiedBy on created Hacker News identities.
services/libs/integrations/src/integrations/groupsio/processData.ts Include verifiedBy on created Groups.io identities (username/email).
services/libs/integrations/src/integrations/gitlab/processData.ts Include verifiedBy on created GitLab identities (username/email/commit author).
services/libs/integrations/src/integrations/github/processData.ts Include verifiedBy on created GitHub identities (members/orgs/email).
services/libs/integrations/src/integrations/discourse/processData.ts Include verifiedBy on created Discourse identities (username/email).
services/libs/integrations/src/integrations/discord/processData.ts Include verifiedBy on created Discord identities.
services/libs/integrations/src/integrations/devto/processData.ts Include verifiedBy on created Dev.to identities.
services/libs/data-access-layer/src/members/projectAffiliations.ts New DAL queries for project segments, manual segment affiliations, and work-experience affiliations.
services/libs/data-access-layer/src/members/index.ts Export new projectAffiliations DAL module.
services/apps/snowflake_connectors/src/integrations/cvent/event-registrations/transformer.ts Include verifiedBy for Cvent-derived identities.
services/apps/members_enrichment_worker/src/workflows/lf-auth0/enrichMemberWithLFAuth0.ts Include verifiedBy for LF Auth0-enriched identities.
services/apps/cron_service/src/main.ts Add Slack alert on cron job timeout failures.
services/apps/cron_service/src/jobs/refreshMvs.job.ts New cron job to refresh materialized views (currently mv_maintainer_roles).
backend/src/utils/mapper.ts Include verifiedBy in work-experience mapping.
backend/src/database/migrations/V1773139177__add-verified-to-member-segment-affiliations.sql Add verified/verifiedBy columns to memberSegmentAffiliations.
backend/src/database/migrations/V1772799041__add-missing-indexes-for-project-affiliations.sql Add missing indexes for memberId lookups (incl. on mv_maintainer_roles).
backend/src/database/migrations/U1773139177__add-verified-to-member-segment-affiliations.sql Undo migration placeholder.
backend/src/database/migrations/U1772799041__add-missing-indexes-for-project-affiliations.sql Undo migration placeholder.
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts New public API handler implementing project-affiliation response assembly.
backend/src/api/public/v1/members/index.ts Register new /:memberId/project-affiliations route + scope.
backend/src/api/public/v1/members/identities/verifyMemberIdentity.ts Return verifiedBy in verify-identity response.
backend/src/api/public/v1/members/identities/getMemberIdentities.ts Return verifiedBy in get-identities response.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Copy link
Contributor

@joanagmaia joanagmaia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just small comment to remove source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants