chore: added missing endpoint, verifiedby and mv refresh job (CM-1030, CM-1038)#3907
chore: added missing endpoint, verifiedby and mv refresh job (CM-1030, CM-1038)#3907
Conversation
Signed-off-by: Uroš Marolt <uros@marolt.me>
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
1 similar comment
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Outdated
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>
backend/src/database/migrations/V1773139177__add-verified-to-member-segment-affiliations.sql
Outdated
Show resolved
Hide resolved
| organizationLogo: a.organizationLogo ?? null, | ||
| verified: a.verified, | ||
| verifiedBy: a.verifiedBy ?? null, | ||
| source: 'ui', |
There was a problem hiding this comment.
Any reason for the source here being hardcoded?
There was a problem hiding this comment.
We don't have a source column in the db for memberSegmentAffiliations table. And since the only time this table is updated is through UI I just set it to ui for now but yea let me know what to set here instead?
There was a problem hiding this comment.
I see what you mean, didn't realise we didn't have it. I would say we can remove the source if it's hardcoded. I'll also update the specs document to remove it from the response.
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Outdated
Show resolved
Hide resolved
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Outdated
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
Adds support for project-affiliation retrieval in the public members API, while standardizing verifiedBy across multiple ingestion/enrichment paths and improving query performance via indexes + a materialized-view refresh job.
Changes:
- Add public endpoint
GET /:memberId/project-affiliationsthat returns per-project contribution segments with maintainer roles and affiliations (manual overrides with work-experience fallback). - Extend integration/enrichment identity payloads and public API outputs to include
verifiedBy(normalized tonullwhen absent). - Add DB migration for
memberSegmentAffiliations.verified/verifiedBy, missing indexes for project-affiliation lookups, and a cron job to refreshmv_maintainer_rolesevery 30 minutes with Slack timeout alerts.
Reviewed changes
Copilot reviewed 25 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| services/libs/integrations/src/integrations/twitter/processData.ts | Include verifiedBy on created Twitter identities. |
| services/libs/integrations/src/integrations/stackoverflow/processData.ts | Include verifiedBy on created StackOverflow identities. |
| services/libs/integrations/src/integrations/slack/processData.ts | Include verifiedBy on created Slack identities (username/email). |
| services/libs/integrations/src/integrations/reddit/processData.ts | Include verifiedBy on created Reddit identities. |
| services/libs/integrations/src/integrations/linkedin/processData.ts | Include verifiedBy on created LinkedIn identities. |
| services/libs/integrations/src/integrations/hackernews/processData.ts | Include verifiedBy on created Hacker News identities. |
| services/libs/integrations/src/integrations/groupsio/processData.ts | Include verifiedBy on created Groups.io identities (username/email). |
| services/libs/integrations/src/integrations/gitlab/processData.ts | Include verifiedBy on created GitLab identities (username/email/commit author). |
| services/libs/integrations/src/integrations/github/processData.ts | Include verifiedBy on created GitHub identities (members/orgs/email). |
| services/libs/integrations/src/integrations/discourse/processData.ts | Include verifiedBy on created Discourse identities (username/email). |
| services/libs/integrations/src/integrations/discord/processData.ts | Include verifiedBy on created Discord identities. |
| services/libs/integrations/src/integrations/devto/processData.ts | Include verifiedBy on created Dev.to identities. |
| services/libs/data-access-layer/src/members/projectAffiliations.ts | New DAL queries for project segments, manual segment affiliations, and work-experience affiliations. |
| services/libs/data-access-layer/src/members/index.ts | Export new projectAffiliations DAL module. |
| services/apps/snowflake_connectors/src/integrations/cvent/event-registrations/transformer.ts | Include verifiedBy for Cvent-derived identities. |
| services/apps/members_enrichment_worker/src/workflows/lf-auth0/enrichMemberWithLFAuth0.ts | Include verifiedBy for LF Auth0-enriched identities. |
| services/apps/cron_service/src/main.ts | Add Slack alert on cron job timeout failures. |
| services/apps/cron_service/src/jobs/refreshMvs.job.ts | New cron job to refresh materialized views (currently mv_maintainer_roles). |
| backend/src/utils/mapper.ts | Include verifiedBy in work-experience mapping. |
| backend/src/database/migrations/V1773139177__add-verified-to-member-segment-affiliations.sql | Add verified/verifiedBy columns to memberSegmentAffiliations. |
| backend/src/database/migrations/V1772799041__add-missing-indexes-for-project-affiliations.sql | Add missing indexes for memberId lookups (incl. on mv_maintainer_roles). |
| backend/src/database/migrations/U1773139177__add-verified-to-member-segment-affiliations.sql | Undo migration placeholder. |
| backend/src/database/migrations/U1772799041__add-missing-indexes-for-project-affiliations.sql | Undo migration placeholder. |
| backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts | New public API handler implementing project-affiliation response assembly. |
| backend/src/api/public/v1/members/index.ts | Register new /:memberId/project-affiliations route + scope. |
| backend/src/api/public/v1/members/identities/verifyMemberIdentity.ts | Return verifiedBy in verify-identity response. |
| backend/src/api/public/v1/members/identities/getMemberIdentities.ts | Return verifiedBy in get-identities response. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
backend/src/api/public/v1/members/project-affiliations/getProjectAffiliations.ts
Show resolved
Hide resolved
joanagmaia
left a comment
There was a problem hiding this comment.
LGTM, just small comment to remove source
Note
Medium Risk
Adds a new public API endpoint backed by new data-access queries and schema/index migrations, plus a new cron job that refreshes a materialized view; these changes can affect production DB performance and API responses. Risk is moderated by additive changes and use of
CONCURRENTLY, but correctness depends on the new joins/refresh cadence and newverifiedByfield propagation.Overview
Adds a new public endpoint
GET /v1/members/:memberId/project-affiliationsthat returns per-project contribution counts, maintainer roles (includingmaintainerFile/repo file URL), and organization affiliations, preferring manualmemberSegmentAffiliationsoverrides and falling back to work experiences.Propagates
verifiedBythroughout member identity/work-experience flows: public identity APIs now returnverifiedBy, DAL bulk identity inserts persistverifiedBy, and multiple ingestion/enrichment pipelines setverifiedBywhen creating verified identities.Introduces DB support for the new affiliations flow by adding
verified/verifiedBycolumns tomemberSegmentAffiliations, adding indexes for member lookups (including onmv_maintainer_roles), and adding a newcron_servicejob toREFRESH MATERIALIZED VIEW CONCURRENTLYformv_maintainer_rolesevery 30 minutes with Slack alerts on timeouts.Written by Cursor Bugbot for commit 51f4d3c. This will update automatically on new commits. Configure here.