Skip to content

feat: Implement %rowIndex environment variable#2573

Open
piotrszul wants to merge 6 commits intorelease/9.6.0from
issue/2560
Open

feat: Implement %rowIndex environment variable#2573
piotrszul wants to merge 6 commits intorelease/9.6.0from
issue/2560

Conversation

@piotrszul
Copy link
Copy Markdown
Collaborator

@piotrszul piotrszul commented Mar 18, 2026

Summary

Implement %rowIndex environment variable for ViewDefinition iterations, as defined in the SQL on FHIR ViewDefinition spec.

%rowIndex provides the 0-based index of the current element within the iterated collection. At the top level (no iteration), it evaluates to 0. Each nesting level maintains independent values.

forEach / forEachOrNull support

Uses Spark's indexed transform(array, (element, index) -> ...) to track element positions during unnesting, threading the index through ProjectionContext into FHIRPath evaluation as a supplied variable.

File Change
ColumnRepresentation Added transformWithIndex() using Spark's indexed transform
SingleResourceEvaluator Added withVariable() for creating evaluators with additional variables
ProjectionContext Added rowIndex field, withRowIndex(), and %rowIndex injection in evalExpression()
UnnestingSelection Rewrote evaluate() to use indexed transform, passing element index as %rowIndex

repeat support

Implements %rowIndex for repeat clauses using a RowIndexCounter that assigns monotonically increasing indices across breadth-first iteration levels, resetting per resource. This required new encoder-level support for injecting index values during repeat expansion.

File Change
RowIndexCounter New thread-safe counter that assigns sequential indices across repeat levels, resetting per resource
ValueFunctions New withRowIndex() UDF that pairs each element with its assigned index
Expressions.scala New repeatWithRowIndex() that threads RowIndexCounter through the repeat expansion loop
RepeatSelection Updated to use repeatWithRowIndex() and pass index into ProjectionContext
ExpressionsBothModesTest Unit tests for repeatWithRowIndex() covering linear chains, branching trees, and counter reset

Test coverage (15 test cases in viewTests/rowindex.json)

forEach / forEachOrNull (8 tests):

  • 3 spec examples: element position, nested iteration with independent indices, unionAll with inherited/independent indices
  • Top-level %rowIndex defaults to 0
  • forEach and forEachOrNull with %rowIndex
  • Nested forEach with independent %rowIndex values
  • forEachOrNull with empty collection (null row)
  • Arithmetic expression (%rowIndex + 1)

repeat (7 tests):

  • Linear chain with sequential indices
  • Arithmetic expression with repeat %rowIndex
  • Repeat with nested forEach (independent indices)
  • Branching tree with breadth-first ordering
  • Counter resets per resource
  • Repeat nested inside repeat (independent indices)
  • Repeat nested inside forEach (independent indices)

Test plan

  • All 15 new %rowIndex tests pass via FhirViewExtraTest
  • All 18 existing view tests pass (no regressions)
  • All existing FhirViewExecutorTest + FhirViewBuilderUnitTest tests pass
  • ExpressionsBothModesTest unit tests pass for repeat index logic
  • Pre-existing compliance test failures unchanged (2 failures unrelated to this change)
  • SonarCloud thread-safety warning addressed with @SuppressWarnings annotation

Closes #2560

🤖 Generated with Claude Code

@piotrszul piotrszul added new feature New feature or request fhirpath Related to fhirpath reference implementation labels Mar 18, 2026
@piotrszul piotrszul moved this to In progress in Pathling Mar 18, 2026
@johngrimes
Copy link
Copy Markdown
Member

johngrimes commented Mar 30, 2026

  • Integrate variant/custom schema unification approach to repeat/repeatAll functions
  • Investigate: change tree-traversing expression to keep track of index

…Each/forEachOrNull

Add support for the %rowIndex environment variable as defined in the
SQL on FHIR ViewDefinition spec. Within forEach and forEachOrNull
iterations, %rowIndex resolves to the 0-based index of the current
element. At the top level (no iteration), it evaluates to 0. Each
nesting level maintains independent %rowIndex values.

The implementation uses Spark's indexed transform(array, (elem, idx) ->)
to track element positions during unnesting, threading the index through
ProjectionContext into the FHIRPath evaluation as a supplied variable.

Closes #2560

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@piotrszul piotrszul changed the base branch from main to release/9.6.0 March 31, 2026 02:09
piotrszul and others added 5 commits April 1, 2026 14:25
…eat clause

Adds support for %rowIndex within the repeat directive, producing a global
0-based traversal-order index across all depth levels of the flattened
recursive tree. Each repeat directive scopes its own counter independently
from enclosing or nested forEach/forEachOrNull/repeat directives.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rning

Replace lazy ThreadLocal initialization with eager init and readObject()
to eliminate a race condition when the instance is shared across threads
via Spark's addReferenceObj(). Suppress S5164 (ThreadLocal.remove()) with
documentation explaining why removal is unnecessary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nt values

Split RowCounter into separate read (RowCounterGet) and increment
(RowCounterIncrement) operations so that multiple references to %rowIndex
within the same repeat element all read the same value. Previously each
reference independently called getAndIncrement(), causing N references to
consume N counter values per element instead of one.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RowCounter (getAndIncrement per evaluation) is superseded by the
RowCounterGet + RowCounterIncrement split. Remove the old expression,
its ValueFunctions helper, the getAndIncrement method, and the four
encoder-level tests that exercised it. The equivalent behavior is now
tested via ViewDefinition-level tests in rowindex.json.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cover the 13 new lines flagged by SonarCloud as uncovered: all methods
on RowIndexCounter (get, increment, reset, serialization) and the three
ValueFunctions entry points (rowCounterGet, rowCounterIncrement,
resetCounter) exercised via a Spark dataset test in both codegen and
interpreted modes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 8, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fhirpath Related to fhirpath reference implementation new feature New feature or request

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

Implement %rowIndex environment variable

2 participants