feat: implement LIST indexing by ITEM for enhanced query capabilities

[robfrank]

This pull request adds support for indexing list-type properties in documents by individual items ("BY ITEM") in ArcadeDB. The changes touch grammar, parser, indexing, and query planning code to enable creation, update, deletion, and querying of indexes on list elements. This allows more flexible and granular indexing and searching within list properties.

Indexing and Querying Enhancements:

• Added support for the BY ITEM modifier in SQL grammar, parser, and index creation, allowing indexes to be defined on individual elements of list properties. (engine/src/main/grammar/SQLGrammar.jjt, engine/src/main/java/com/arcadedb/query/sql/parser/CreateIndexStatement.java) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
• Updated the DocumentIndexer to handle creation, update, and deletion of index entries for each list item when the BY ITEM modifier is present, including logic to compute changes (delta) during updates. (engine/src/main/java/com/arcadedb/database/DocumentIndexer.java) [1] [2] [3] [4]

Query Planning and Execution:

• Extended query planning to recognize and utilize indexes defined with the BY ITEM modifier, including stripping modifiers from field names and checking for item-based indexing. (engine/src/main/java/com/arcadedb/query/sql/executor/SelectExecutionPlanner.java) [1] [2]
• Enhanced IndexSearchInfo to track and expose whether an index is defined with the BY ITEM modifier for proper query execution. (engine/src/main/java/com/arcadedb/query/sql/executor/IndexSearchInfo.java) [1] [2]

General Code Maintenance:

• Refactored imports for clarity and consistency across affected files. (engine/src/main/java/com/arcadedb/database/DocumentIndexer.java, engine/src/main/java/com/arcadedb/query/sql/parser/CreateIndexStatement.java) [1] [2]

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ +0.07%	✅ 85.94%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (0c1618a)	72884	46134	63.30%
Head commit (47cb558)	73027 (+143)	46277 (+143)	63.37% (+0.07%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#2620)	1736	1492	85.94%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

[Copilot]

Pull Request Overview

This pull request implements support for indexing list-type properties by individual items using the "BY ITEM" modifier in ArcadeDB. This enhancement allows creating indexes on individual elements within list properties rather than treating the entire list as a single indexed value, enabling more granular querying capabilities.

• Added SQL grammar support for the "BY ITEM" modifier in CREATE INDEX statements
• Implemented indexing logic to create separate index entries for each list element
• Enhanced query planning to recognize and utilize "BY ITEM" indexes for optimized queries

Reviewed Changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
engine/src/main/grammar/SQLGrammar.jjt	Added ITEM token and grammar rules for "BY ITEM" modifier
engine/src/main/java/com/arcadedb/query/sql/parser/CreateIndexStatement.java	Enhanced parser to handle "BY ITEM" property in index creation statements
engine/src/main/java/com/arcadedb/database/DocumentIndexer.java	Implemented core indexing logic for list items including create, update, and delete operations
engine/src/main/java/com/arcadedb/query/sql/executor/SelectExecutionPlanner.java	Updated query planner to recognize "BY ITEM" indexes and strip modifiers
engine/src/main/java/com/arcadedb/query/sql/executor/IndexSearchInfo.java	Added tracking for "BY ITEM" modifier in index search information
engine/src/main/java/com/arcadedb/schema/TypeIndexBuilder.java	Added validation and type handling for "BY ITEM" indexes
engine/src/test/java/com/arcadedb/index/ListIndexByItemTest.java	Comprehensive test suite covering all aspects of list item indexing
engine/src/test/java/com/arcadedb/query/sql/parser/CreateIndexStatementTestParserTest.java	Added parser tests for "BY ITEM" syntax
engine/src/test/java/com/arcadedb/query/sql/function/sql/SQLFunctionsTest.java	Fixed flaky date test by using fixed timestamps
engine/src/test/java/com/arcadedb/event/RecordEncryptionTest.java	Fixed counter increment timing in encryption event handlers

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[DMCTowns]

@robfrank - quick question on this work. From the test case it looks like you can index a list of strings in a document. Will it also work for a list of objects? E.g.

CREATE VERTEX TYPE Photo;
CREATE PROPERTY Photo.id INTEGER;
CREATE DOCUMENT TYPE Tag;
CREATE PROPERTY Tag.id INTEGER;
CREATE PROPERTY Tag.name STRING;
CREATE PROPERTY Photo.tags LIST OF Tag;
CREATE INDEX ON Photo (id) UNIQUE;
CREATE INDEX ON Photo (tags.id BY ITEM) NOTUNIQUE;

[gramian]

AFAIK this will not work. You could index tags BY ITEM but this means the object map as a whole become an index entry.