fix(tmq): race condition in tmq snapshot for data transformation

[wangmm0220]

Description

Please briefly describe the code changes in this pull request.

Jira: https://jira.taosdata.com:18080/browse/TD-

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Summary of Changes

Hello @wangmm0220, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical race condition within the TMQ snapshot mechanism during data transformation. The changes primarily focus on enhancing thread safety by introducing metadata read locks, standardizing error handling with robust resource cleanup, and removing a function (tqReaderSetVtableInfo) that was likely contributing to the instability or was no longer required in the new, more robust design. These modifications aim to improve the reliability and stability of TMQ snapshot operations.

Highlights

• Race Condition Fix: Implemented read locks (metaRLock/metaULock) for metadata operations during TMQ snapshot creation and retrieval to prevent race conditions, ensuring data consistency in concurrent environments.
• Enhanced Error Handling and Resource Management: Standardized error checking and resource cleanup across several functions using TSDB_CHECK_NULL, TSDB_CHECK_CODE, and TSDB_CHECK_CONDITION macros, coupled with goto END blocks. This ensures consistent resource deallocation (e.g., tDecoderClear, tdbTbcClose, taosArrayDestroy) on all execution paths, improving robustness and preventing memory leaks.
• Removed Obsolete Virtual Table Functionality: Eliminated the tqReaderSetVtableInfo function, its declaration, and its function pointer from the SStoreTqReader struct, along with the createStreamVtableBlock function. This streamlines the TMQ snapshot process by removing a component that was likely either problematic or no longer necessary.
• Clarified Resource Ownership: Explicitly transferred ownership of the snapshot context (sContext) in createTmqRawScanOperatorInfo by setting the original handle's context to NULL, preventing potential double-free or use-after-free issues.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request primarily addresses a race condition in the TMQ snapshot functionality by introducing locking around the creation and use of the snapshot context. The changes also include significant refactoring to standardize error handling with macros, which improves code consistency and readability. Additionally, there's some good code cleanup, such as removing an unused function and clarifying resource ownership transfer. My review includes a couple of points: one regarding a non-portable header include that should be changed, and a more critical issue about a potential memory leak caused by a variable being re-declared in a loop. Overall, the core logic for the race condition fix appears sound.