[ChatTemplate] Fix chat template for Gemma when answer is contained within question.

[lvdongyi]

PR types

Bug fixes

PR changes

Others

Description

Fix chat template for Gemma when answer is contained within question:

1. generate all split_positions.
2. filter split_positions according to <start_of_turn>. (The last <start_of_turn> before split_position must be immediately followed by model. )
3. generate non_learnable_parts according to filtered_positions.

This PR is related to #9444

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 53.03%. Comparing base (79bba4f) to head (ecc3177).
Report is 228 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/gemma/tokenizer.py	91.66%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #9462      +/-   ##
===========================================
+ Coverage    52.70%   53.03%   +0.32%     
===========================================
  Files          677      679       +2     
  Lines       109391   109626     +235     
===========================================
+ Hits         57658    58137     +479     
+ Misses       51733    51489     -244     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[DrownFish19]

逻辑没有问题，不过这种方式和chat_template依赖太大。后续Todo设计通用方法，适配多种chat_template。