vera-pissa method added by TranscenderNing · Pull Request #8722 · PaddlePaddle/PaddleNLP

TranscenderNing · 2024-07-05T13:19:36Z

PR types

New features

PR changes

Add vera-pissa in peft/vera

Description

根据review意见修改

paddle-bot · 2024-07-05T13:19:41Z

Thanks for your contribution!

CLAassistant · 2024-07-05T13:19:41Z

All committers have signed the CLA.

codecov · 2024-07-06T02:42:58Z

Codecov Report

Attention: Patch coverage is 80.70740% with 60 lines in your changes missing coverage. Please review.

Project coverage is 55.51%. Comparing base (d8ddba9) to head (d4810c1).
Report is 246 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/peft/vera/vera_model.py	77.34%	41 Missing ⚠️
paddlenlp/peft/vera/vera_layers.py	77.94%	15 Missing ⚠️
paddlenlp/trainer/trainer.py	50.00%	3 Missing ⚠️
paddlenlp/trainer/integrations.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8722      +/-   ##
===========================================
- Coverage    55.73%   55.51%   -0.22%     
===========================================
  Files          623      630       +7     
  Lines        97464    98374     +910     
===========================================
+ Hits         54324    54616     +292     
- Misses       43140    43758     +618

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lugimzzz · 2024-07-19T04:15:22Z

@@ -0,0 +1,187 @@
+out_features = 16  # Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


删掉out_features = 16

lugimzzz · 2024-07-19T06:27:11Z

+            isinstance(self.model, LoRAModel)
+            or isinstance(self.model, PrefixModelForCausalLM)
+            or isinstance(self.model, VeRAModel)
+        ):


测试一下VeRAModel 重新加载和热启的时候能否正常使用

重新加载就是训练的时候设置 load_best_model_at_end 为 True，看时候能够正常加载最好的checkpoint

热启指的是训练过程中，output_dir中包含原有训练checkpoint，trainer可以启用resume_from_checkpoint去加载到最后一个checkpoint继续训练

测试可以重新加载 done

适配了热启动，测试可以 done

lugimzzz · 2024-07-19T06:30:18Z

+            self.model = self.get_vera_model(model, vera_config)
+        self.is_pipelinemodel = False
+        if issubclass(type(self.model), PipelineLayer):
+            self.is_pipelinemodel = True


目前vera也不支持pp，建议raise NotImplementedError("vera don't support pipeline parallel now")

lugimzzz · 2024-07-19T06:30:57Z

+        vera_model = cls(model, vera_config)
+
+        # define vera weight name
+        if vera_config_tensor_parallel_degree > 1:


目前不支持vera都可以先删除tensor_parallel_degree相关的分支

lugimzzz · 2024-07-19T06:31:52Z

+        trainable_state_dict = OrderedDict()
+        for name, weight in self.model.state_dict().items():
+            # get vera parameter & QAT scale parameter
+            if not weight.stop_gradient or "activation_quanter" in name or "weight_quanter" in name:


不支持quant相关，建议也把quant相关删除

lugimzzz · 2024-07-19T06:34:29Z

+                        # freezeB=False, vera_b, vera_d 可训练
+                        if "vera" in name:
+                            weight.stop_gradient = False
+                        elif "lora_B" in name and notfreezeB:


什么情况会出现weight中含有lora？

之前vera_model中参数名是lora_ 已经全部统一成vera_
done

lugimzzz · 2024-07-19T06:36:47Z

+
+    def train(self):
+        super().train()
+        if self.merge_weights and self.merged:


merge_weight已经删除，新增为一个merge函数不再与train和eval耦合，可以参考这个prhttps://github.com//pull/8674/files

lugimzzz · 2024-07-19T06:38:54Z

+
+        else:
+            # Actual trainable parameters
+            self.lora_A = self.create_parameter(


不要叫lora，改为vera_A和vera_B

lugimzzz · 2024-07-19T06:40:38Z

@@ -0,0 +1,104 @@
+# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.


是否验证过merge后的模型正确性？

验证过，用merge后的模型可以正确预测。 done

lugimzzz · 2024-07-19T06:41:20Z

+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.


模仿lora写一个test vera https://github.com/PaddlePaddle/PaddleNLP/blob/develop/tests/llm/test_lora.py

lugimzzz · 2024-07-22T07:13:40Z

+            "For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' "
+        },
+    )
+    vera_alpha: int = field(default=8, metadata={"help": "Lora alpha"})


lora记得改成vera

最好整个pr扫一遍

lugimzzz · 2024-07-22T07:15:47Z

+        r: int = 0,
+        vera_alpha: int = 1,
+        vera_dropout: float = 0.0,
+        merge_weights: bool = True,


去掉merge_weights

lugimzzz · 2024-07-22T07:17:04Z

+        if enable_vera is None:
+            if isinstance(module, nn.Linear):
+                vera_module = VeRALinear(
+                    # 将要替换的层传递过去


注释用英文，不要中文

lugimzzz · 2024-07-22T07:20:30Z

+                isinstance(vera_config.enable_vera_list, List)
+                and all(isinstance(item, bool) for item in vera_config.enable_vera_list)
+            ):
+                enable_vera_list = [vera_config.enable_vera_list]


enable_vera_list 这个应该是直接复用lora的，vera并没有对应的功能，建议把enable_vera_list相关全部删除，走代码里为None的分支就好

应该是在vera_config层面就把enable_vera_list全部删除,因为我们不需要这个参数，我看现在代码还保留着？

lugimzzz · 2024-07-22T07:25:33Z

        self.run_predictor({"inference_model": False})
-
-
-# @parameterized_class(


这块不要删除

lugimzzz · 2024-07-22T07:25:51Z

+        ["baichuan"],
+    ],
+)
+class VeraTest(LLMTest, unittest.TestCase):


本地测试一下是否能正常运行

cd PaddleNLP
python -m pytest tests/llm/test_vera.py

可以正常运行 done

lugimzzz · 2024-07-22T10:59:15Z

+    ) and args.device == "cpu":
+        raise ValueError("We can not apply bfloat16 or nf4/fp4 vera merge on cpu.")
+
+    vera_config.merge_weights = False


vera_config.merge_weights没有merge weight了，记得去掉，否则会报错

lugimzzz · 2024-07-22T11:01:34Z

单测覆盖率要增加不足要增加具体看detail提示

lugimzzz · 2024-07-23T03:12:23Z

        self.merged = False

        if pissa_init:
-            assert self.vera_alpha == self.r, "pissa method requires vera_alpha=r, scaling=1"


为什么把这个删除了

为了增加代码的覆盖率，重新加回去了并添加相应的异常测试

lugimzzz · 2024-07-23T03:13:31Z

+                isinstance(vera_config.enable_vera_list, List)
+                and all(isinstance(item, bool) for item in vera_config.enable_vera_list)
+            ):
+                enable_vera_list = [vera_config.enable_vera_list]


应该是在vera_config层面就把enable_vera_list全部删除,因为我们不需要这个参数，我看现在代码还保留着？

lugimzzz · 2024-07-23T03:13:50Z

@@ -0,0 +1,15 @@
+{
+  "base_model_name_or_path": null,


这个文件的作用是什么?

测试用的，已删除，done

已把vera_config层就把enable_vera_list全部删除

lugimzzz

lgtm

vera-pissa method added

1d51f1f

paddle-bot Bot added the contributor label Jul 5, 2024

paddle-bot Bot assigned ZHUI Jul 5, 2024

TranscenderNing added 3 commits July 5, 2024 21:40

add vera-pissa

aa95de7

Add vera-pissa and test

d0f9689

Add vera-pissa and tests and correct the lint format

92f8773

lugimzzz reviewed Jul 19, 2024

View reviewed changes

TranscenderNing added 2 commits July 20, 2024 16:41

Revise according to the review comments and add tests

277e388

Revise according to the review comments and pass tests

1baf39a

lugimzzz reviewed Jul 22, 2024

View reviewed changes

Revise according to the review comments and pass tests 1

0aef75a

lugimzzz reviewed Jul 22, 2024

View reviewed changes

Revise according to the review comments and pass tests 2

6c6c708

lugimzzz reviewed Jul 23, 2024

View reviewed changes

TranscenderNing added 3 commits July 23, 2024 15:29

Revise according to the review comments and pass tests 3

dd86a6c

Revise according to the review comments and pass tests 31

ddc939a

Revise according to the review comments and pass tests 32

d4810c1

lugimzzz approved these changes Jul 23, 2024

View reviewed changes

lugimzzz merged commit de7d103 into PaddlePaddle:develop Jul 23, 2024

lugimzzz added the Beijing Innovation Consortium label Jul 29, 2024

lugimzzz mentioned this pull request Oct 17, 2024

Adding LoKrModel Class to paddle.peft library #9269

Merged

		@@ -0,0 +1,187 @@
		out_features = 16 # Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

		@@ -0,0 +1,104 @@
		# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.

		self.run_predictor({"inference_model": False})


		# @parameterized_class(

Conversation

TranscenderNing commented Jul 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Description

Uh oh!

paddle-bot Bot commented Jul 5, 2024

Uh oh!

CLAassistant commented Jul 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TranscenderNing commented Jul 5, 2024 •

edited

Loading

CLAassistant commented Jul 5, 2024 •

edited

Loading

codecov Bot commented Jul 6, 2024 •

edited

Loading