support tensor index. #34824

hbwx24 · 2021-08-11T12:16:28Z

PR types

Function optimization

PR changes

APIs

Describe

1.整体支持：

2.具体功能点支持：

3. 支持 tensor类型索引，示例如下：

array = np.arange(4*3*2).reshape([4, 3, 2])
value = np.arange(12*3).reshape([3, 2, 3, 2])
index = [[0, 0], [3, 1]]

index_t = paddle.to_tensor(index)
index_np = np.array(index)
tt = paddle.to_tensor(array)

plist = paddle.index_select(tt, index_t, axis=0)

nplist = array[index_np]

print(np.array_equal(plist.numpy(), nplist))

paddle-bot-old · 2021-08-11T12:16:35Z

✅ This PR's description meets the template requirements!
Please wait for other CI results.

paddle-bot-old · 2021-08-11T12:16:36Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

chenwhql · 2021-08-18T11:58:20Z

paddle/fluid/operators/index_select_op.cc

+
+    std::vector<int64_t> output_dim(input_dim.size() + index_dim.size() - 1);
+
+    for (int i = 0; i < static_cast<int>(output_dim.size()); i++) {


use size_t i = 0 directly?

revert the modification of index_select_op.

LGTM，动态图现在复用太多Python逻辑，后续需要解决由此引入的性能问题

好的

chenwhql · 2021-08-19T02:15:13Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

+
+            if isinstance(item, np.ndarray):
+                return True
+            if not isinstance(item, (tuple, list)):


do we need to support set?

revert the modification of index_select_op.

chenwhql · 2021-08-19T02:17:49Z

python/paddle/fluid/tests/unittests/test_index_select_op.py

+        y = y * y
+        loss = y.sum()
+        loss.backward()
+        grad_torch = np.array([[[0., 2.], [4., 6.], [8., 10.]],


rename this var

revert the modification of index_select_op.

chenwhql · 2021-08-19T02:19:48Z

python/paddle/fluid/variable_index.py


    # Remove Variable to skip bug when counting Ellipsis
-    item_remove_var = [ele for ele in item if not isinstance(ele, Variable)]
+    item_remove_var = [


why need this skip

item_remove_var.count(Ellipsis)计算包含Ellipsis的个数，如果对象为Variable或者ndarray，count函数将报错。

chenwhql · 2021-08-19T02:28:07Z

paddle/fluid/operators/index_select_op.cc

            "to be in range of [-%d, %d]. But received Attr(dim) = %d.",
            input_dim.size(), input_dim.size() - 1, dim));

-    PADDLE_ENFORCE_EQ(


放开这个口子的话，我们是否需要一些别的检查？index_dim会不会有不合理的输入？

revert the modification of index_select_op.

chenwhql · 2021-08-19T02:29:31Z

python/paddle/fluid/variable_index.py

+
+def index_tensor(tensor, offsets, strides):
+    from . import layers
+    from .framework import Variable


why not import in beginning?

改为通过paddle.xx引用。

chenwhql · 2021-08-19T02:30:29Z

python/paddle/fluid/variable_index.py

+
+def getitem_list_index(var, list_index):
+    from . import layers
+    from .framework import Variable


chenwhql · 2021-08-19T02:30:44Z

python/paddle/fluid/variable_index.py

+
+
+def setitem_list_index(var, index_list, value):
+    from . import layers


chenwhql · 2021-08-19T02:34:32Z

python/paddle/fluid/variable_index.py

            for i in slice_item:
-                if not isinstance(i, (int, bool)):
-                    raise TypeError("Only support int or bool in index list.")
+                if not isinstance(i, (int, bool, list)):


Isn't it a tuple here?

chenwhql · 2021-08-25T09:08:20Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

+            return False
+
+        if contain_tensor(item):
+            # 1. Call _getitem_impl_ when item contains tensor.


chenwhql · 2021-08-25T09:13:34Z

python/paddle/fluid/variable_index.py

+
+    def update(self, index):
+        if is_list_tuple(index, int) or isinstance(
+                index, (paddle.fluid.core.VarBase, paddle.fluid.Variable,


only keep Variable is ok, when in dygraph mode, VarBase is Variable

chenwhql · 2021-08-25T09:13:40Z

python/paddle/fluid/variable_index.py

+        if is_list_tuple(index, int) or isinstance(
+                index, (paddle.fluid.core.VarBase, paddle.fluid.Variable,
+                        np.ndarray)):  # Tensor
+            if not isinstance(index, (paddle.fluid.core.VarBase,


same above, fix all other places

chenwhql · 2021-08-25T09:39:30Z

python/paddle/fluid/variable_index.py

-                    "When index contains a Tensor, its length must be 1, but received {}.".
-                    format(len(item)))
+        elif isinstance(slice_item, np.ndarray):
+            # delete


what delete mean?

Forgot to delete the comment. Has deleted it.

zyfncg · 2021-08-25T09:12:54Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

+        def contain_tensor(item):
+            if not isinstance(item, tuple):
+                item = [item]
+
+            for slice_item in item:
+                if isinstance(slice_item, slice):
+                    if isinstance(slice_item.start, Variable)  \
+                        or isinstance(slice_item.stop, Variable) \
+                           or isinstance(slice_item.step, Variable):
+                        return True
+                else:
+                    if isinstance(slice_item, Variable):
+                        return True
+            return False


是否可以和__getitem__共用一份contain_tensor代码 ?

zyfncg · 2021-08-25T16:04:53Z

python/paddle/fluid/variable_index.py

+                    "only support list/tensor index, but received {}.".format(
+                        type(index)))
+
+        # if len(self.indexes)>1:


zyfncg · 2021-08-25T16:12:02Z

python/paddle/fluid/variable_index.py

+        return reduce(lambda x, y: x * y, shape)
+
+    def get_offset_stride(self, tensor_shape):
+        for i in range(len(self.indexes)):


use index in self.indexes derectly?

zyfncg · 2021-08-26T02:15:14Z

python/paddle/fluid/tests/unittests/test_variable.py

+
+        index_shape = [2, 3, 4, 5, 6]
+        index = np.arange(self.numel(index_shape)).reshape(index_shape)
+        for i in range(len(inps_shape) - 1):


Are test cases same in the loop?

zyfncg · 2021-08-26T02:16:00Z

python/paddle/fluid/tests/unittests/test_variable.py

+        index_shape = [3, 3, 2, 1]
+        index = np.arange(self.numel(index_shape)).reshape(index_shape)
+
+        for i in range(3):


Are test cases same in the loop?

zyfncg · 2021-08-26T02:17:42Z

python/paddle/fluid/tests/unittests/test_variable.py

+        value_np = np.arange(
+            self.numel(value_shape), dtype='float32').reshape(value_shape) + 100
+
+        for i in range(3):


zyfncg · 2021-08-26T02:18:07Z

python/paddle/fluid/tests/unittests/test_variable.py

+        value_np = np.arange(
+            self.numel(value_shape), dtype='float32').reshape(value_shape) + 100
+
+        for i in range(4):


zyfncg · 2021-08-26T02:18:57Z

python/paddle/fluid/tests/unittests/test_variable.py

+        value_shape = [4]
+        value_np = np.arange(
+            self.numel(value_shape), dtype='float32').reshape(value_shape) + 100
+        for zz_ in range(3):


zyfncg · 2021-08-26T02:19:12Z

python/paddle/fluid/tests/unittests/test_variable.py

+        index2 = np.arange(
+            self.numel(index_shape), dtype='int32').reshape(index_shape) + 2
+
+        for zz_ in range(3):


zyfncg · 2021-08-26T02:19:22Z

python/paddle/fluid/tests/unittests/test_variable.py

+        index2 = np.arange(
+            self.numel(index_shape), dtype='int32').reshape(index_shape) + 2
+
+        for zz_ in range(3):


chenwhql

LGTM，动态图现在复用太多Python逻辑，后续需要解决由此引入的性能问题

paddle-bot-old bot referenced this pull request Aug 11, 2021

support tensor index.

ec94322

paddle-bot-old bot referenced this pull request Aug 11, 2021

polish code .

c0e70c7

paddle-bot-old bot referenced this pull request Aug 12, 2021

Merge remote-tracking branch 'upstream/develop' into slice/list

56e7384

paddle-bot-old bot referenced this pull request Aug 18, 2021

polish code.

13bcb58

chenwhql reviewed Aug 19, 2021

View reviewed changes

polish code

6f8681e

hbwx24 force-pushed the slice/list branch from 026fe87 to 6f8681e Compare August 24, 2021 12:09

hbwx24 added 2 commits August 25, 2021 00:23

polish code.

23eb322

polish code.

ce208bc

chenwhql reviewed Aug 25, 2021

View reviewed changes

zyfncg reviewed Aug 26, 2021

View reviewed changes

hbwx24 added 2 commits August 26, 2021 05:51

polish code.

2eca229

polish code.

ab9fdee

chenwhql approved these changes Aug 26, 2021

View reviewed changes

hbwx24 merged commit e7df47e into PaddlePaddle:develop Aug 26, 2021


		std::vector<int64_t> output_dim(input_dim.size() + index_dim.size() - 1);

		for (int i = 0; i < static_cast<int>(output_dim.size()); i++) {



		def setitem_list_index(var, index_list, value):
		from . import layers

support tensor index. #34824

support tensor index. #34824

Uh oh!

Conversation

hbwx24 commented Aug 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Describe

1.整体支持：

2.具体功能点支持：

3. 支持 tensor类型索引，示例如下：

Uh oh!

paddle-bot-old bot commented Aug 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddle-bot-old bot commented Aug 11, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hbwx24 Aug 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hbwx24 commented Aug 11, 2021 •

edited

Loading

paddle-bot-old bot commented Aug 11, 2021 •

edited

Loading

hbwx24 Aug 25, 2021 •

edited

Loading