Skip to content

Replace 2 transpose() with 1 permute in TransformerBlock()`#5645

Merged
glenn-jocher merged 1 commit intoultralytics:masterfrom
dingyiwei:Transformer
Nov 15, 2021
Merged

Replace 2 transpose() with 1 permute in TransformerBlock()`#5645
glenn-jocher merged 1 commit intoultralytics:masterfrom
dingyiwei:Transformer

Conversation

@dingyiwei
Copy link
Copy Markdown
Contributor

@dingyiwei dingyiwei commented Nov 15, 2021

As discussed in the comment of #2329, using permute() to change dimensions of the feature in TransformerBlock instead of 2 transpose()s looks more elegant and readable.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhanced tensor manipulation for efficiency in YOLOv5 models.

📊 Key Changes

  • Replaced a sequence of tensor operations (unsqueeze, transpose, squeeze) with a single permute operation.
  • Simplified tensor reshaping in forward method to improve code readability.

🎯 Purpose & Impact

  • 🔍 Clarity: The new tensor operations are easier to understand.
  • Performance: Could potentially increase the efficiency of the model by using less memory-intensive tensor operations.
  • 🛠 Maintainability: Simpler codebase for future updates and maintenance.

@glenn-jocher glenn-jocher changed the title Use permute() instead of 2 transpose() in TransformerBlock Use permute() instead of 2 transpose() in Transformer C3TR() Nov 15, 2021
@glenn-jocher glenn-jocher changed the title Use permute() instead of 2 transpose() in Transformer C3TR() Use permute() instead of 2 transpose() in TransformerBlock()` Nov 15, 2021
@glenn-jocher glenn-jocher linked an issue Nov 15, 2021 that may be closed by this pull request
@glenn-jocher
Copy link
Copy Markdown
Member

/rebase

@glenn-jocher glenn-jocher changed the title Use permute() instead of 2 transpose() in TransformerBlock()` Replace 2 transpose() with 1 permute in TransformerBlock()` Nov 15, 2021
@glenn-jocher glenn-jocher merged commit c2523be into ultralytics:master Nov 15, 2021
@glenn-jocher
Copy link
Copy Markdown
Member

@dingyiwei PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply Transformer in the backbone

2 participants