Skip to content

Add Impeller migration step for camera plugin authors #11008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sfshaza2 merged 22 commits into from
Aug 20, 2024
Merged

Add Impeller migration step for camera plugin authors #11008

Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
d3d5327
draft
camsim99 Aug 5, 2024
7acd6a7
reword + 80 chars
camsim99 Aug 6, 2024
6ea3140
add plaintext
camsim99 Aug 6, 2024
63cf4bb
Merge remote-tracking branch 'upstream/main' into impeller_packages
camsim99 Aug 6, 2024
2fe05b2
Merge branch 'main' into impeller_packages
parlough Aug 8, 2024
db7cd6b
Address review
camsim99 Aug 12, 2024
167dead
Merge remote-tracking branch 'refs/remotes/origin/impeller_packages' …
camsim99 Aug 12, 2024
9fee67b
Change dev orientation instructions to point to official documentation
camsim99 Aug 12, 2024
056c5ec
Clarify device orientation degrees
camsim99 Aug 12, 2024
11b707f
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
99c5b22
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
7da2899
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
45d947f
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
adcf9ff
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
fbf4da2
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
0263532
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
c9a14ef
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
ef521f1
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
48cf2a2
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
f5a9c29
Update src/content/release/breaking-changes/android-surface-plugins.md
camsim99 Aug 20, 2024
6a7522c
remove unused link
camsim99 Aug 20, 2024
c9bde67
Merge remote-tracking branch 'refs/remotes/origin/impeller_packages' …
camsim99 Aug 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion src/content/release/breaking-changes/android-surface-plugins.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,36 @@ surfaceProducer.setCallback(
A full example of using this new API can be found in [PR 6989][] for the
`video_player_android` plugin.

## Note on camera previews

If your plugin implements a camera preview, your migration might also require
fixing the rotation of that preview. This is because `Surface`s produced by the
`SurfaceProducer` might not contain the transformation information that Android
libraries need to correctly rotate the preview automatically.

In order to correct the rotation, you need to rotate the preview with
respect to the camera sensor orientation and the device orientation according
to the equation:

```plaintext
rotation = (sensorOrientationDegrees - deviceOrientationDegrees * sign + 360) % 360
Copy link
Contributor

@navaronbracke navaronbracke Aug 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we provide more context on how to retrieve sensorOrientationDegrees and deviceOrientationDegrees
directly in this document? (instead of pointing people to a GitHub PR)

We provide code samples for other things in this migration guide (see line 59 in this file), so can we do the same for this addition as well? I don't feel strongly about this, but maybe not everyone writing plugins / reading this doc is okay with going through a potentially long Github PR to find what they need, to migrate.

I would definitely keep the PR link, though, so people have a completed migration to look at as well.

Copy link
Contributor Author

@camsim99 camsim99 Aug 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some links for further clarification because I agree that the PR isn't enough. We also had to make some Flutter camera plugin-specific fixes in it, so I don't want the focus to be there.

I think I'll leave it here unless others feel strongly because I think camera plugin authors are familiar with the APIs and could make the migration with this information.

```

where `deviceOrientationDegrees` is counterclockwise degrees and `sign` is 1 for
front-facing cameras and -1 for back-facing cameras.

To calculate this rotation,
- Retrieve the sensor orientation degrees by retrieving the value of
[`CameraCharacteristics.SENSOR_ORIENTATION`][].
- Retrieve the device orientation degrees in one of the ways that the
[Android orientation calculation documentation][] details.

To apply this rotation, you can use a [`RotatedBox`][] widget.

For more information on this calculation, check out the
[Android orientation calculation documentation][]. For a full example of making
this fix, check out [this `camera_android_camerax` PR][].

## Timeline

Landed in version: 3.22
Expand Down Expand Up @@ -116,7 +146,11 @@ Relevant PRs:
[`createSurfaceProducer`]: {{site.api}}/javadoc/io/flutter/view/TextureRegistry.html#createSurfaceProducer()
[`createSurfaceTexture`]: {{site.api}}/javadoc/io/flutter/view/TextureRegistry.html#createSurfaceTexture()
[`getSurface()`]: {{site.api}}/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#getSurface()
[`setCallback`]: https://main-api.flutter.dev/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#setCallback(io.flutter.view.TextureRegistry.SurfaceProducer.Callback)
[`setCallback`]: {{site.api}}/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#setCallback(io.flutter.view.TextureRegistry.SurfaceProducer.Callback)
[`CameraCharacteristics.SENSOR_ORIENTATION`]: {{site.android-dev}}/reference/android/hardware/camera2/CameraCharacteristics#SENSOR_ORIENTATION
[`RotatedBox`]: {{site.api}}/flutter/widgets/RotatedBox-class.html
[Android orientation calculation documentation]: {{site.android-dev}}/media/camera/camera2/camera-preview#orientation_calculation
[this`camera_android_camerax` PR]: {{site.repo.flutter}}/packages/pull/7044
[Issue 139702]: {{site.repo.flutter}}/issues/139702
[Issue 145930]: {{site.repo.flutter}}/issues/145930
[PR 51061]: {{site.repo.engine}}/pull/51061
Expand Down