Add section on obtaining vision measurements for pose estimation#3137
Open
jasondaming wants to merge 5 commits intowpilibsuite:mainfrom
Open
Add section on obtaining vision measurements for pose estimation#3137jasondaming wants to merge 5 commits intowpilibsuite:mainfrom
jasondaming wants to merge 5 commits intowpilibsuite:mainfrom
Conversation
Adds comprehensive "Obtaining Vision Measurements" section explaining how to get robot pose estimates from vision systems like PhotonVision and Limelight for use with AddVisionMeasurement(). Fixes wpilibsuite#1413
sciencewhiz
reviewed
Oct 13, 2025
source/docs/software/advanced-controls/state-space/state-space-pose-estimators.rst
Outdated
Show resolved
Hide resolved
Collaborator
|
Thinking about this more, I think most of this probably belongs in the vision processing docs, rather then here. |
Per review feedback, moved the detailed vision measurement information from the pose estimator docs to a new article in the vision processing section. Changes: - Created new apriltag-pose-estimation.rst in vision-processing/apriltag/ - Comprehensive guide covering PhotonVision, Limelight, and custom solutions - Detailed sections on timestamps, standard deviations, and rejecting bad measurements - Code examples in Java, C++, and Python for both libraries - Updated apriltag/index.rst to include new article - Replaced detailed section in state-space-pose-estimators.rst with a seealso link The pose estimator docs now focus on the API usage, while the vision processing docs explain how to obtain the measurements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Changed links in "See Also" section to anonymous hyperlinks (double underscores) to avoid conflicts with similar link text used earlier in the document. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…d-layout file Removed references to apriltag-field-layout.rst which doesn't exist in this PR: - Removed from toctree in index.rst - Removed from "See Also" section in apriltag-pose-estimation.rst The apriltag-field-layout article is being created in PR wpilibsuite#3138. Once that PR merges, the references can be added back. This fixes the build errors: - "toctree contains reference to nonexisting document" - "unknown document: 'apriltag-field-layout'" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
sciencewhiz
reviewed
Nov 24, 2025
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
- Add setup instructions for PhotonVision/Limelight AprilTag detection - Fix RST formatting (blank line before bullet list) - Fix standard deviation formula using AdvantageKit approach with proper baseline values (0.02m xy, 0.06rad theta) scaled by distance²/tagCount - Add conditional to use MAX_VALUE for theta with single tag (ambiguous) - Clarify that rejection example only shows field boundary checking - Add 0.5m tolerance margin for edge-of-field measurements 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
sciencewhiz
approved these changes
Nov 30, 2025
Gold856
suggested changes
Dec 4, 2025
Member
Gold856
left a comment
There was a problem hiding this comment.
Looks good to me, just some minor changes to match current photonlib.
|
|
||
| ## Using Vision Libraries | ||
|
|
||
| Most teams use existing vision processing libraries that handle the complex mathematics for you. Before using these libraries in your robot code, you'll need to configure the vision coprocessor to detect AprilTags. See the `PhotonVision AprilTag documentation <https://docs.photonvision.org/en/latest/docs/apriltag-pipelines/index.html>`__ or `Limelight AprilTag documentation <https://docs.limelightvision.io/docs/docs-limelight/pipeline-apriltag/apriltag-introduction>`__ for setup instructions. |
Comment on lines
+32
to
+44
| camera, // PhotonCamera | ||
| robotToCam // Transform3d from robot to camera | ||
| ); | ||
|
|
||
| // In your periodic method | ||
| var result = photonPoseEstimator.update(); | ||
| if (result.isPresent()) { | ||
| var estimatedPose = result.get(); | ||
| poseEstimator.addVisionMeasurement( | ||
| estimatedPose.estimatedPose.toPose2d(), | ||
| estimatedPose.timestampSeconds | ||
| ); | ||
| } |
Member
There was a problem hiding this comment.
Suggested change
| camera, // PhotonCamera | |
| robotToCam // Transform3d from robot to camera | |
| ); | |
| // In your periodic method | |
| var result = photonPoseEstimator.update(); | |
| if (result.isPresent()) { | |
| var estimatedPose = result.get(); | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| robotToCam // Transform3d from robot to camera | |
| ); | |
| // In your periodic method | |
| for (var result : camera.getAllUnreadResults()) { | |
| var estimate = photonPoseEstimator.update(result); | |
| if (estimate.isPresent()) { | |
| var estimatedPose = estimate.get(); | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } |
| ) | ||
|
|
||
| # In your periodic method | ||
| result = photon_pose_estimator.update() |
Member
There was a problem hiding this comment.
I'll PR an update to this when we've updated the API.
Comment on lines
+54
to
+65
| camera, // photon::PhotonCamera | ||
| robotToCam // frc::Transform3d from robot to camera | ||
| }; | ||
|
|
||
| // In your periodic method | ||
| auto result = photonPoseEstimator.Update(); | ||
| if (result) { | ||
| poseEstimator.AddVisionMeasurement( | ||
| result->estimatedPose.ToPose2d(), | ||
| result->timestamp | ||
| ); | ||
| } |
Member
There was a problem hiding this comment.
Suggested change
| camera, // photon::PhotonCamera | |
| robotToCam // frc::Transform3d from robot to camera | |
| }; | |
| // In your periodic method | |
| auto result = photonPoseEstimator.Update(); | |
| if (result) { | |
| poseEstimator.AddVisionMeasurement( | |
| result->estimatedPose.ToPose2d(), | |
| result->timestamp | |
| ); | |
| } | |
| robotToCam // frc::Transform3d from robot to camera | |
| }; | |
| // In your periodic method | |
| for (const auto& result : camera.GetAllUnreadResults()) { | |
| auto estimate = photonPoseEstimator.Update(result); | |
| if (estimate) { | |
| poseEstimator.AddVisionMeasurement( | |
| estimate->estimatedPose.ToPose2d(), | |
| estimate->timestamp | |
| ); | |
| } | |
| } |
Comment on lines
+262
to
+278
| var result = photonPoseEstimator.update(); | ||
| if (result.isPresent()) { | ||
| var estimatedPose = result.get(); | ||
|
|
||
| // Check if pose is reasonable (within field boundaries with tolerance) | ||
| double margin = 0.5; // meters of tolerance for edge measurements | ||
| if (estimatedPose.estimatedPose.getX() >= -margin && | ||
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | ||
| estimatedPose.estimatedPose.getY() >= -margin && | ||
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | ||
|
|
||
| poseEstimator.addVisionMeasurement( | ||
| estimatedPose.estimatedPose.toPose2d(), | ||
| estimatedPose.timestampSeconds | ||
| ); | ||
| } | ||
| } |
Member
There was a problem hiding this comment.
Suggested change
| var result = photonPoseEstimator.update(); | |
| if (result.isPresent()) { | |
| var estimatedPose = result.get(); | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| double margin = 0.5; // meters of tolerance for edge measurements | |
| if (estimatedPose.estimatedPose.getX() >= -margin && | |
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | |
| estimatedPose.estimatedPose.getY() >= -margin && | |
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } | |
| for (var result : camera.getAllUnreadResults()) { | |
| var estimate = photonPoseEstimator.update(result); | |
| if (estimate.isPresent()) { | |
| var estimatedPose = estimate.get(); | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| double margin = 0.5; // meters of tolerance for edge measurements | |
| if (estimatedPose.estimatedPose.getX() >= -margin && | |
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | |
| estimatedPose.estimatedPose.getY() >= -margin && | |
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } | |
| } |
Comment on lines
+282
to
+296
| auto result = photonPoseEstimator.Update(); | ||
| if (result) { | ||
| // Check if pose is reasonable (within field boundaries with tolerance) | ||
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | ||
| if (result->estimatedPose.X() >= -margin && | ||
| result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | ||
| result->estimatedPose.Y() >= -margin && | ||
| result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | ||
|
|
||
| poseEstimator.AddVisionMeasurement( | ||
| result->estimatedPose.ToPose2d(), | ||
| result->timestamp | ||
| ); | ||
| } | ||
| } |
Member
There was a problem hiding this comment.
Suggested change
| auto result = photonPoseEstimator.Update(); | |
| if (result) { | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | |
| if (result->estimatedPose.X() >= -margin && | |
| result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | |
| result->estimatedPose.Y() >= -margin && | |
| result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | |
| poseEstimator.AddVisionMeasurement( | |
| result->estimatedPose.ToPose2d(), | |
| result->timestamp | |
| ); | |
| } | |
| } | |
| for (const auto& result : camera.GetAllUnreadResults()) { | |
| auto estimate = photonPoseEstimator.Update(result); | |
| if (estimate) { | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | |
| if (estimate->estimatedPose.X() >= -margin && | |
| estimate->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | |
| estimate->estimatedPose.Y() >= -margin && | |
| estimate->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | |
| poseEstimator.AddVisionMeasurement( | |
| estimate->estimatedPose.ToPose2d(), | |
| estimate->timestamp | |
| ); | |
| } | |
| } | |
| } |
|
|
||
| Most vision libraries provide this timestamp: | ||
|
|
||
| - PhotonVision: ``result.timestampSeconds`` |
Member
There was a problem hiding this comment.
Suggested change
| - PhotonVision: ``result.timestampSeconds`` | |
| - PhotonVision: ``estimate.timestampSeconds`` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Creates a comprehensive guide for using AprilTags for robot pose estimation, addressing the lack of documentation on obtaining vision measurements for pose estimators.
Changes
New file:
apriltag-pose-estimation.rstin vision processing sectionModified:
state-space-pose-estimators.rstModified:
apriltag/index.rstOrganization
Per review feedback, vision measurement content now lives in vision processing docs (where teams look for vision guidance) rather than pose estimator docs (which focus on the API).
Fixes #1413