Skip to content

Add notebook for Rubik Model conversion#2006

Merged
samfreund merged 13 commits intoPhotonVision:mainfrom
samfreund:model-comversion-docs
Aug 7, 2025
Merged

Add notebook for Rubik Model conversion#2006
samfreund merged 13 commits intoPhotonVision:mainfrom
samfreund:model-comversion-docs

Conversation

@samfreund
Copy link
Member

@samfreund samfreund commented Jul 20, 2025

Description

Adds a notebook that can be run on Google Colab for converting a PyTorch model to tflite for Rubik Pi.

closes #1951
closes #1953
closes #1435

Meta

Merge checklist:

  • Pull Request title is short, imperative summary of proposed changes
  • The description documents the what and why
  • If this PR changes behavior or adds a feature, user documentation is updated
  • If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
  • If this PR touches configuration, this is backwards compatible with settings back to v2024.3.1
  • If this PR touches pipeline settings or anything related to data exchange, the frontend typing is updated
  • If this PR addresses a bug, a regression test for it is added

@samfreund samfreund requested a review from a team as a code owner July 20, 2025 18:34
@samfreund samfreund marked this pull request as draft July 20, 2025 18:34
@samfreund samfreund force-pushed the model-comversion-docs branch 2 times, most recently from 058a529 to c2826d6 Compare July 22, 2025 23:01
@samfreund samfreund changed the title Add docs for model conversion Add docs for Rubik Model conversion Aug 3, 2025
@samfreund samfreund force-pushed the model-comversion-docs branch from 5a6e78b to c35c08b Compare August 3, 2025 22:31
@samfreund samfreund added documentation Anything relating to https://docs.photonvision.org enhancement New feature or request labels Aug 4, 2025
@samfreund samfreund changed the title Add docs for Rubik Model conversion Add notebook for Rubik Model conversion Aug 4, 2025
@samfreund samfreund force-pushed the model-comversion-docs branch from c35c08b to bb19e49 Compare August 5, 2025 02:18
@github-actions github-actions bot added backend Things relating to photon-core and photon-server and removed documentation Anything relating to https://docs.photonvision.org labels Aug 5, 2025
@samfreund samfreund marked this pull request as ready for review August 5, 2025 22:52
@samfreund samfreund force-pushed the model-comversion-docs branch from bb19e49 to 96d44e7 Compare August 5, 2025 22:52
@samfreund samfreund removed the backend Things relating to photon-core and photon-server label Aug 5, 2025
@github-actions github-actions bot added the backend Things relating to photon-core and photon-server label Aug 5, 2025
@samfreund samfreund force-pushed the model-comversion-docs branch 3 times, most recently from 83c534c to 0002250 Compare August 6, 2025 00:04
@samfreund
Copy link
Member Author

Documentation will be added in #2005

@samfreund samfreund requested a review from Gold856 August 6, 2025 13:49
@Gold856 Gold856 removed the backend Things relating to photon-core and photon-server label Aug 6, 2025
@boomermath
Copy link
Contributor

boomermath commented Aug 6, 2025

After trying it out here's what I think:

  • It seems to work without the Numpy Fix, although later it does prompt a session restart. You could possibly consider removing numpy pip install cell, and then just inform users to restart when prompted.
  • It took me a second to figure out for a Y/n prompt how to click on the box in the cell output and answer Y, maybe some instructions/notice abt that possibly?
  • It would be nice to have an overview of how the process works, for people who may be new to this, maybe don't need to walk thru Qualcomm signup, but maybe explain providing custom quantization samples for the tflite model (for better quantized performance, as I've seen, it seems to use COCO for quantization right now), and the jobs that the script creates on your qualcomm acct. Would be helpful to know so that people don't get confused by the job links/ONNX model that it creates, and wait til the end for the tflite model.

@samfreund samfreund force-pushed the model-comversion-docs branch from 0002250 to 77aa335 Compare August 7, 2025 00:06
@samfreund samfreund force-pushed the model-comversion-docs branch from a2ab448 to d11fc94 Compare August 7, 2025 04:52
@samfreund samfreund force-pushed the model-comversion-docs branch from d11fc94 to 3d508e1 Compare August 7, 2025 05:24
@samfreund samfreund requested a review from Gold856 August 7, 2025 05:26
@samfreund samfreund force-pushed the model-comversion-docs branch 3 times, most recently from 8981005 to c76b108 Compare August 7, 2025 05:45
@samfreund samfreund force-pushed the model-comversion-docs branch from c76b108 to 47ef806 Compare August 7, 2025 05:46
@samfreund
Copy link
Member Author

@boomermath if u could give this a final once-over, I think it's good to go

@boomermath
Copy link
Contributor

lgtm

@samfreund samfreund merged commit 65c214a into PhotonVision:main Aug 7, 2025
69 of 70 checks passed
@samfreund samfreund deleted the model-comversion-docs branch August 7, 2025 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TFLITE for Rubik Pi 3 Conversion Docs Notebooks for .pt conversion to Object Detection Models

3 participants