-
Notifications
You must be signed in to change notification settings - Fork 234
Remove implicit package installs #1413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Shah, Karan <[email protected]>
2504e60 to
d0ce082
Compare
Signed-off-by: Shah, Karan <[email protected]>
Signed-off-by: Shah, Karan <[email protected]>
eb0ccbe to
567ea37
Compare
kminhta
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a reasonable change. It adds an expected amount of flexibility back to the user and may even reduce some dependency issues if users can freely manage their env packages rather than rely entirely on pip install -r requirements.txt (which should be the case, IMO)
|
One issue I foresee is, if user mesh-up with requirements, user will know it at until aggregator/collaborator starts. |
|
Although I agree that we should not install packages during |
|
While I agree that a missing package is an inconvenience, the fix is generally very straightforward. In real-world deployments, dependency installations are a common step and explicitly stated. In addition Overall, it is preferable for functions to be idempotent and for them to be consistent with what a user would expect it to do by its name. It is strange for Related: principle of least astonishment. |
psfoley
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this will break GaNDLF's CI in it's current state:
https://github.com/mlcommons/GaNDLF/blob/master/.github/workflows/openfl-test.yml#L55
See here for the missing call to pip install : https://github.com/securefederatedai/openfl/blob/develop/tests/github/test_gandlf.py#L50-L63
Once you can confirm that this is either tested, or you add a commit to add the pip install, I think this is ready to merge.
Signed-off-by: Shah, Karan <[email protected]>
|
Good catch, resolved in f76bd7b |
* Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]>
* Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]>
* Bump matplotlib from 3.10.0 to 3.10.1 (#1416) Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.10.0 to 3.10.1. - [Release notes](https://github.com/matplotlib/matplotlib/releases) - [Commits](https://github.com/matplotlib/matplotlib/compare/v3.10.0...v3.10.1) --- updated-dependencies: - dependency-name: matplotlib dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Remove implicit package installs (#1413) * Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chages Signed-off-by: yes <[email protected]> * Adding Secure Aggregation Test (#1419) * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Format issue Signed-off-by: Chaurasiya, Payal <[email protected]> * Put Secure Agg Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * [tutorial] Fix `KeyError: 'w_old'` (#1425) Signed-off-by: Situ Singh <[email protected]> Signed-off-by: yes <[email protected]> * Moving GaNDLF test to pytest framework (#1423) * Upload artifacts for GanDLF Signed-off-by: noopur <[email protected]> * Moving GaNDLF to pytest Signed-off-by: noopur <[email protected]> * Move papermill import isndie relevant function Signed-off-by: noopur <[email protected]> * Add paramiko install in the step Signed-off-by: noopur <[email protected]> * Use model_name with request.config Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Corrected the marker Signed-off-by: noopur <[email protected]> * Corrected the range loop for collab data Signed-off-by: noopur <[email protected]> * Export workspace after certify steps Signed-off-by: noopur <[email protected]> * Make GaNDLF workflow as E2E Signed-off-by: noopur <[email protected]> * Correction in conditions Signed-off-by: noopur <[email protected]> * More checks at pytest level Signed-off-by: noopur <[email protected]> * Added missing import Signed-off-by: noopur <[email protected]> * Debug stmts Signed-off-by: noopur <[email protected]> * Corrected path for config seg file Signed-off-by: noopur <[email protected]> * Corrected curr work dir Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction due to code rebase Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> * Correction in checkout of gandlf step Signed-off-by: noopur <[email protected]> * Final changes in gandlf.yml Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Cleanup (#1422) * DCE, rename variable Signed-off-by: Shah, Karan <[email protected]> * Move to module-level loggers Signed-off-by: Shah, Karan <[email protected]> * Plan.logger to logger Signed-off-by: Shah, Karan <[email protected]> * Fixup Signed-off-by: Shah, Karan <[email protected]> * Also for workspaces Signed-off-by: Shah, Karan <[email protected]> * Fix again for logger Signed-off-by: Shah, Karan <[email protected]> * Remove run_simulation tests Signed-off-by: Shah, Karan <[email protected]> * Header creation is stateless and common across both Signed-off-by: Shah, Karan <[email protected]> * Remove TASK_REGISTRY code; deprecated with interactive Signed-off-by: Shah, Karan <[email protected]> * More use_delta_updates changes Signed-off-by: Shah, Karan <[email protected]> * lets see if this revert works Signed-off-by: Shah, Karan <[email protected]> * Remove openfl install in notebook Signed-off-by: Shah, Karan <[email protected]> * revert few flags Signed-off-by: Shah, Karan <[email protected]> * please please please this should work Signed-off-by: Shah, Karan <[email protected]> * Migrate stateless function to common Signed-off-by: Shah, Karan <[email protected]> * Cleanup outdated docstrings Signed-off-by: Shah, Karan <[email protected]> * Remove Interactive API RPCs Signed-off-by: Shah, Karan <[email protected]> * Remove set devices call; not used anywhere Signed-off-by: Shah, Karan <[email protected]> * Replace checks with inline assertions Signed-off-by: Shah, Karan <[email protected]> * Unfold one-liner functions Signed-off-by: Shah, Karan <[email protected]> * Remove moot tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * B413 bandit issue ignored (#1427) * B413 bandit issue ignored Signed-off-by: yes <[email protected]> * code formatted Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory) fix (#1424) * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * fix(plan reading): use yaml safe_load (#1432) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * [bandit] Ignore B614: Use of unsafe `torch.load` (#1426) * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * testing changes Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * formatted code Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Use Single Function For Logging With Rich Text Handler (#1435) * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * More params to Rich constructor Signed-off-by: noopur <[email protected]> * Do not run straggler Signed-off-by: noopur <[email protected]> * Use formatter for rich Signed-off-by: noopur <[email protected]> * Missed a comma Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * E2E: use rich text handler Signed-off-by: noopur <[email protected]> * Keep formatter with message only Signed-off-by: noopur <[email protected]> * Use logs file Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Retain same function name Signed-off-by: noopur <[email protected]> * Code formatting fixed Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add collaborator name attribute to `AggregatorGRPCClient` (#1436) * Remove explicit collaborator name arg within RPC calls Signed-off-by: Shah, Karan <[email protected]> * Update tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * Bump keras from 3.8.0 to 3.9.0 in /openfl-workspace/keras/2dunet (#1439) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Deleting AggregatorBasedWorkflow tutorials and examples (#1438) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * updated keras (#1447) Signed-off-by: yes <[email protected]> * updated keras (#1446) Signed-off-by: yes <[email protected]> * updated keras (#1445) Signed-off-by: yes <[email protected]> * Bump keras in /openfl-workspace/keras/tensorflow/mnist (#1444) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shailesh Tanwar <[email protected]> Signed-off-by: yes <[email protected]> * Remove the usage of `shell=True` when running commands using `subprocess` (#1443) * fix(bandit b602): remove the usage of shell=True when running commands using subprocess Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): keyword argument name Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): value type in dict Signed-off-by: Pant, Akshay <[email protected]> * fix(dockerize): command args Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Fix typos (#1449) Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: yes <[email protected]> * Upgrading nbdev package version from 2.3.12 to 2.3.37 to fix ImportError (#1456) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E: Remove ANSI escape codes and unwanted patterns from participant log files with LOG_FILE usage (#1455) * Use LOG_FILE for logging to participant log files Signed-off-by: noopur <[email protected]> * Append & for background processes Signed-off-by: noopur <[email protected]> * Do not show date in log files Signed-off-by: noopur <[email protected]> * Param correction Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Using & Signed-off-by: noopur <[email protected]> * Keep bg file running Signed-off-by: noopur <[email protected]> * Upgrade nbdev from 2.3.12 to 2.3.37 Signed-off-by: noopur <[email protected]> * Skip csv from artifact upload Signed-off-by: noopur <[email protected]> * Variable moved up Signed-off-by: noopur <[email protected]> * Move time.sleep before process id check Signed-off-by: noopur <[email protected]> * Remove current round logging Signed-off-by: noopur <[email protected]> * Do not force rich text to terminal Signed-off-by: noopur <[email protected]> * Specific console for conftest Signed-off-by: noopur <[email protected]> * Use single rich handler in conftest Signed-off-by: noopur <[email protected]> * Change log file path for dockerized workspace Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Update `installation.rst` (#1452) * Documentation update to specify: - requirements for python versions compatible with OpenFL - requirements for platforms compatible with the OpenFL docker image Signed-off-by: Lior Malka <[email protected]> * Responding to PR comments Signed-off-by: Lior Malka <[email protected]> * Paraphrasing python supported versions Signed-off-by: Lior Malka <[email protected]> --------- Signed-off-by: Lior Malka <[email protected]> Signed-off-by: yes <[email protected]> * dependency(ray): update to 2.43.0 from 2.9.2 (#1448) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Introducing Task Runner workspace to enable interoperability with Flower workloads (#1433) * enable flwr workspace Signed-off-by: kta-intel <[email protected]> * add tmp dir to minimize how much patching is needed Signed-off-by: kta-intel <[email protected]> * add hash verification to setup data Signed-off-by: kta-intel <[email protected]> * remove patch Signed-off-by: kta-intel <[email protected]> * fix save location Signed-off-by: kta-intel <[email protected]> * docstring Signed-off-by: kta-intel <[email protected]> * remove superfluous edit Signed-off-by: kta-intel <[email protected]> * update readme Signed-off-by: kta-intel <[email protected]> * remove debugger Signed-off-by: kta-intel <[email protected]> * update aggregator client Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * remove todo Signed-off-by: kta-intel <[email protected]> * minor functionality fixes Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update README instructions Signed-off-by: kta-intel <[email protected]> * do not add connector to settings unless connector exists Signed-off-by: kta-intel <[email protected]> * check attribute for connector availability Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * code cleanup Signed-off-by: kta-intel <[email protected]> * fix readme Signed-off-by: kta-intel <[email protected]> * update plan to reflect refactoring Signed-off-by: kta-intel <[email protected]> * grammar fix Signed-off-by: kta-intel <[email protected]> * remove __all__ Signed-off-by: kta-intel <[email protected]> * remove superfluous init Signed-off-by: kta-intel <[email protected]> * change flwr home dir Signed-off-by: kta-intel <[email protected]> * remove patch comment, .sort() added in flwr 1.16 Signed-off-by: kta-intel <[email protected]> * update name for grpc protocols and components Signed-off-by: kta-intel <[email protected]> * remove Connector abc Signed-off-by: kta-intel <[email protected]> * refactoring local grpc to interop Signed-off-by: kta-intel <[email protected]> * import fixes Signed-off-by: kta-intel <[email protected]> * remove duplicate self.callback Signed-off-by: kta-intel <[email protected]> * fix hashes Signed-off-by: kta-intel <[email protected]> * move into workspace for relative path installation Signed-off-by: kta-intel <[email protected]> * update readme to remove connector ABC Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Introduce a default `WaitForAllPolicy` for straggler handling (#1461) * feat(straggler handling): add NoPolicy Signed-off-by: Pant, Akshay <[email protected]> * feat(straggler handling): use NoPolicy as the default Signed-off-by: Pant, Akshay <[email protected]> * restructure(straggler handling): rename NoPolicy to WaitForAllPolicy Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Get current round for non FedEval scenarios (#1463) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * [taskrunner] Secure Aggregation cleanup (#1420) * fix(dependency): remove pycryptodome from openfl setup Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg): removed bootstrap steps from aggregator to helper class Signed-off-by: Pant, Akshay <[email protected]> * restructure(collaborator secagg): move masking step to utility function Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): import utility functions only when used Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): add check for pycrptodome installation when importing Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add future enhancements that are not supported Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): bootstrap class e2e Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): update bootstrap class usage and not masking metrics Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): move masked input vector calculation to collaborator class and remove masking from metrics Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): rename methods more appropriately Signed-off-by: Pant, Akshay <[email protected]> * fix(aggregator): remove duplicate callbacklist initialisation Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg setup): remove unnecesary else conditional block Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add missing word Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): use importlib instead of pkg_resources to check module installation Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): check pycrytodome install Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator): agg function slection Signed-off-by: Pant, Akshay <[email protected]> * fix: remove repeated line Signed-off-by: Pant, Akshay <[email protected]> * feat(plan): add plan verify to check for incompatible features Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add explicit mention for using WaitForAllPolicy with secagg Signed-off-by: Pant, Akshay <[email protected]> * fix(plan): change verify to non-static method Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Added notes for non-TLS in Quickstart documentation (#1462) * Separate section for non-TLS Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Keep non TLS at the top Signed-off-by: noopur <[email protected]> * Optional step Signed-off-by: noopur <[email protected]> * Review comment incorp Signed-off-by: noopur <[email protected]> * Forgot to save the changes earlier Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Security TLS certificate report tests (#1460) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Reduce Aggregator Memory Usage (#1459) * Changes to reduce unnecessary allocations Signed-off-by: Shah, Karan <[email protected]> * Remove compression/decompression for _prepare_trained Signed-off-by: Shah, Karan <[email protected]> * Execute SendLocalTaskResults in a queued call Signed-off-by: Shah, Karan <[email protected]> * Update test with the remove col_name argument Signed-off-by: Shah, Karan <[email protected]> * Use allclose instead of equal Signed-off-by: Shah, Karan <[email protected]> * Revert "Remove compression/decompression for _prepare_trained" This reverts commit cb0adebcf80bc95f47dc2f28c416d2a622bfc039. Signed-off-by: Shah, Karan <[email protected]> * Move lock decorator to common Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chnages Signed-off-by: yes <[email protected]> * code changes as per comments Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * Skipping GaNDLF run from PR pipeline (#1482) * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file (#1479) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Update taskrunner tutorial Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * update README.md and plan.yaml for `flower-app-pytorch` workspace (#1470) * chore: update openfl version to 1.8 (#1464) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1467) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * update README.md Signed-off-by: kta-intel <[email protected]> * 1.8: Raise error if data is not found when running flower-app-pytorch workspace (#1473) * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * raise error if data is not found Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: Akshay Pant <[email protected]> * add fqdn flag to aggregator commands, fix auto_shutdown key, add additional notes to experiment configuration Signed-off-by: kta-intel <[email protected]> * restructure: remove trailing whitespace (#1483) Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Remove additional folder permissioning (#1485) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Issue with Gandlf solved (#1486) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * OpenFL v1.8 release notes (#1487) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * revert core components changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * E2E automation for `flower-app-pytorch` workspace (#1494) * Test Flower automation Signed-off-by: noopur <[email protected]> * Run only once Signed-off-by: noopur <[email protected]> * Added Flower as separate job Signed-off-by: noopur <[email protected]> * Extra closing bracket Signed-off-by: noopur <[email protected]> * Removed dependencies installation step Signed-off-by: noopur <[email protected]> * Correction of step name and logging Signed-off-by: noopur <[email protected]> * New marker for flower model Signed-off-by: noopur <[email protected]> * Separate workflow for Flower Signed-off-by: noopur <[email protected]> * Changes for dockerized ws Signed-off-by: noopur <[email protected]> * Modified test name to reflect flower model Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * 1.8: Address Coverity issues related to `flower-app-pytorch` workspace (#1488) (#1490) * bind socket to a specific interface * add input validation and allow list * address filepath coverity issue by creating a function to check path safety using set of allowed characters * pin upper bound to flwr * formatting --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Hotfix for PQ pipeline (#1497) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * changes as per comments Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * plan updated Signed-off-by: yes <[email protected]> * Added conda installation instructions (#1508) * added publications Signed-off-by: sarthakpati <[email protected]> * added conda installation instructions Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest-asyncio from 0.25.3 to 0.26.0 (#1509) Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.25.3 to 0.26.0. - [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases) - [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.25.3...v0.26.0) --- updated-dependencies: - dependency-name: pytest-asyncio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * add hippmapp3r (#1498) * add hippmapp3r Signed-off-by: porteratzo <[email protected]> * addresed comments Signed-off-by: porteratzo <[email protected]> * add reference Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Signed-off-by: yes <[email protected]> * Bump ruff from 0.9.9 to 0.11.2 (#1515) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Simplify the components' `start` and `stop` processes (#1514) * Run resiliency 3 times Signed-off-by: noopur <[email protected]> * Run 50 rounds Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Upgraded pytest from 8.3.4 to 8.3.5 Signed-off-by: noopur <[email protected]> * Specific functions to fetch pids and kill processes Signed-off-by: noopur <[email protected]> * Multiple changes done Signed-off-by: noopur <[email protected]> * File changes as part of lint fixing Signed-off-by: noopur <[email protected]> * Minor logging fixes and copyright year correction Signed-off-by: noopur <[email protected]> * Review comments addressed Signed-off-by: noopur <[email protected]> * Revert pytest version upgrade change Signed-off-by: noopur <[email protected]> * Check if start_process is present Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest from 8.3.4 to 8.3.5 (#1510) Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.3.4 to 8.3.5. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/8.3.4...8.3.5) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump aquasecurity/trivy-action in the github-actions group (#1512) Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action). Updates `aquasecurity/trivy-action` from 0.29.0 to 0.30.0 - [Release notes](https://github.com/aquasecurity/trivy-action/releases) - [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0) --- updated-dependencies: - dependency-name: aquasecurity/trivy-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Signed-off-by: yes <[email protected]> * Change to add time taken for each round in metrics (#1517) * Change to add time taken for each round in metrics Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * [Workflow Interface] Rename and Refactor WorkspaceExport into NotebookTools (#1364) * Experimental-export-module-refactoring Signed-off-by: refai06 <[email protected]> * Incorporated comments Signed-off-by: refai06 <[email protected]> * Improvements added Signed-off-by: refai06 <[email protected]> * Added testcase Signed-off-by: refai06 <[email protected]> * Code Enhancement Signed-off-by: refai06 <[email protected]> * Docstring & loggers update Signed-off-by: refai06 <[email protected]> * Added NotebookTools module Testcase Signed-off-by: refai06 <[email protected]> * Testcase update Signed-off-by: refai06 <[email protected]> * Review_comments_incorporated Signed-off-by: refai06 <[email protected]> * Fix testcase Signed-off-by: refai06 <[email protected]> * Added review comments and testcase update Signed-off-by: refai06 <[email protected]> * Review comments Signed-off-by: refai06 <[email protected]> * Remove testcase Signed-off-by: refai06 <[email protected]> * Enhance code and incorporate review comments Signed-off-by: refai06 <[email protected]> * Code changes Signed-off-by: refai06 <[email protected]> * revert tutorial change Signed-off-by: refai06 <[email protected]> * Incorporate comments Signed-off-by: refai06 <[email protected]> * Docstring update Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Publish `openfl-nightly` package to `pypi` on daily basis (#1504) * PQ Pipeline: Publish openfl-nightly package to pypi on daily basis Signed-off-by: noopur <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: noopur <[email protected]> * Run Trivy and Bandit on PQ scheduled run Signed-off-by: noopur <[email protected]> * Syntax correction Signed-off-by: noopur <[email protected]> * Added required permissions for Trivy and Bandit Signed-off-by: noopur <[email protected]> * Remove generic permissions Signed-off-by: noopur <[email protected]> * PQ pipeline - run trivy and bandit for workflow_dispatch as well Signed-off-by: noopur <[email protected]> * Use version from setup.pt Signed-off-by: noopur <[email protected]> * Extra files for lint related fixes Signed-off-by: noopur <[email protected]> * Call publish wf from pq with specific commit id Signed-off-by: noopur <[email protected]> * All jobs to run for scheduler from main branch or manual trigger only Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Changes in PQ Pipeline & Publish Package workflows for PYPI_API_TOKEN (#1520) * Added dev environment Signed-off-by: noopur <[email protected]> * Added dev environment Signed-off-by: noopur <[email protected]> * Do not continue on error Signed-off-by: noopur <[email protected]> * Added permission for token read write Signed-off-by: noopur <[email protected]> * Use shell commands Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Changes in PQ pipeline as well Signed-off-by: noopur <[email protected]> * Comment out jobs fopr quick testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass on token Signed-off-by: noopur <[email protected]> * Inherit secret Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Revert the testing changes Signed-off-by: noopur <[email protected]> * Renmote printing job Signed-off-by: noopur <[email protected]> * Run publish step for manual trigger as well Signed-off-by: noopur <[email protected]> * Ignore case for Aggregated model validation score Signed-off-by: noopur <[email protected]> * Read 10 lines Signed-off-by: noopur <[email protected]> * For trusted publishing Signed-off-by: noopur <[email protected]> * Revert previous change Signed-off-by: noopur <[email protected]> * Search string in lowercase Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Collaborator CLI command for pinging the aggregator (#1516) * Adding a collaborator command for pinging the aggregator, without starting any tasks Signed-off-by: Teodor Parvanov <[email protected]> * Addressing review comments Signed-off-by: Teodor Parvanov <[email protected]> * Additional logging for the "fx collaborator ping" command Signed-off-by: Teodor Parvanov <[email protected]> * Additional documentation on the usage of fx collaborator ping Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Handling for .proto files in openfl-nightly package (#1523) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix for openfl-nightly package Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert changes from pre-test Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * Run an E2E test before publishing openfl-nightly package (#1525) * Check wheel file name Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Minor corrections Signed-off-by: noopur <[email protected]> * Github summary step Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Remove extra WHL_FILE assignment Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Verifiable Dataset (#1434) * Verifiable Dataset Signed-off-by: Dar, Efrat <[email protected]> * Pytorch classes for verifaiable dataset loading and verify Signed-off-by: Dar, Efrat <[email protected]> * Move code to openfl.federated.data.sources Signed-off-by: Dar, Efrat <[email protected]> * New validation method (comb of concise and verbose): When create hash, save in memory both root_hash (concise) and all_hashes (verbose) Then when validate the whole dataset, verify against root_hash only, and when validate a single object, verify against relevant hash from all_hashes. When serialize to json, save only root_hash. Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> * Fix: move base_path to LocalDataSource Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Update HippMapp3r plan, requirments and readme (#1524) * fixes Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: yes <[email protected]> * Task callback enhancement / Bugfix (#1469) * task callback support * adjust save model to reflect right tensor key * bump round number after saving model so tasks does not use advanced round number until model is updated * set extra params on the callback object and not function for flexibility based on CR comments * use round as is from persistent store * attempt for lambda filtering to avoid duplicate iterations --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Removed keras/jax/mnist from dockerized ws run (#1532) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * No-op workspace for TaskRunner API (#1527) * Initializing a no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Refactor no-op task runner and data loader to workspace Signed-off-by: Teodor Parvanov <[email protected]> * Fixed copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Workaround for model initialization for the no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Additionally standardizing the copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Adding a README.md Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Add empty requirements.txt to the no-op workspace template (#1536) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Removing the references of federated-evaluation/aggregator.yaml (#1535) Signed-off-by: yes <[email protected]> * undo round number updates (#1537) Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Adding workflow for creating openfl Dependency list (#1530) * create ospdt dependency workflow * dependency list * fixed code format * create ospdt dependency workflow * dependency list * fixed code format * modified filename as release_version-date.xlsx * added format * resolved comments * resolved comments --------- Co-authored-by: Rajith <[email protected]> Signed-off-by: yes <[email protected]> * Run a FedEval testcase in PR pipeline (#1539) * Run FedEval test case in PR pipeline Signed-off-by: noopur <[email protected]> * Added condition for non-draft PR as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Fixing bandit issue (#1540) Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * return from save_model immediately if federation evaluation mode is enabled (#1538) * return from save_model immediately if federation evaluation mode is enabled Signed-off-by: Rahul Garg <[email protected]> * fix the format issue Signed-off-by: Rahul Garg <[email protected]> * Format issue Signed-off-by: Rahul Garg <[email protected]> * Fix the test case and move the location to skip the model save Signed-off-by: Rahul Garg <[email protected]> * remove unnecessary code Signed-off-by: Rahul Garg <[email protected]> * formatting issue fix Signed-off-by: Rahul Garg <[email protected]> * fix linting issue Signed-off-by: Rahul Garg <[email protected]> * addressing comments Signed-off-by: Rahul Garg <[email protected]> --------- Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner FedEval - fetch model score from metric file instead of tensor db (#1543) * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Variable name correction Signed-off-by: noopur <[email protected]> * Consider only numeric value of the line Signed-off-by: noopur <[email protected]> * File converted to json and some lint fixes Signed-off-by: noopur <[email protected]> * Explicit run of test-requirements.txt in GaNDLF Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Load native model with fx plan initialize (#1528) * enable loading of model path in native format Signed-off-by: kta-intel <[email protected]> * add load_native to runner_xgb.py Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * check pbuf file extension Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Minor bug fix to `runner_xgb.py` to properly rebuild model during aggregated model validation (#1529) * fix conditional to rebuild model only during aggregated validation Signed-off-by: kta-intel <[email protected]> * remove print statement Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E - Added missing `needs` for straggler and eden jobs (#1545) * Sort collaborators while displaying Signed-off-by: noopur <[email protected]> * Added needs in straggler and eden jobs Signed-off-by: noopur <[email protected]> * Input selection fix Signed-off-by: noopur <[email protected]> * Syntax fix Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Revert sorting logic Signed-off-by: noopur <[email protected]> * Simplified the no of lines logic Signed-off-by: noopur <[email protected]> * Consider all lines if lesser than the index Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Adding Federation Runtime to Federated Evaluation workflow (#1519) * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Update openfl-tutorials/experimental/workflow/FederatedEvaluation/workspace/MNIST_FederatedEvaluation.ipynb Co-authored-by: Copilot <[email protected]> * added doc string --------- Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: yes <[email protected]> * Documentation: minor correction in `MNIST Example: Workflow API` (#1549) * Minor correction in Workflow API doc Signed-off-by: noopur <[email protected]> * Revert some unintended changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * rename analysis to analytics, moved code to src folder Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * added callback on_round_end for FA Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: yes <[email protected]> Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Situ Singh <[email protected]> Signed-off-by: noopur <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: Lior Malka <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: payalcha <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: porteratzo <[email protected]> Signed-off-by: refai06 <[email protected]> Signed-off-by: Dar, Efrat <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Karan Shah <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Situ Singh <[email protected]> Co-authored-by: Noopur <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]> Co-authored-by: Yaroslav Halchenko <[email protected]> Co-authored-by: Lior Malka <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: refai06 <[email protected]> Co-authored-by: Efrat1 <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: senthil <[email protected]> Co-authored-by: rahulga1 <[email protected]> Co-authored-by: rajithkrishnegowda <[email protected]> Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]>
…edai#1493) * Bump matplotlib from 3.10.0 to 3.10.1 (#1416) Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.10.0 to 3.10.1. - [Release notes](https://github.com/matplotlib/matplotlib/releases) - [Commits](https://github.com/matplotlib/matplotlib/compare/v3.10.0...v3.10.1) --- updated-dependencies: - dependency-name: matplotlib dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Remove implicit package installs (#1413) * Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chages Signed-off-by: yes <[email protected]> * Adding Secure Aggregation Test (#1419) * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Format issue Signed-off-by: Chaurasiya, Payal <[email protected]> * Put Secure Agg Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * [tutorial] Fix `KeyError: 'w_old'` (#1425) Signed-off-by: Situ Singh <[email protected]> Signed-off-by: yes <[email protected]> * Moving GaNDLF test to pytest framework (#1423) * Upload artifacts for GanDLF Signed-off-by: noopur <[email protected]> * Moving GaNDLF to pytest Signed-off-by: noopur <[email protected]> * Move papermill import isndie relevant function Signed-off-by: noopur <[email protected]> * Add paramiko install in the step Signed-off-by: noopur <[email protected]> * Use model_name with request.config Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Corrected the marker Signed-off-by: noopur <[email protected]> * Corrected the range loop for collab data Signed-off-by: noopur <[email protected]> * Export workspace after certify steps Signed-off-by: noopur <[email protected]> * Make GaNDLF workflow as E2E Signed-off-by: noopur <[email protected]> * Correction in conditions Signed-off-by: noopur <[email protected]> * More checks at pytest level Signed-off-by: noopur <[email protected]> * Added missing import Signed-off-by: noopur <[email protected]> * Debug stmts Signed-off-by: noopur <[email protected]> * Corrected path for config seg file Signed-off-by: noopur <[email protected]> * Corrected curr work dir Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction due to code rebase Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> * Correction in checkout of gandlf step Signed-off-by: noopur <[email protected]> * Final changes in gandlf.yml Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Cleanup (#1422) * DCE, rename variable Signed-off-by: Shah, Karan <[email protected]> * Move to module-level loggers Signed-off-by: Shah, Karan <[email protected]> * Plan.logger to logger Signed-off-by: Shah, Karan <[email protected]> * Fixup Signed-off-by: Shah, Karan <[email protected]> * Also for workspaces Signed-off-by: Shah, Karan <[email protected]> * Fix again for logger Signed-off-by: Shah, Karan <[email protected]> * Remove run_simulation tests Signed-off-by: Shah, Karan <[email protected]> * Header creation is stateless and common across both Signed-off-by: Shah, Karan <[email protected]> * Remove TASK_REGISTRY code; deprecated with interactive Signed-off-by: Shah, Karan <[email protected]> * More use_delta_updates changes Signed-off-by: Shah, Karan <[email protected]> * lets see if this revert works Signed-off-by: Shah, Karan <[email protected]> * Remove openfl install in notebook Signed-off-by: Shah, Karan <[email protected]> * revert few flags Signed-off-by: Shah, Karan <[email protected]> * please please please this should work Signed-off-by: Shah, Karan <[email protected]> * Migrate stateless function to common Signed-off-by: Shah, Karan <[email protected]> * Cleanup outdated docstrings Signed-off-by: Shah, Karan <[email protected]> * Remove Interactive API RPCs Signed-off-by: Shah, Karan <[email protected]> * Remove set devices call; not used anywhere Signed-off-by: Shah, Karan <[email protected]> * Replace checks with inline assertions Signed-off-by: Shah, Karan <[email protected]> * Unfold one-liner functions Signed-off-by: Shah, Karan <[email protected]> * Remove moot tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * B413 bandit issue ignored (#1427) * B413 bandit issue ignored Signed-off-by: yes <[email protected]> * code formatted Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory) fix (#1424) * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * fix(plan reading): use yaml safe_load (#1432) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * [bandit] Ignore B614: Use of unsafe `torch.load` (#1426) * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * testing changes Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * formatted code Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Use Single Function For Logging With Rich Text Handler (#1435) * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * More params to Rich constructor Signed-off-by: noopur <[email protected]> * Do not run straggler Signed-off-by: noopur <[email protected]> * Use formatter for rich Signed-off-by: noopur <[email protected]> * Missed a comma Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * E2E: use rich text handler Signed-off-by: noopur <[email protected]> * Keep formatter with message only Signed-off-by: noopur <[email protected]> * Use logs file Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Retain same function name Signed-off-by: noopur <[email protected]> * Code formatting fixed Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add collaborator name attribute to `AggregatorGRPCClient` (#1436) * Remove explicit collaborator name arg within RPC calls Signed-off-by: Shah, Karan <[email protected]> * Update tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * Bump keras from 3.8.0 to 3.9.0 in /openfl-workspace/keras/2dunet (#1439) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Deleting AggregatorBasedWorkflow tutorials and examples (#1438) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * updated keras (#1447) Signed-off-by: yes <[email protected]> * updated keras (#1446) Signed-off-by: yes <[email protected]> * updated keras (#1445) Signed-off-by: yes <[email protected]> * Bump keras in /openfl-workspace/keras/tensorflow/mnist (#1444) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shailesh Tanwar <[email protected]> Signed-off-by: yes <[email protected]> * Remove the usage of `shell=True` when running commands using `subprocess` (#1443) * fix(bandit b602): remove the usage of shell=True when running commands using subprocess Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): keyword argument name Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): value type in dict Signed-off-by: Pant, Akshay <[email protected]> * fix(dockerize): command args Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Fix typos (#1449) Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: yes <[email protected]> * Upgrading nbdev package version from 2.3.12 to 2.3.37 to fix ImportError (#1456) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E: Remove ANSI escape codes and unwanted patterns from participant log files with LOG_FILE usage (#1455) * Use LOG_FILE for logging to participant log files Signed-off-by: noopur <[email protected]> * Append & for background processes Signed-off-by: noopur <[email protected]> * Do not show date in log files Signed-off-by: noopur <[email protected]> * Param correction Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Using & Signed-off-by: noopur <[email protected]> * Keep bg file running Signed-off-by: noopur <[email protected]> * Upgrade nbdev from 2.3.12 to 2.3.37 Signed-off-by: noopur <[email protected]> * Skip csv from artifact upload Signed-off-by: noopur <[email protected]> * Variable moved up Signed-off-by: noopur <[email protected]> * Move time.sleep before process id check Signed-off-by: noopur <[email protected]> * Remove current round logging Signed-off-by: noopur <[email protected]> * Do not force rich text to terminal Signed-off-by: noopur <[email protected]> * Specific console for conftest Signed-off-by: noopur <[email protected]> * Use single rich handler in conftest Signed-off-by: noopur <[email protected]> * Change log file path for dockerized workspace Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Update `installation.rst` (#1452) * Documentation update to specify: - requirements for python versions compatible with OpenFL - requirements for platforms compatible with the OpenFL docker image Signed-off-by: Lior Malka <[email protected]> * Responding to PR comments Signed-off-by: Lior Malka <[email protected]> * Paraphrasing python supported versions Signed-off-by: Lior Malka <[email protected]> --------- Signed-off-by: Lior Malka <[email protected]> Signed-off-by: yes <[email protected]> * dependency(ray): update to 2.43.0 from 2.9.2 (#1448) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Introducing Task Runner workspace to enable interoperability with Flower workloads (#1433) * enable flwr workspace Signed-off-by: kta-intel <[email protected]> * add tmp dir to minimize how much patching is needed Signed-off-by: kta-intel <[email protected]> * add hash verification to setup data Signed-off-by: kta-intel <[email protected]> * remove patch Signed-off-by: kta-intel <[email protected]> * fix save location Signed-off-by: kta-intel <[email protected]> * docstring Signed-off-by: kta-intel <[email protected]> * remove superfluous edit Signed-off-by: kta-intel <[email protected]> * update readme Signed-off-by: kta-intel <[email protected]> * remove debugger Signed-off-by: kta-intel <[email protected]> * update aggregator client Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * remove todo Signed-off-by: kta-intel <[email protected]> * minor functionality fixes Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update README instructions Signed-off-by: kta-intel <[email protected]> * do not add connector to settings unless connector exists Signed-off-by: kta-intel <[email protected]> * check attribute for connector availability Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * code cleanup Signed-off-by: kta-intel <[email protected]> * fix readme Signed-off-by: kta-intel <[email protected]> * update plan to reflect refactoring Signed-off-by: kta-intel <[email protected]> * grammar fix Signed-off-by: kta-intel <[email protected]> * remove __all__ Signed-off-by: kta-intel <[email protected]> * remove superfluous init Signed-off-by: kta-intel <[email protected]> * change flwr home dir Signed-off-by: kta-intel <[email protected]> * remove patch comment, .sort() added in flwr 1.16 Signed-off-by: kta-intel <[email protected]> * update name for grpc protocols and components Signed-off-by: kta-intel <[email protected]> * remove Connector abc Signed-off-by: kta-intel <[email protected]> * refactoring local grpc to interop Signed-off-by: kta-intel <[email protected]> * import fixes Signed-off-by: kta-intel <[email protected]> * remove duplicate self.callback Signed-off-by: kta-intel <[email protected]> * fix hashes Signed-off-by: kta-intel <[email protected]> * move into workspace for relative path installation Signed-off-by: kta-intel <[email protected]> * update readme to remove connector ABC Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Introduce a default `WaitForAllPolicy` for straggler handling (#1461) * feat(straggler handling): add NoPolicy Signed-off-by: Pant, Akshay <[email protected]> * feat(straggler handling): use NoPolicy as the default Signed-off-by: Pant, Akshay <[email protected]> * restructure(straggler handling): rename NoPolicy to WaitForAllPolicy Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Get current round for non FedEval scenarios (#1463) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * [taskrunner] Secure Aggregation cleanup (#1420) * fix(dependency): remove pycryptodome from openfl setup Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg): removed bootstrap steps from aggregator to helper class Signed-off-by: Pant, Akshay <[email protected]> * restructure(collaborator secagg): move masking step to utility function Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): import utility functions only when used Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): add check for pycrptodome installation when importing Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add future enhancements that are not supported Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): bootstrap class e2e Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): update bootstrap class usage and not masking metrics Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): move masked input vector calculation to collaborator class and remove masking from metrics Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): rename methods more appropriately Signed-off-by: Pant, Akshay <[email protected]> * fix(aggregator): remove duplicate callbacklist initialisation Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg setup): remove unnecesary else conditional block Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add missing word Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): use importlib instead of pkg_resources to check module installation Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): check pycrytodome install Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator): agg function slection Signed-off-by: Pant, Akshay <[email protected]> * fix: remove repeated line Signed-off-by: Pant, Akshay <[email protected]> * feat(plan): add plan verify to check for incompatible features Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add explicit mention for using WaitForAllPolicy with secagg Signed-off-by: Pant, Akshay <[email protected]> * fix(plan): change verify to non-static method Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Added notes for non-TLS in Quickstart documentation (#1462) * Separate section for non-TLS Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Keep non TLS at the top Signed-off-by: noopur <[email protected]> * Optional step Signed-off-by: noopur <[email protected]> * Review comment incorp Signed-off-by: noopur <[email protected]> * Forgot to save the changes earlier Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Security TLS certificate report tests (#1460) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Reduce Aggregator Memory Usage (#1459) * Changes to reduce unnecessary allocations Signed-off-by: Shah, Karan <[email protected]> * Remove compression/decompression for _prepare_trained Signed-off-by: Shah, Karan <[email protected]> * Execute SendLocalTaskResults in a queued call Signed-off-by: Shah, Karan <[email protected]> * Update test with the remove col_name argument Signed-off-by: Shah, Karan <[email protected]> * Use allclose instead of equal Signed-off-by: Shah, Karan <[email protected]> * Revert "Remove compression/decompression for _prepare_trained" This reverts commit cb0adebcf80bc95f47dc2f28c416d2a622bfc039. Signed-off-by: Shah, Karan <[email protected]> * Move lock decorator to common Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chnages Signed-off-by: yes <[email protected]> * code changes as per comments Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * Skipping GaNDLF run from PR pipeline (#1482) * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file (#1479) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Update taskrunner tutorial Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * update README.md and plan.yaml for `flower-app-pytorch` workspace (#1470) * chore: update openfl version to 1.8 (#1464) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1467) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * update README.md Signed-off-by: kta-intel <[email protected]> * 1.8: Raise error if data is not found when running flower-app-pytorch workspace (#1473) * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * raise error if data is not found Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: Akshay Pant <[email protected]> * add fqdn flag to aggregator commands, fix auto_shutdown key, add additional notes to experiment configuration Signed-off-by: kta-intel <[email protected]> * restructure: remove trailing whitespace (#1483) Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Remove additional folder permissioning (#1485) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Issue with Gandlf solved (#1486) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * OpenFL v1.8 release notes (#1487) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * revert core components changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * E2E automation for `flower-app-pytorch` workspace (#1494) * Test Flower automation Signed-off-by: noopur <[email protected]> * Run only once Signed-off-by: noopur <[email protected]> * Added Flower as separate job Signed-off-by: noopur <[email protected]> * Extra closing bracket Signed-off-by: noopur <[email protected]> * Removed dependencies installation step Signed-off-by: noopur <[email protected]> * Correction of step name and logging Signed-off-by: noopur <[email protected]> * New marker for flower model Signed-off-by: noopur <[email protected]> * Separate workflow for Flower Signed-off-by: noopur <[email protected]> * Changes for dockerized ws Signed-off-by: noopur <[email protected]> * Modified test name to reflect flower model Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * 1.8: Address Coverity issues related to `flower-app-pytorch` workspace (#1488) (#1490) * bind socket to a specific interface * add input validation and allow list * address filepath coverity issue by creating a function to check path safety using set of allowed characters * pin upper bound to flwr * formatting --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Hotfix for PQ pipeline (#1497) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * changes as per comments Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * plan updated Signed-off-by: yes <[email protected]> * Added conda installation instructions (#1508) * added publications Signed-off-by: sarthakpati <[email protected]> * added conda installation instructions Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest-asyncio from 0.25.3 to 0.26.0 (#1509) Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.25.3 to 0.26.0. - [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases) - [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.25.3...v0.26.0) --- updated-dependencies: - dependency-name: pytest-asyncio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * add hippmapp3r (#1498) * add hippmapp3r Signed-off-by: porteratzo <[email protected]> * addresed comments Signed-off-by: porteratzo <[email protected]> * add reference Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Signed-off-by: yes <[email protected]> * Bump ruff from 0.9.9 to 0.11.2 (#1515) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Simplify the components' `start` and `stop` processes (#1514) * Run resiliency 3 times Signed-off-by: noopur <[email protected]> * Run 50 rounds Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Upgraded pytest from 8.3.4 to 8.3.5 Signed-off-by: noopur <[email protected]> * Specific functions to fetch pids and kill processes Signed-off-by: noopur <[email protected]> * Multiple changes done Signed-off-by: noopur <[email protected]> * File changes as part of lint fixing Signed-off-by: noopur <[email protected]> * Minor logging fixes and copyright year correction Signed-off-by: noopur <[email protected]> * Review comments addressed Signed-off-by: noopur <[email protected]> * Revert pytest version upgrade change Signed-off-by: noopur <[email protected]> * Check if start_process is present Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest from 8.3.4 to 8.3.5 (#1510) Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.3.4 to 8.3.5. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/8.3.4...8.3.5) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump aquasecurity/trivy-action in the github-actions group (#1512) Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action). Updates `aquasecurity/trivy-action` from 0.29.0 to 0.30.0 - [Release notes](https://github.com/aquasecurity/trivy-action/releases) - [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0) --- updated-dependencies: - dependency-name: aquasecurity/trivy-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Signed-off-by: yes <[email protected]> * Change to add time taken for each round in metrics (#1517) * Change to add time taken for each round in metrics Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * [Workflow Interface] Rename and Refactor WorkspaceExport into NotebookTools (#1364) * Experimental-export-module-refactoring Signed-off-by: refai06 <[email protected]> * Incorporated comments Signed-off-by: refai06 <[email protected]> * Improvements added Signed-off-by: refai06 <[email protected]> * Added testcase Signed-off-by: refai06 <[email protected]> * Code Enhancement Signed-off-by: refai06 <[email protected]> * Docstring & loggers update Signed-off-by: refai06 <[email protected]> * Added NotebookTools module Testcase Signed-off-by: refai06 <[email protected]> * Testcase update Signed-off-by: refai06 <[email protected]> * Review_comments_incorporated Signed-off-by: refai06 <[email protected]> * Fix testcase Signed-off-by: refai06 <[email protected]> * Added review comments and testcase update Signed-off-by: refai06 <[email protected]> * Review comments Signed-off-by: refai06 <[email protected]> * Remove testcase Signed-off-by: refai06 <[email protected]> * Enhance code and incorporate review comments Signed-off-by: refai06 <[email protected]> * Code changes Signed-off-by: refai06 <[email protected]> * revert tutorial change Signed-off-by: refai06 <[email protected]> * Incorporate comments Signed-off-by: refai06 <[email protected]> * Docstring update Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Publish `openfl-nightly` package to `pypi` on daily basis (#1504) * PQ Pipeline: Publish openfl-nightly package to pypi on daily basis Signed-off-by: noopur <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: noopur <[email protected]> * Run Trivy and Bandit on PQ scheduled run Signed-off-by: noopur <[email protected]> * Syntax correction Signed-off-by: noopur <[email protected]> * Added required permissions for Trivy and Bandit Signed-off-by: noopur <[email protected]> * Remove generic permissions Signed-off-by: noopur <[email protected]> * PQ pipeline - run trivy and bandit for workflow_dispatch as well Signed-off-by: noopur <[email protected]> * Use version from setup.pt Signed-off-by: noopur <[email protected]> * Extra files for lint related fixes Signed-off-by: noopur <[email protected]> * Call publish wf from pq with specific commit id Signed-off-by: noopur <[email protected]> * All jobs to run for scheduler from main branch or manual trigger only Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Changes in PQ Pipeline & Publish Package workflows for PYPI_API_TOKEN (#1520) * Added dev environment Signed-off-by: noopur <[email protected]> * Added dev environment Signed-off-by: noopur <[email protected]> * Do not continue on error Signed-off-by: noopur <[email protected]> * Added permission for token read write Signed-off-by: noopur <[email protected]> * Use shell commands Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Changes in PQ pipeline as well Signed-off-by: noopur <[email protected]> * Comment out jobs fopr quick testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass on token Signed-off-by: noopur <[email protected]> * Inherit secret Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Revert the testing changes Signed-off-by: noopur <[email protected]> * Renmote printing job Signed-off-by: noopur <[email protected]> * Run publish step for manual trigger as well Signed-off-by: noopur <[email protected]> * Ignore case for Aggregated model validation score Signed-off-by: noopur <[email protected]> * Read 10 lines Signed-off-by: noopur <[email protected]> * For trusted publishing Signed-off-by: noopur <[email protected]> * Revert previous change Signed-off-by: noopur <[email protected]> * Search string in lowercase Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Collaborator CLI command for pinging the aggregator (#1516) * Adding a collaborator command for pinging the aggregator, without starting any tasks Signed-off-by: Teodor Parvanov <[email protected]> * Addressing review comments Signed-off-by: Teodor Parvanov <[email protected]> * Additional logging for the "fx collaborator ping" command Signed-off-by: Teodor Parvanov <[email protected]> * Additional documentation on the usage of fx collaborator ping Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Handling for .proto files in openfl-nightly package (#1523) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix for openfl-nightly package Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert changes from pre-test Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * Run an E2E test before publishing openfl-nightly package (#1525) * Check wheel file name Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Minor corrections Signed-off-by: noopur <[email protected]> * Github summary step Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Remove extra WHL_FILE assignment Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Verifiable Dataset (#1434) * Verifiable Dataset Signed-off-by: Dar, Efrat <[email protected]> * Pytorch classes for verifaiable dataset loading and verify Signed-off-by: Dar, Efrat <[email protected]> * Move code to openfl.federated.data.sources Signed-off-by: Dar, Efrat <[email protected]> * New validation method (comb of concise and verbose): When create hash, save in memory both root_hash (concise) and all_hashes (verbose) Then when validate the whole dataset, verify against root_hash only, and when validate a single object, verify against relevant hash from all_hashes. When serialize to json, save only root_hash. Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> * Fix: move base_path to LocalDataSource Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Update HippMapp3r plan, requirments and readme (#1524) * fixes Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: yes <[email protected]> * Task callback enhancement / Bugfix (#1469) * task callback support * adjust save model to reflect right tensor key * bump round number after saving model so tasks does not use advanced round number until model is updated * set extra params on the callback object and not function for flexibility based on CR comments * use round as is from persistent store * attempt for lambda filtering to avoid duplicate iterations --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Removed keras/jax/mnist from dockerized ws run (#1532) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * No-op workspace for TaskRunner API (#1527) * Initializing a no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Refactor no-op task runner and data loader to workspace Signed-off-by: Teodor Parvanov <[email protected]> * Fixed copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Workaround for model initialization for the no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Additionally standardizing the copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Adding a README.md Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Add empty requirements.txt to the no-op workspace template (#1536) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Removing the references of federated-evaluation/aggregator.yaml (#1535) Signed-off-by: yes <[email protected]> * undo round number updates (#1537) Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Adding workflow for creating openfl Dependency list (#1530) * create ospdt dependency workflow * dependency list * fixed code format * create ospdt dependency workflow * dependency list * fixed code format * modified filename as release_version-date.xlsx * added format * resolved comments * resolved comments --------- Co-authored-by: Rajith <[email protected]> Signed-off-by: yes <[email protected]> * Run a FedEval testcase in PR pipeline (#1539) * Run FedEval test case in PR pipeline Signed-off-by: noopur <[email protected]> * Added condition for non-draft PR as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Fixing bandit issue (#1540) Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * return from save_model immediately if federation evaluation mode is enabled (#1538) * return from save_model immediately if federation evaluation mode is enabled Signed-off-by: Rahul Garg <[email protected]> * fix the format issue Signed-off-by: Rahul Garg <[email protected]> * Format issue Signed-off-by: Rahul Garg <[email protected]> * Fix the test case and move the location to skip the model save Signed-off-by: Rahul Garg <[email protected]> * remove unnecessary code Signed-off-by: Rahul Garg <[email protected]> * formatting issue fix Signed-off-by: Rahul Garg <[email protected]> * fix linting issue Signed-off-by: Rahul Garg <[email protected]> * addressing comments Signed-off-by: Rahul Garg <[email protected]> --------- Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner FedEval - fetch model score from metric file instead of tensor db (#1543) * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Variable name correction Signed-off-by: noopur <[email protected]> * Consider only numeric value of the line Signed-off-by: noopur <[email protected]> * File converted to json and some lint fixes Signed-off-by: noopur <[email protected]> * Explicit run of test-requirements.txt in GaNDLF Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Load native model with fx plan initialize (#1528) * enable loading of model path in native format Signed-off-by: kta-intel <[email protected]> * add load_native to runner_xgb.py Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * check pbuf file extension Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Minor bug fix to `runner_xgb.py` to properly rebuild model during aggregated model validation (#1529) * fix conditional to rebuild model only during aggregated validation Signed-off-by: kta-intel <[email protected]> * remove print statement Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E - Added missing `needs` for straggler and eden jobs (#1545) * Sort collaborators while displaying Signed-off-by: noopur <[email protected]> * Added needs in straggler and eden jobs Signed-off-by: noopur <[email protected]> * Input selection fix Signed-off-by: noopur <[email protected]> * Syntax fix Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Revert sorting logic Signed-off-by: noopur <[email protected]> * Simplified the no of lines logic Signed-off-by: noopur <[email protected]> * Consider all lines if lesser than the index Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Adding Federation Runtime to Federated Evaluation workflow (#1519) * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Update openfl-tutorials/experimental/workflow/FederatedEvaluation/workspace/MNIST_FederatedEvaluation.ipynb Co-authored-by: Copilot <[email protected]> * added doc string --------- Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: yes <[email protected]> * Documentation: minor correction in `MNIST Example: Workflow API` (#1549) * Minor correction in Workflow API doc Signed-off-by: noopur <[email protected]> * Revert some unintended changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * rename analysis to analytics, moved code to src folder Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * added callback on_round_end for FA Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: yes <[email protected]> Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Situ Singh <[email protected]> Signed-off-by: noopur <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: Lior Malka <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: payalcha <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: porteratzo <[email protected]> Signed-off-by: refai06 <[email protected]> Signed-off-by: Dar, Efrat <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Karan Shah <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Situ Singh <[email protected]> Co-authored-by: Noopur <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]> Co-authored-by: Yaroslav Halchenko <[email protected]> Co-authored-by: Lior Malka <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: refai06 <[email protected]> Co-authored-by: Efrat1 <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: senthil <[email protected]> Co-authored-by: rahulga1 <[email protected]> Co-authored-by: rajithkrishnegowda <[email protected]> Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]>
…er (#1574) * Enhancement in pypi publish (#1567) * Introduct test pypi Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test successful Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix boolean for testpypi Signed-off-by: Chaurasiya, Payal <[email protected]> * End of round callback adjustment (#1568) * fix exp end callback Signed-off-by: Balakrishnan, Senthil <[email protected]> * try invoking e.o.r callback after saving model Signed-off-by: Balakrishnan, Senthil <[email protected]> --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Environment variable can't be treated as boolean Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Co-authored-by: senthil <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Change which can enable Ray for Workflow API and RPC for TaskRunner Signed-off-by: Chaurasiya, Payal <[email protected]> * Change which can enable Ray for Workflow API and RPC for TaskRunner Signed-off-by: Chaurasiya, Payal <[email protected]> * Add logs Signed-off-by: Chaurasiya, Payal <[email protected]> * Edar/datasets s3 (#1513) * Support S3 data sources Signed-off-by: Dar, Efrat <[email protected]> * Restructure datasources Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Verify that one collaborator cannot get tensors from another one. (#1570) Signed-off-by: yuliasherman <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert "End of round callback adjustment (#1568)" (#1573) This reverts commit e848af18a8f5eb75716599c8e4d0adb175c38213. Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Setup Python package in Publish nightly (#1577) * Setup python version Signed-off-by: Chaurasiya, Payal <[email protected]> * Setup python version Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> * Restore "End of round callback adjustment (#1568)" (#1579) Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * add more robust device handling in torch task runner (#1581) Signed-off-by: kta-intel <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * [Task Runner API] [Flower Interoperability] Enable option to run `ClientApp` as a separate process in SGX (#1580) * add clientapp isolation Signed-off-by: kta-intel <[email protected]> * enforce the sequence for spawning and terminating clientapp relative to supernode Signed-off-by: kta-intel <[email protected]> * move is_port_open() check to inside conditional, add a small delay after port is confirmed to be open Signed-off-by: kta-intel <[email protected]> * refactor patch > SGX_enabled Signed-off-by: kta-intel <[email protected]> * refactor local_grpc_server > interop_server Signed-off-by: kta-intel <[email protected]> * format Signed-off-by: kta-intel <[email protected]> * change SGX_enabled to lowercase sgx_enabled for visual consistency Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Retry download of `openfl-nightly` package after its publish (#1586) * Added retry for download package step Signed-off-by: noopur <[email protected]> * Added retry for download package step Signed-off-by: noopur <[email protected]> * Use variables for max attempt and wait time Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Federated Analytics and Histogram Taskrunner Workspace (#1493) * Bump matplotlib from 3.10.0 to 3.10.1 (#1416) Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.10.0 to 3.10.1. - [Release notes](https://github.com/matplotlib/matplotlib/releases) - [Commits](https://github.com/matplotlib/matplotlib/compare/v3.10.0...v3.10.1) --- updated-dependencies: - dependency-name: matplotlib dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Remove implicit package installs (#1413) * Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chages Signed-off-by: yes <[email protected]> * Adding Secure Aggregation Test (#1419) * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Format issue Signed-off-by: Chaurasiya, Payal <[email protected]> * Put Secure Agg Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * [tutorial] Fix `KeyError: 'w_old'` (#1425) Signed-off-by: Situ Singh <[email protected]> Signed-off-by: yes <[email protected]> * Moving GaNDLF test to pytest framework (#1423) * Upload artifacts for GanDLF Signed-off-by: noopur <[email protected]> * Moving GaNDLF to pytest Signed-off-by: noopur <[email protected]> * Move papermill import isndie relevant function Signed-off-by: noopur <[email protected]> * Add paramiko install in the step Signed-off-by: noopur <[email protected]> * Use model_name with request.config Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Corrected the marker Signed-off-by: noopur <[email protected]> * Corrected the range loop for collab data Signed-off-by: noopur <[email protected]> * Export workspace after certify steps Signed-off-by: noopur <[email protected]> * Make GaNDLF workflow as E2E Signed-off-by: noopur <[email protected]> * Correction in conditions Signed-off-by: noopur <[email protected]> * More checks at pytest level Signed-off-by: noopur <[email protected]> * Added missing import Signed-off-by: noopur <[email protected]> * Debug stmts Signed-off-by: noopur <[email protected]> * Corrected path for config seg file Signed-off-by: noopur <[email protected]> * Corrected curr work dir Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction due to code rebase Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> * Correction in checkout of gandlf step Signed-off-by: noopur <[email protected]> * Final changes in gandlf.yml Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Cleanup (#1422) * DCE, rename variable Signed-off-by: Shah, Karan <[email protected]> * Move to module-level loggers Signed-off-by: Shah, Karan <[email protected]> * Plan.logger to logger Signed-off-by: Shah, Karan <[email protected]> * Fixup Signed-off-by: Shah, Karan <[email protected]> * Also for workspaces Signed-off-by: Shah, Karan <[email protected]> * Fix again for logger Signed-off-by: Shah, Karan <[email protected]> * Remove run_simulation tests Signed-off-by: Shah, Karan <[email protected]> * Header creation is stateless and common across both Signed-off-by: Shah, Karan <[email protected]> * Remove TASK_REGISTRY code; deprecated with interactive Signed-off-by: Shah, Karan <[email protected]> * More use_delta_updates changes Signed-off-by: Shah, Karan <[email protected]> * lets see if this revert works Signed-off-by: Shah, Karan <[email protected]> * Remove openfl install in notebook Signed-off-by: Shah, Karan <[email protected]> * revert few flags Signed-off-by: Shah, Karan <[email protected]> * please please please this should work Signed-off-by: Shah, Karan <[email protected]> * Migrate stateless function to common Signed-off-by: Shah, Karan <[email protected]> * Cleanup outdated docstrings Signed-off-by: Shah, Karan <[email protected]> * Remove Interactive API RPCs Signed-off-by: Shah, Karan <[email protected]> * Remove set devices call; not used anywhere Signed-off-by: Shah, Karan <[email protected]> * Replace checks with inline assertions Signed-off-by: Shah, Karan <[email protected]> * Unfold one-liner functions Signed-off-by: Shah, Karan <[email protected]> * Remove moot tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * B413 bandit issue ignored (#1427) * B413 bandit issue ignored Signed-off-by: yes <[email protected]> * code formatted Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory) fix (#1424) * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * fix(plan reading): use yaml safe_load (#1432) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * [bandit] Ignore B614: Use of unsafe `torch.load` (#1426) * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * testing changes Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * formatted code Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Use Single Function For Logging With Rich Text Handler (#1435) * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * More params to Rich constructor Signed-off-by: noopur <[email protected]> * Do not run straggler Signed-off-by: noopur <[email protected]> * Use formatter for rich Signed-off-by: noopur <[email protected]> * Missed a comma Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * E2E: use rich text handler Signed-off-by: noopur <[email protected]> * Keep formatter with message only Signed-off-by: noopur <[email protected]> * Use logs file Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Retain same function name Signed-off-by: noopur <[email protected]> * Code formatting fixed Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add collaborator name attribute to `AggregatorGRPCClient` (#1436) * Remove explicit collaborator name arg within RPC calls Signed-off-by: Shah, Karan <[email protected]> * Update tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * Bump keras from 3.8.0 to 3.9.0 in /openfl-workspace/keras/2dunet (#1439) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Deleting AggregatorBasedWorkflow tutorials and examples (#1438) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * updated keras (#1447) Signed-off-by: yes <[email protected]> * updated keras (#1446) Signed-off-by: yes <[email protected]> * updated keras (#1445) Signed-off-by: yes <[email protected]> * Bump keras in /openfl-workspace/keras/tensorflow/mnist (#1444) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shailesh Tanwar <[email protected]> Signed-off-by: yes <[email protected]> * Remove the usage of `shell=True` when running commands using `subprocess` (#1443) * fix(bandit b602): remove the usage of shell=True when running commands using subprocess Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): keyword argument name Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): value type in dict Signed-off-by: Pant, Akshay <[email protected]> * fix(dockerize): command args Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Fix typos (#1449) Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: yes <[email protected]> * Upgrading nbdev package version from 2.3.12 to 2.3.37 to fix ImportError (#1456) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E: Remove ANSI escape codes and unwanted patterns from participant log files with LOG_FILE usage (#1455) * Use LOG_FILE for logging to participant log files Signed-off-by: noopur <[email protected]> * Append & for background processes Signed-off-by: noopur <[email protected]> * Do not show date in log files Signed-off-by: noopur <[email protected]> * Param correction Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Using & Signed-off-by: noopur <[email protected]> * Keep bg file running Signed-off-by: noopur <[email protected]> * Upgrade nbdev from 2.3.12 to 2.3.37 Signed-off-by: noopur <[email protected]> * Skip csv from artifact upload Signed-off-by: noopur <[email protected]> * Variable moved up Signed-off-by: noopur <[email protected]> * Move time.sleep before process id check Signed-off-by: noopur <[email protected]> * Remove current round logging Signed-off-by: noopur <[email protected]> * Do not force rich text to terminal Signed-off-by: noopur <[email protected]> * Specific console for conftest Signed-off-by: noopur <[email protected]> * Use single rich handler in conftest Signed-off-by: noopur <[email protected]> * Change log file path for dockerized workspace Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Update `installation.rst` (#1452) * Documentation update to specify: - requirements for python versions compatible with OpenFL - requirements for platforms compatible with the OpenFL docker image Signed-off-by: Lior Malka <[email protected]> * Responding to PR comments Signed-off-by: Lior Malka <[email protected]> * Paraphrasing python supported versions Signed-off-by: Lior Malka <[email protected]> --------- Signed-off-by: Lior Malka <[email protected]> Signed-off-by: yes <[email protected]> * dependency(ray): update to 2.43.0 from 2.9.2 (#1448) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Introducing Task Runner workspace to enable interoperability with Flower workloads (#1433) * enable flwr workspace Signed-off-by: kta-intel <[email protected]> * add tmp dir to minimize how much patching is needed Signed-off-by: kta-intel <[email protected]> * add hash verification to setup data Signed-off-by: kta-intel <[email protected]> * remove patch Signed-off-by: kta-intel <[email protected]> * fix save location Signed-off-by: kta-intel <[email protected]> * docstring Signed-off-by: kta-intel <[email protected]> * remove superfluous edit Signed-off-by: kta-intel <[email protected]> * update readme Signed-off-by: kta-intel <[email protected]> * remove debugger Signed-off-by: kta-intel <[email protected]> * update aggregator client Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * remove todo Signed-off-by: kta-intel <[email protected]> * minor functionality fixes Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update README instructions Signed-off-by: kta-intel <[email protected]> * do not add connector to settings unless connector exists Signed-off-by: kta-intel <[email protected]> * check attribute for connector availability Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * code cleanup Signed-off-by: kta-intel <[email protected]> * fix readme Signed-off-by: kta-intel <[email protected]> * update plan to reflect refactoring Signed-off-by: kta-intel <[email protected]> * grammar fix Signed-off-by: kta-intel <[email protected]> * remove __all__ Signed-off-by: kta-intel <[email protected]> * remove superfluous init Signed-off-by: kta-intel <[email protected]> * change flwr home dir Signed-off-by: kta-intel <[email protected]> * remove patch comment, .sort() added in flwr 1.16 Signed-off-by: kta-intel <[email protected]> * update name for grpc protocols and components Signed-off-by: kta-intel <[email protected]> * remove Connector abc Signed-off-by: kta-intel <[email protected]> * refactoring local grpc to interop Signed-off-by: kta-intel <[email protected]> * import fixes Signed-off-by: kta-intel <[email protected]> * remove duplicate self.callback Signed-off-by: kta-intel <[email protected]> * fix hashes Signed-off-by: kta-intel <[email protected]> * move into workspace for relative path installation Signed-off-by: kta-intel <[email protected]> * update readme to remove connector ABC Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Introduce a default `WaitForAllPolicy` for straggler handling (#1461) * feat(straggler handling): add NoPolicy Signed-off-by: Pant, Akshay <[email protected]> * feat(straggler handling): use NoPolicy as the default Signed-off-by: Pant, Akshay <[email protected]> * restructure(straggler handling): rename NoPolicy to WaitForAllPolicy Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Get current round for non FedEval scenarios (#1463) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * [taskrunner] Secure Aggregation cleanup (#1420) * fix(dependency): remove pycryptodome from openfl setup Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg): removed bootstrap steps from aggregator to helper class Signed-off-by: Pant, Akshay <[email protected]> * restructure(collaborator secagg): move masking step to utility function Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): import utility functions only when used Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): add check for pycrptodome installation when importing Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add future enhancements that are not supported Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): bootstrap class e2e Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): update bootstrap class usage and not masking metrics Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): move masked input vector calculation to collaborator class and remove masking from metrics Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): rename methods more appropriately Signed-off-by: Pant, Akshay <[email protected]> * fix(aggregator): remove duplicate callbacklist initialisation Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg setup): remove unnecesary else conditional block Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add missing word Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): use importlib instead of pkg_resources to check module installation Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): check pycrytodome install Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator): agg function slection Signed-off-by: Pant, Akshay <[email protected]> * fix: remove repeated line Signed-off-by: Pant, Akshay <[email protected]> * feat(plan): add plan verify to check for incompatible features Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add explicit mention for using WaitForAllPolicy with secagg Signed-off-by: Pant, Akshay <[email protected]> * fix(plan): change verify to non-static method Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Added notes for non-TLS in Quickstart documentation (#1462) * Separate section for non-TLS Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Keep non TLS at the top Signed-off-by: noopur <[email protected]> * Optional step Signed-off-by: noopur <[email protected]> * Review comment incorp Signed-off-by: noopur <[email protected]> * Forgot to save the changes earlier Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Security TLS certificate report tests (#1460) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Reduce Aggregator Memory Usage (#1459) * Changes to reduce unnecessary allocations Signed-off-by: Shah, Karan <[email protected]> * Remove compression/decompression for _prepare_trained Signed-off-by: Shah, Karan <[email protected]> * Execute SendLocalTaskResults in a queued call Signed-off-by: Shah, Karan <[email protected]> * Update test with the remove col_name argument Signed-off-by: Shah, Karan <[email protected]> * Use allclose instead of equal Signed-off-by: Shah, Karan <[email protected]> * Revert "Remove compression/decompression for _prepare_trained" This reverts commit cb0adebcf80bc95f47dc2f28c416d2a622bfc039. Signed-off-by: Shah, Karan <[email protected]> * Move lock decorator to common Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chnages Signed-off-by: yes <[email protected]> * code changes as per comments Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * Skipping GaNDLF run from PR pipeline (#1482) * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file (#1479) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Update taskrunner tutorial Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * update README.md and plan.yaml for `flower-app-pytorch` workspace (#1470) * chore: update openfl version to 1.8 (#1464) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1467) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * update README.md Signed-off-by: kta-intel <[email protected]> * 1.8: Raise error if data is not found when running flower-app-pytorch workspace (#1473) * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * raise error if data is not found Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: Akshay Pant <[email protected]> * add fqdn flag to aggregator commands, fix auto_shutdown key, add additional notes to experiment configuration Signed-off-by: kta-intel <[email protected]> * restructure: remove trailing whitespace (#1483) Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Remove additional folder permissioning (#1485) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Issue with Gandlf solved (#1486) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * OpenFL v1.8 release notes (#1487) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * revert core components changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * E2E automation for `flower-app-pytorch` workspace (#1494) * Test Flower automation Signed-off-by: noopur <[email protected]> * Run only once Signed-off-by: noopur <[email protected]> * Added Flower as separate job Signed-off-by: noopur <[email protected]> * Extra closing bracket Signed-off-by: noopur <[email protected]> * Removed dependencies installation step Signed-off-by: noopur <[email protected]> * Correction of step name and logging Signed-off-by: noopur <[email protected]> * New marker for flower model Signed-off-by: noopur <[email protected]> * Separate workflow for Flower Signed-off-by: noopur <[email protected]> * Changes for dockerized ws Signed-off-by: noopur <[email protected]> * Modified test name to reflect flower model Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * 1.8: Address Coverity issues related to `flower-app-pytorch` workspace (#1488) (#1490) * bind socket to a specific interface * add input validation and allow list * address filepath coverity issue by creating a function to check path safety using set of allowed characters * pin upper bound to flwr * formatting --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Hotfix for PQ pipeline (#1497) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * changes as per comments Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * plan updated Signed-off-by: yes <[email protected]> * Added conda installation instructions (#1508) * added publications Signed-off-by: sarthakpati <[email protected]> * added conda installation instructions Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest-asyncio from 0.25.3 to 0.26.0 (#1509) Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.25.3 to 0.26.0. - [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases) - [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.25.3...v0.26.0) --- updated-dependencies: - dependency-name: pytest-asyncio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * add hippmapp3r (#1498) * add hippmapp3r Signed-off-by: porteratzo <[email protected]> * addresed comments Signed-off-by: porteratzo <[email protected]> * add reference Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Signed-off-by: yes <[email protected]> * Bump ruff from 0.9.9 to 0.11.2 (#1515) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Simplify the components' `start` and `stop` processes (#1514) * Run resiliency 3 times Signed-off-by: noopur <[email protected]> * Run 50 rounds Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Upgraded pytest from 8.3.4 to 8.3.5 Signed-off-by: noopur <[email protected]> * Specific functions to fetch pids and kill processes Signed-off-by: noopur <[email protected]> * Multiple changes done Signed-off-by: noopur <[email protected]> * File changes as part of lint fixing Signed-off-by: noopur <[email protected]> * Minor logging fixes and copyright year correction Signed-off-by: noopur <[email protected]> * Review comments addressed Signed-off-by: noopur <[email protected]> * Revert pytest version upgrade change Signed-off-by: noopur <[email protected]> * Check if start_process is present Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest from 8.3.4 to 8.3.5 (#1510) Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.3.4 to 8.3.5. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/8.3.4...8.3.5) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump aquasecurity/trivy-action in the github-actions group (#1512) Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action). Updates `aquasecurity/trivy-action` from 0.29.0 to 0.30.0 - [Release notes](https://github.com/aquasecurity/trivy-action/releases) - [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0) --- updated-dependencies: - dependency-name: aquasecurity/trivy-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Signed-off-by: yes <[email protected]> * Change to add time taken for each round in metrics (#1517) * Change to add time taken for each round in metrics Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * [Workflow Interface] Rename and Refactor WorkspaceExport into NotebookTools (#1364) * Experimental-export-module-refactoring Signed-off-by: refai06 <[email protected]> * Incorporated comments Signed-off-by: refai06 <[email protected]> * Improvements added Signed-off-by: refai06 <[email protected]> * Added testcase Signed-off-by: refai06 <[email protected]> * Code Enhancement Signed-off-by: refai06 <[email protected]> * Docstring & loggers update Signed-off-by: refai06 <[email protected]> * Added NotebookTools module Testcase Signed-off-by: refai06 <[email protected]> * Testcase update Signed-off-by: refai06 <[email protected]> * Review_comments_incorporated Signed-off-by: refai06 <[email protected]> * Fix testcase Signed-off-by: refai06 <[email protected]> * Added review comments and testcase update Signed-off-by: refai06 <[email protected]> * Review comments Signed-off-by: refai06 <[email protected]> * Remove testcase Signed-off-by: refai06 <[email protected]> * Enhance code and incorporate review comments Signed-off-by: refai06 <[email protected]> * Code changes Signed-off-by: refai06 <[email protected]> * revert tutorial change Signed-off-by: refai06 <[email protected]> * Incorporate comments Signed-off-by: refai06 <[email protected]> * Docstring update Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Publish `openfl-nightly` package to `pypi` on daily basis (#1504) * PQ Pipeline: Publish openfl-nightly package to pypi on daily basis Signed-off-by: noopur <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: noopur <[email protected]> * Run Trivy and Bandit on PQ scheduled run Signed-off-by: noopur <[email protected]> * Syntax correction Signed-off-by: noopur <[email protected]> * Added required permissions for Trivy and Bandit Signed-off-by: noopur <[email protected]> * Remove generic permissions Signed-off-by: noopur <[email protected]> * PQ pipeline - run trivy and bandit for workflow_dispatch as well Signed-off-by: noopur <[email protected]> * Use version from setup.pt Signed-off-by: noopur <[email protected]> * Extra files for lint related fixes Signed-off-by: noopur <[email protected]> * Call publish wf from pq with specific commit id Signed-off-by: noopur <[email protected]> * All jobs to run for scheduler from main branch or manual trigger only Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Changes in PQ Pipeline & Publish Package workflows for PYPI_API_TOKEN (#1520) * Added dev environment Signed-off-by: noopur <[email protected]> * Added dev environment Signed-off-by: noopur <[email protected]> * Do not continue on error Signed-off-by: noopur <[email protected]> * Added permission for token read write Signed-off-by: noopur <[email protected]> * Use shell commands Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Changes in PQ pipeline as well Signed-off-by: noopur <[email protected]> * Comment out jobs fopr quick testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass on token Signed-off-by: noopur <[email protected]> * Inherit secret Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Revert the testing changes Signed-off-by: noopur <[email protected]> * Renmote printing job Signed-off-by: noopur <[email protected]> * Run publish step for manual trigger as well Signed-off-by: noopur <[email protected]> * Ignore case for Aggregated model validation score Signed-off-by: noopur <[email protected]> * Read 10 lines Signed-off-by: noopur <[email protected]> * For trusted publishing Signed-off-by: noopur <[email protected]> * Revert previous change Signed-off-by: noopur <[email protected]> * Search string in lowercase Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Collaborator CLI command for pinging the aggregator (#1516) * Adding a collaborator command for pinging the aggregator, without starting any tasks Signed-off-by: Teodor Parvanov <[email protected]> * Addressing review comments Signed-off-by: Teodor Parvanov <[email protected]> * Additional logging for the "fx collaborator ping" command Signed-off-by: Teodor Parvanov <[email protected]> * Additional documentation on the usage of fx collaborator ping Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Handling for .proto files in openfl-nightly package (#1523) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix for openfl-nightly package Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert changes from pre-test Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * Run an E2E test before publishing openfl-nightly package (#1525) * Check wheel file name Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Minor corrections Signed-off-by: noopur <[email protected]> * Github summary step Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Remove extra WHL_FILE assignment Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Verifiable Dataset (#1434) * Verifiable Dataset Signed-off-by: Dar, Efrat <[email protected]> * Pytorch classes for verifaiable dataset loading and verify Signed-off-by: Dar, Efrat <[email protected]> * Move code to openfl.federated.data.sources Signed-off-by: Dar, Efrat <[email protected]> * New validation method (comb of concise and verbose): When create hash, save in memory both root_hash (concise) and all_hashes (verbose) Then when validate the whole dataset, verify against root_hash only, and when validate a single object, verify against relevant hash from all_hashes. When serialize to json, save only root_hash. Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> * Fix: move base_path to LocalDataSource Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Update HippMapp3r plan, requirments and readme (#1524) * fixes Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: yes <[email protected]> * Task callback enhancement / Bugfix (#1469) * task callback support * adjust save model to reflect right tensor key * bump round number after saving model so tasks does not use advanced round number until model is updated * set extra params on the callback object and not function for flexibility based on CR comments * use round as is from persistent store * attempt for lambda filtering to avoid duplicate iterations --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Removed keras/jax/mnist from dockerized ws run (#1532) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * No-op workspace for TaskRunner API (#1527) * Initializing a no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Refactor no-op task runner and data loader to workspace Signed-off-by: Teodor Parvanov <[email protected]> * Fixed copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Workaround for model initialization for the no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Additionally standardizing the copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Adding a README.md Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Add empty requirements.txt to the no-op workspace template (#1536) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Removing the references of federated-evaluation/aggregator.yaml (#1535) Signed-off-by: yes <[email protected]> * undo round number updates (#1537) Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Adding workflow for creating openfl Dependency list (#1530) * create ospdt dependency workflow * dependency list * fixed code format * create ospdt dependency workflow * dependency list * fixed code format * modified filename as release_version-date.xlsx * added format * resolved comments * resolved comments --------- Co-authored-by: Rajith <[email protected]> Signed-off-by: yes <[email protected]> * Run a FedEval testcase in PR pipeline (#1539) * Run FedEval test case in PR pipeline Signed-off-by: noopur <[email protected]> * Added condition for non-draft PR as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Fixing bandit issue (#1540) Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * return from save_model immediately if federation evaluation mode is enabled (#1538) * return from save_model immediately if federation evaluation mode is enabled Signed-off-by: Rahul Garg <[email protected]> * fix the format issue Signed-off-by: Rahul Garg <[email protected]> * Format issue Signed-off-by: Rahul Garg <[email protected]> * Fix the test case and move the location to skip the model save Signed-off-by: Rahul Garg <[email protected]> * remove unnecessary code Signed-off-by: Rahul Garg <[email protected]> * formatting issue fix Signed-off-by: Rahul Garg <[email protected]> * fix linting issue Signed-off-by: Rahul Garg <[email protected]> * addressing comments Signed-off-by: Rahul Garg <[email protected]> --------- Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner FedEval - fetch model score from metric file instead of tensor db (#1543) * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Variable name correction Signed-off-by: noopur <[email protected]> * Consider only numeric value of the line Signed-off-by: noopur <[email protected]> * File converted to json and some lint fixes Signed-off-by: noopur <[email protected]> * Explicit run of test-requirements.txt in GaNDLF Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Load native model with fx plan initialize (#1528) * enable loading of model path in native format Signed-off-by: kta-intel <[email protected]> * add load_native to runner_xgb.py Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * check pbuf file extension Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Minor bug fix to `runner_xgb.py` to properly rebuild model during aggregated model validation (#1529) * fix conditional to rebuild model only during aggregated validation Signed-off-by: kta-intel <[email protected]> * remove print statement Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E - Added missing `needs` for straggler and eden jobs (#1545) * Sort collaborators while displaying Signed-off-by: noopur <[email protected]> * Added needs in straggler and eden jobs Signed-off-by: noopur <[email protected]> * Input selection fix Signed-off-by: noopur <[email protected]> * Syntax fix Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Revert sorting logic Signed-off-by: noopur <[email protected]> * Simplified the no of lines logic Signed-off-by: noopur <[email protected]> * Consider all lines if lesser than the index Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Adding Federation Runtime to Federated Evaluation workflow (#1519) * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Update openfl-tutorials/experimental/workflow/FederatedEvaluation/workspace/MNIST_FederatedEvaluation.ipynb Co-authored-by: Copilot <[email protected]> * added doc string --------- Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: yes <[email protected]> * Documentation: minor correction in `MNIST Example: Workflow API` (#1549) * Minor correction in Workflow API doc Signed-off-by: noopur <[email protected]> * Revert some unintended changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * rename analysis to analytics, moved code to src folder Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * added callback on_round_end for FA Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: yes <[email protected]> Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Situ Singh <[email protected]> Signed-off-by: noopur <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: Lior Malka <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: payalcha <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: porteratzo <[email protected]> Signed-off-by: refai06 <[email protected]> Signed-off-by: Dar, Efrat <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Karan Shah <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Situ Singh <[email protected]> Co-authored-by: Noopur <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]> Co-authored-by: Yaroslav Halchenko <[email protected]> Co-authored-by: Lior Malka <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: refai06 <[email protected]> Co-authored-by: Efrat1 <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: senthil <[email protected]> Co-authored-by: rahulga1 <[email protected]> Co-authored-by: rajithkrishnegowda <[email protected]> Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]…
…edai#1493) * Bump matplotlib from 3.10.0 to 3.10.1 (#1416) Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.10.0 to 3.10.1. - [Release notes](https://github.com/matplotlib/matplotlib/releases) - [Commits](https://github.com/matplotlib/matplotlib/compare/v3.10.0...v3.10.1) --- updated-dependencies: - dependency-name: matplotlib dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Remove implicit package installs (#1413) * Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chages Signed-off-by: yes <[email protected]> * Adding Secure Aggregation Test (#1419) * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Format issue Signed-off-by: Chaurasiya, Payal <[email protected]> * Put Secure Agg Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * [tutorial] Fix `KeyError: 'w_old'` (#1425) Signed-off-by: Situ Singh <[email protected]> Signed-off-by: yes <[email protected]> * Moving GaNDLF test to pytest framework (#1423) * Upload artifacts for GanDLF Signed-off-by: noopur <[email protected]> * Moving GaNDLF to pytest Signed-off-by: noopur <[email protected]> * Move papermill import isndie relevant function Signed-off-by: noopur <[email protected]> * Add paramiko install in the step Signed-off-by: noopur <[email protected]> * Use model_name with request.config Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Corrected the marker Signed-off-by: noopur <[email protected]> * Corrected the range loop for collab data Signed-off-by: noopur <[email protected]> * Export workspace after certify steps Signed-off-by: noopur <[email protected]> * Make GaNDLF workflow as E2E Signed-off-by: noopur <[email protected]> * Correction in conditions Signed-off-by: noopur <[email protected]> * More checks at pytest level Signed-off-by: noopur <[email protected]> * Added missing import Signed-off-by: noopur <[email protected]> * Debug stmts Signed-off-by: noopur <[email protected]> * Corrected path for config seg file Signed-off-by: noopur <[email protected]> * Corrected curr work dir Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction due to code rebase Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> * Correction in checkout of gandlf step Signed-off-by: noopur <[email protected]> * Final changes in gandlf.yml Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Cleanup (#1422) * DCE, rename variable Signed-off-by: Shah, Karan <[email protected]> * Move to module-level loggers Signed-off-by: Shah, Karan <[email protected]> * Plan.logger to logger Signed-off-by: Shah, Karan <[email protected]> * Fixup Signed-off-by: Shah, Karan <[email protected]> * Also for workspaces Signed-off-by: Shah, Karan <[email protected]> * Fix again for logger Signed-off-by: Shah, Karan <[email protected]> * Remove run_simulation tests Signed-off-by: Shah, Karan <[email protected]> * Header creation is stateless and common across both Signed-off-by: Shah, Karan <[email protected]> * Remove TASK_REGISTRY code; deprecated with interactive Signed-off-by: Shah, Karan <[email protected]> * More use_delta_updates changes Signed-off-by: Shah, Karan <[email protected]> * lets see if this revert works Signed-off-by: Shah, Karan <[email protected]> * Remove openfl install in notebook Signed-off-by: Shah, Karan <[email protected]> * revert few flags Signed-off-by: Shah, Karan <[email protected]> * please please please this should work Signed-off-by: Shah, Karan <[email protected]> * Migrate stateless function to common Signed-off-by: Shah, Karan <[email protected]> * Cleanup outdated docstrings Signed-off-by: Shah, Karan <[email protected]> * Remove Interactive API RPCs Signed-off-by: Shah, Karan <[email protected]> * Remove set devices call; not used anywhere Signed-off-by: Shah, Karan <[email protected]> * Replace checks with inline assertions Signed-off-by: Shah, Karan <[email protected]> * Unfold one-liner functions Signed-off-by: Shah, Karan <[email protected]> * Remove moot tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * B413 bandit issue ignored (#1427) * B413 bandit issue ignored Signed-off-by: yes <[email protected]> * code formatted Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory) fix (#1424) * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * fix(plan reading): use yaml safe_load (#1432) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * [bandit] Ignore B614: Use of unsafe `torch.load` (#1426) * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * testing changes Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * formatted code Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Use Single Function For Logging With Rich Text Handler (#1435) * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * More params to Rich constructor Signed-off-by: noopur <[email protected]> * Do not run straggler Signed-off-by: noopur <[email protected]> * Use formatter for rich Signed-off-by: noopur <[email protected]> * Missed a comma Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * E2E: use rich text handler Signed-off-by: noopur <[email protected]> * Keep formatter with message only Signed-off-by: noopur <[email protected]> * Use logs file Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Retain same function name Signed-off-by: noopur <[email protected]> * Code formatting fixed Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add collaborator name attribute to `AggregatorGRPCClient` (#1436) * Remove explicit collaborator name arg within RPC calls Signed-off-by: Shah, Karan <[email protected]> * Update tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * Bump keras from 3.8.0 to 3.9.0 in /openfl-workspace/keras/2dunet (#1439) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Deleting AggregatorBasedWorkflow tutorials and examples (#1438) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * updated keras (#1447) Signed-off-by: yes <[email protected]> * updated keras (#1446) Signed-off-by: yes <[email protected]> * updated keras (#1445) Signed-off-by: yes <[email protected]> * Bump keras in /openfl-workspace/keras/tensorflow/mnist (#1444) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shailesh Tanwar <[email protected]> Signed-off-by: yes <[email protected]> * Remove the usage of `shell=True` when running commands using `subprocess` (#1443) * fix(bandit b602): remove the usage of shell=True when running commands using subprocess Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): keyword argument name Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): value type in dict Signed-off-by: Pant, Akshay <[email protected]> * fix(dockerize): command args Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Fix typos (#1449) Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: yes <[email protected]> * Upgrading nbdev package version from 2.3.12 to 2.3.37 to fix ImportError (#1456) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E: Remove ANSI escape codes and unwanted patterns from participant log files with LOG_FILE usage (#1455) * Use LOG_FILE for logging to participant log files Signed-off-by: noopur <[email protected]> * Append & for background processes Signed-off-by: noopur <[email protected]> * Do not show date in log files Signed-off-by: noopur <[email protected]> * Param correction Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Using & Signed-off-by: noopur <[email protected]> * Keep bg file running Signed-off-by: noopur <[email protected]> * Upgrade nbdev from 2.3.12 to 2.3.37 Signed-off-by: noopur <[email protected]> * Skip csv from artifact upload Signed-off-by: noopur <[email protected]> * Variable moved up Signed-off-by: noopur <[email protected]> * Move time.sleep before process id check Signed-off-by: noopur <[email protected]> * Remove current round logging Signed-off-by: noopur <[email protected]> * Do not force rich text to terminal Signed-off-by: noopur <[email protected]> * Specific console for conftest Signed-off-by: noopur <[email protected]> * Use single rich handler in conftest Signed-off-by: noopur <[email protected]> * Change log file path for dockerized workspace Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Update `installation.rst` (#1452) * Documentation update to specify: - requirements for python versions compatible with OpenFL - requirements for platforms compatible with the OpenFL docker image Signed-off-by: Lior Malka <[email protected]> * Responding to PR comments Signed-off-by: Lior Malka <[email protected]> * Paraphrasing python supported versions Signed-off-by: Lior Malka <[email protected]> --------- Signed-off-by: Lior Malka <[email protected]> Signed-off-by: yes <[email protected]> * dependency(ray): update to 2.43.0 from 2.9.2 (#1448) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Introducing Task Runner workspace to enable interoperability with Flower workloads (#1433) * enable flwr workspace Signed-off-by: kta-intel <[email protected]> * add tmp dir to minimize how much patching is needed Signed-off-by: kta-intel <[email protected]> * add hash verification to setup data Signed-off-by: kta-intel <[email protected]> * remove patch Signed-off-by: kta-intel <[email protected]> * fix save location Signed-off-by: kta-intel <[email protected]> * docstring Signed-off-by: kta-intel <[email protected]> * remove superfluous edit Signed-off-by: kta-intel <[email protected]> * update readme Signed-off-by: kta-intel <[email protected]> * remove debugger Signed-off-by: kta-intel <[email protected]> * update aggregator client Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * remove todo Signed-off-by: kta-intel <[email protected]> * minor functionality fixes Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update README instructions Signed-off-by: kta-intel <[email protected]> * do not add connector to settings unless connector exists Signed-off-by: kta-intel <[email protected]> * check attribute for connector availability Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * code cleanup Signed-off-by: kta-intel <[email protected]> * fix readme Signed-off-by: kta-intel <[email protected]> * update plan to reflect refactoring Signed-off-by: kta-intel <[email protected]> * grammar fix Signed-off-by: kta-intel <[email protected]> * remove __all__ Signed-off-by: kta-intel <[email protected]> * remove superfluous init Signed-off-by: kta-intel <[email protected]> * change flwr home dir Signed-off-by: kta-intel <[email protected]> * remove patch comment, .sort() added in flwr 1.16 Signed-off-by: kta-intel <[email protected]> * update name for grpc protocols and components Signed-off-by: kta-intel <[email protected]> * remove Connector abc Signed-off-by: kta-intel <[email protected]> * refactoring local grpc to interop Signed-off-by: kta-intel <[email protected]> * import fixes Signed-off-by: kta-intel <[email protected]> * remove duplicate self.callback Signed-off-by: kta-intel <[email protected]> * fix hashes Signed-off-by: kta-intel <[email protected]> * move into workspace for relative path installation Signed-off-by: kta-intel <[email protected]> * update readme to remove connector ABC Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Introduce a default `WaitForAllPolicy` for straggler handling (#1461) * feat(straggler handling): add NoPolicy Signed-off-by: Pant, Akshay <[email protected]> * feat(straggler handling): use NoPolicy as the default Signed-off-by: Pant, Akshay <[email protected]> * restructure(straggler handling): rename NoPolicy to WaitForAllPolicy Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Get current round for non FedEval scenarios (#1463) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * [taskrunner] Secure Aggregation cleanup (#1420) * fix(dependency): remove pycryptodome from openfl setup Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg): removed bootstrap steps from aggregator to helper class Signed-off-by: Pant, Akshay <[email protected]> * restructure(collaborator secagg): move masking step to utility function Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): import utility functions only when used Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): add check for pycrptodome installation when importing Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add future enhancements that are not supported Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): bootstrap class e2e Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): update bootstrap class usage and not masking metrics Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): move masked input vector calculation to collaborator class and remove masking from metrics Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): rename methods more appropriately Signed-off-by: Pant, Akshay <[email protected]> * fix(aggregator): remove duplicate callbacklist initialisation Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg setup): remove unnecesary else conditional block Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add missing word Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): use importlib instead of pkg_resources to check module installation Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): check pycrytodome install Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator): agg function slection Signed-off-by: Pant, Akshay <[email protected]> * fix: remove repeated line Signed-off-by: Pant, Akshay <[email protected]> * feat(plan): add plan verify to check for incompatible features Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add explicit mention for using WaitForAllPolicy with secagg Signed-off-by: Pant, Akshay <[email protected]> * fix(plan): change verify to non-static method Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Added notes for non-TLS in Quickstart documentation (#1462) * Separate section for non-TLS Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Keep non TLS at the top Signed-off-by: noopur <[email protected]> * Optional step Signed-off-by: noopur <[email protected]> * Review comment incorp Signed-off-by: noopur <[email protected]> * Forgot to save the changes earlier Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Security TLS certificate report tests (#1460) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Reduce Aggregator Memory Usage (#1459) * Changes to reduce unnecessary allocations Signed-off-by: Shah, Karan <[email protected]> * Remove compression/decompression for _prepare_trained Signed-off-by: Shah, Karan <[email protected]> * Execute SendLocalTaskResults in a queued call Signed-off-by: Shah, Karan <[email protected]> * Update test with the remove col_name argument Signed-off-by: Shah, Karan <[email protected]> * Use allclose instead of equal Signed-off-by: Shah, Karan <[email protected]> * Revert "Remove compression/decompression for _prepare_trained" This reverts commit cb0adebcf80bc95f47dc2f28c416d2a622bfc039. Signed-off-by: Shah, Karan <[email protected]> * Move lock decorator to common Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chnages Signed-off-by: yes <[email protected]> * code changes as per comments Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * Skipping GaNDLF run from PR pipeline (#1482) * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file (#1479) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Update taskrunner tutorial Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * update README.md and plan.yaml for `flower-app-pytorch` workspace (#1470) * chore: update openfl version to 1.8 (#1464) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1467) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * update README.md Signed-off-by: kta-intel <[email protected]> * 1.8: Raise error if data is not found when running flower-app-pytorch workspace (#1473) * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * raise error if data is not found Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: Akshay Pant <[email protected]> * add fqdn flag to aggregator commands, fix auto_shutdown key, add additional notes to experiment configuration Signed-off-by: kta-intel <[email protected]> * restructure: remove trailing whitespace (#1483) Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Remove additional folder permissioning (#1485) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Issue with Gandlf solved (#1486) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * OpenFL v1.8 release notes (#1487) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * revert core components changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * E2E automation for `flower-app-pytorch` workspace (#1494) * Test Flower automation Signed-off-by: noopur <[email protected]> * Run only once Signed-off-by: noopur <[email protected]> * Added Flower as separate job Signed-off-by: noopur <[email protected]> * Extra closing bracket Signed-off-by: noopur <[email protected]> * Removed dependencies installation step Signed-off-by: noopur <[email protected]> * Correction of step name and logging Signed-off-by: noopur <[email protected]> * New marker for flower model Signed-off-by: noopur <[email protected]> * Separate workflow for Flower Signed-off-by: noopur <[email protected]> * Changes for dockerized ws Signed-off-by: noopur <[email protected]> * Modified test name to reflect flower model Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * 1.8: Address Coverity issues related to `flower-app-pytorch` workspace (#1488) (#1490) * bind socket to a specific interface * add input validation and allow list * address filepath coverity issue by creating a function to check path safety using set of allowed characters * pin upper bound to flwr * formatting --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Hotfix for PQ pipeline (#1497) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * changes as per comments Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * plan updated Signed-off-by: yes <[email protected]> * Added conda installation instructions (#1508) * added publications Signed-off-by: sarthakpati <[email protected]> * added conda installation instructions Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest-asyncio from 0.25.3 to 0.26.0 (#1509) Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.25.3 to 0.26.0. - [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases) - [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.25.3...v0.26.0) --- updated-dependencies: - dependency-name: pytest-asyncio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * add hippmapp3r (#1498) * add hippmapp3r Signed-off-by: porteratzo <[email protected]> * addresed comments Signed-off-by: porteratzo <[email protected]> * add reference Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Signed-off-by: yes <[email protected]> * Bump ruff from 0.9.9 to 0.11.2 (#1515) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Simplify the components' `start` and `stop` processes (#1514) * Run resiliency 3 times Signed-off-by: noopur <[email protected]> * Run 50 rounds Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Upgraded pytest from 8.3.4 to 8.3.5 Signed-off-by: noopur <[email protected]> * Specific functions to fetch pids and kill processes Signed-off-by: noopur <[email protected]> * Multiple changes done Signed-off-by: noopur <[email protected]> * File changes as part of lint fixing Signed-off-by: noopur <[email protected]> * Minor logging fixes and copyright year correction Signed-off-by: noopur <[email protected]> * Review comments addressed Signed-off-by: noopur <[email protected]> * Revert pytest version upgrade change Signed-off-by: noopur <[email protected]> * Check if start_process is present Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest from 8.3.4 to 8.3.5 (#1510) Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.3.4 to 8.3.5. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/8.3.4...8.3.5) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump aquasecurity/trivy-action in the github-actions group (#1512) Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action). Updates `aquasecurity/trivy-action` from 0.29.0 to 0.30.0 - [Release notes](https://github.com/aquasecurity/trivy-action/releases) - [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0) --- updated-dependencies: - dependency-name: aquasecurity/trivy-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Signed-off-by: yes <[email protected]> * Change to add time taken for each round in metrics (#1517) * Change to add time taken for each round in metrics Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * [Workflow Interface] Rename and Refactor WorkspaceExport into NotebookTools (#1364) * Experimental-export-module-refactoring Signed-off-by: refai06 <[email protected]> * Incorporated comments Signed-off-by: refai06 <[email protected]> * Improvements added Signed-off-by: refai06 <[email protected]> * Added testcase Signed-off-by: refai06 <[email protected]> * Code Enhancement Signed-off-by: refai06 <[email protected]> * Docstring & loggers update Signed-off-by: refai06 <[email protected]> * Added NotebookTools module Testcase Signed-off-by: refai06 <[email protected]> * Testcase update Signed-off-by: refai06 <[email protected]> * Review_comments_incorporated Signed-off-by: refai06 <[email protected]> * Fix testcase Signed-off-by: refai06 <[email protected]> * Added review comments and testcase update Signed-off-by: refai06 <[email protected]> * Review comments Signed-off-by: refai06 <[email protected]> * Remove testcase Signed-off-by: refai06 <[email protected]> * Enhance code and incorporate review comments Signed-off-by: refai06 <[email protected]> * Code changes Signed-off-by: refai06 <[email protected]> * revert tutorial change Signed-off-by: refai06 <[email protected]> * Incorporate comments Signed-off-by: refai06 <[email protected]> * Docstring update Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Publish `openfl-nightly` package to `pypi` on daily basis (#1504) * PQ Pipeline: Publish openfl-nightly package to pypi on daily basis Signed-off-by: noopur <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: noopur <[email protected]> * Run Trivy and Bandit on PQ scheduled run Signed-off-by: noopur <[email protected]> * Syntax correction Signed-off-by: noopur <[email protected]> * Added required permissions for Trivy and Bandit Signed-off-by: noopur <[email protected]> * Remove generic permissions Signed-off-by: noopur <[email protected]> * PQ pipeline - run trivy and bandit for workflow_dispatch as well Signed-off-by: noopur <[email protected]> * Use version from setup.pt Signed-off-by: noopur <[email protected]> * Extra files for lint related fixes Signed-off-by: noopur <[email protected]> * Call publish wf from pq with specific commit id Signed-off-by: noopur <[email protected]> * All jobs to run for scheduler from main branch or manual trigger only Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Changes in PQ Pipeline & Publish Package workflows for PYPI_API_TOKEN (#1520) * Added dev environment Signed-off-by: noopur <[email protected]> * Added dev environment Signed-off-by: noopur <[email protected]> * Do not continue on error Signed-off-by: noopur <[email protected]> * Added permission for token read write Signed-off-by: noopur <[email protected]> * Use shell commands Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Changes in PQ pipeline as well Signed-off-by: noopur <[email protected]> * Comment out jobs fopr quick testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass on token Signed-off-by: noopur <[email protected]> * Inherit secret Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Revert the testing changes Signed-off-by: noopur <[email protected]> * Renmote printing job Signed-off-by: noopur <[email protected]> * Run publish step for manual trigger as well Signed-off-by: noopur <[email protected]> * Ignore case for Aggregated model validation score Signed-off-by: noopur <[email protected]> * Read 10 lines Signed-off-by: noopur <[email protected]> * For trusted publishing Signed-off-by: noopur <[email protected]> * Revert previous change Signed-off-by: noopur <[email protected]> * Search string in lowercase Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Collaborator CLI command for pinging the aggregator (#1516) * Adding a collaborator command for pinging the aggregator, without starting any tasks Signed-off-by: Teodor Parvanov <[email protected]> * Addressing review comments Signed-off-by: Teodor Parvanov <[email protected]> * Additional logging for the "fx collaborator ping" command Signed-off-by: Teodor Parvanov <[email protected]> * Additional documentation on the usage of fx collaborator ping Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Handling for .proto files in openfl-nightly package (#1523) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix for openfl-nightly package Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert changes from pre-test Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * Run an E2E test before publishing openfl-nightly package (#1525) * Check wheel file name Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Minor corrections Signed-off-by: noopur <[email protected]> * Github summary step Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Remove extra WHL_FILE assignment Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Verifiable Dataset (#1434) * Verifiable Dataset Signed-off-by: Dar, Efrat <[email protected]> * Pytorch classes for verifaiable dataset loading and verify Signed-off-by: Dar, Efrat <[email protected]> * Move code to openfl.federated.data.sources Signed-off-by: Dar, Efrat <[email protected]> * New validation method (comb of concise and verbose): When create hash, save in memory both root_hash (concise) and all_hashes (verbose) Then when validate the whole dataset, verify against root_hash only, and when validate a single object, verify against relevant hash from all_hashes. When serialize to json, save only root_hash. Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> * Fix: move base_path to LocalDataSource Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Update HippMapp3r plan, requirments and readme (#1524) * fixes Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: yes <[email protected]> * Task callback enhancement / Bugfix (#1469) * task callback support * adjust save model to reflect right tensor key * bump round number after saving model so tasks does not use advanced round number until model is updated * set extra params on the callback object and not function for flexibility based on CR comments * use round as is from persistent store * attempt for lambda filtering to avoid duplicate iterations --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Removed keras/jax/mnist from dockerized ws run (#1532) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * No-op workspace for TaskRunner API (#1527) * Initializing a no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Refactor no-op task runner and data loader to workspace Signed-off-by: Teodor Parvanov <[email protected]> * Fixed copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Workaround for model initialization for the no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Additionally standardizing the copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Adding a README.md Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Add empty requirements.txt to the no-op workspace template (#1536) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Removing the references of federated-evaluation/aggregator.yaml (#1535) Signed-off-by: yes <[email protected]> * undo round number updates (#1537) Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Adding workflow for creating openfl Dependency list (#1530) * create ospdt dependency workflow * dependency list * fixed code format * create ospdt dependency workflow * dependency list * fixed code format * modified filename as release_version-date.xlsx * added format * resolved comments * resolved comments --------- Co-authored-by: Rajith <[email protected]> Signed-off-by: yes <[email protected]> * Run a FedEval testcase in PR pipeline (#1539) * Run FedEval test case in PR pipeline Signed-off-by: noopur <[email protected]> * Added condition for non-draft PR as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Fixing bandit issue (#1540) Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * return from save_model immediately if federation evaluation mode is enabled (#1538) * return from save_model immediately if federation evaluation mode is enabled Signed-off-by: Rahul Garg <[email protected]> * fix the format issue Signed-off-by: Rahul Garg <[email protected]> * Format issue Signed-off-by: Rahul Garg <[email protected]> * Fix the test case and move the location to skip the model save Signed-off-by: Rahul Garg <[email protected]> * remove unnecessary code Signed-off-by: Rahul Garg <[email protected]> * formatting issue fix Signed-off-by: Rahul Garg <[email protected]> * fix linting issue Signed-off-by: Rahul Garg <[email protected]> * addressing comments Signed-off-by: Rahul Garg <[email protected]> --------- Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner FedEval - fetch model score from metric file instead of tensor db (#1543) * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Variable name correction Signed-off-by: noopur <[email protected]> * Consider only numeric value of the line Signed-off-by: noopur <[email protected]> * File converted to json and some lint fixes Signed-off-by: noopur <[email protected]> * Explicit run of test-requirements.txt in GaNDLF Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Load native model with fx plan initialize (#1528) * enable loading of model path in native format Signed-off-by: kta-intel <[email protected]> * add load_native to runner_xgb.py Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * check pbuf file extension Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Minor bug fix to `runner_xgb.py` to properly rebuild model during aggregated model validation (#1529) * fix conditional to rebuild model only during aggregated validation Signed-off-by: kta-intel <[email protected]> * remove print statement Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E - Added missing `needs` for straggler and eden jobs (#1545) * Sort collaborators while displaying Signed-off-by: noopur <[email protected]> * Added needs in straggler and eden jobs Signed-off-by: noopur <[email protected]> * Input selection fix Signed-off-by: noopur <[email protected]> * Syntax fix Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Revert sorting logic Signed-off-by: noopur <[email protected]> * Simplified the no of lines logic Signed-off-by: noopur <[email protected]> * Consider all lines if lesser than the index Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Adding Federation Runtime to Federated Evaluation workflow (#1519) * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Update openfl-tutorials/experimental/workflow/FederatedEvaluation/workspace/MNIST_FederatedEvaluation.ipynb Co-authored-by: Copilot <[email protected]> * added doc string --------- Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: yes <[email protected]> * Documentation: minor correction in `MNIST Example: Workflow API` (#1549) * Minor correction in Workflow API doc Signed-off-by: noopur <[email protected]> * Revert some unintended changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * rename analysis to analytics, moved code to src folder Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * added callback on_round_end for FA Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: yes <[email protected]> Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Situ Singh <[email protected]> Signed-off-by: noopur <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: Lior Malka <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: payalcha <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: porteratzo <[email protected]> Signed-off-by: refai06 <[email protected]> Signed-off-by: Dar, Efrat <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Karan Shah <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Situ Singh <[email protected]> Co-authored-by: Noopur <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]> Co-authored-by: Yaroslav Halchenko <[email protected]> Co-authored-by: Lior Malka <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: refai06 <[email protected]> Co-authored-by: Efrat1 <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: senthil <[email protected]> Co-authored-by: rahulga1 <[email protected]> Co-authored-by: rajithkrishnegowda <[email protected]> Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: Tayfun Ceylan <tayfun.ceylan>
…er (securefederatedai#1574) * Enhancement in pypi publish (#1567) * Introduct test pypi Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * only for test, will revert Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test it Signed-off-by: Chaurasiya, Payal <[email protected]> * test successful Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * write seperate script Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Add script instead of inline changes Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix boolean for testpypi Signed-off-by: Chaurasiya, Payal <[email protected]> * End of round callback adjustment (#1568) * fix exp end callback Signed-off-by: Balakrishnan, Senthil <[email protected]> * try invoking e.o.r callback after saving model Signed-off-by: Balakrishnan, Senthil <[email protected]> --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Environment variable can't be treated as boolean Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Co-authored-by: senthil <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Change which can enable Ray for Workflow API and RPC for TaskRunner Signed-off-by: Chaurasiya, Payal <[email protected]> * Change which can enable Ray for Workflow API and RPC for TaskRunner Signed-off-by: Chaurasiya, Payal <[email protected]> * Add logs Signed-off-by: Chaurasiya, Payal <[email protected]> * Edar/datasets s3 (#1513) * Support S3 data sources Signed-off-by: Dar, Efrat <[email protected]> * Restructure datasources Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Verify that one collaborator cannot get tensors from another one. (#1570) Signed-off-by: yuliasherman <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert "End of round callback adjustment (#1568)" (#1573) This reverts commit e848af18a8f5eb75716599c8e4d0adb175c38213. Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Setup Python package in Publish nightly (#1577) * Setup python version Signed-off-by: Chaurasiya, Payal <[email protected]> * Setup python version Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> * Restore "End of round callback adjustment (#1568)" (#1579) Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * add more robust device handling in torch task runner (#1581) Signed-off-by: kta-intel <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * [Task Runner API] [Flower Interoperability] Enable option to run `ClientApp` as a separate process in SGX (#1580) * add clientapp isolation Signed-off-by: kta-intel <[email protected]> * enforce the sequence for spawning and terminating clientapp relative to supernode Signed-off-by: kta-intel <[email protected]> * move is_port_open() check to inside conditional, add a small delay after port is confirmed to be open Signed-off-by: kta-intel <[email protected]> * refactor patch > SGX_enabled Signed-off-by: kta-intel <[email protected]> * refactor local_grpc_server > interop_server Signed-off-by: kta-intel <[email protected]> * format Signed-off-by: kta-intel <[email protected]> * change SGX_enabled to lowercase sgx_enabled for visual consistency Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Retry download of `openfl-nightly` package after its publish (#1586) * Added retry for download package step Signed-off-by: noopur <[email protected]> * Added retry for download package step Signed-off-by: noopur <[email protected]> * Use variables for max attempt and wait time Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> * Federated Analytics and Histogram Taskrunner Workspace (#1493) * Bump matplotlib from 3.10.0 to 3.10.1 (#1416) Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.10.0 to 3.10.1. - [Release notes](https://github.com/matplotlib/matplotlib/releases) - [Commits](https://github.com/matplotlib/matplotlib/compare/v3.10.0...v3.10.1) --- updated-dependencies: - dependency-name: matplotlib dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Remove implicit package installs (#1413) * Remove implicit package installs Signed-off-by: Shah, Karan <[email protected]> * Update CI Signed-off-by: Shah, Karan <[email protected]> * Install pkg cmd for tests Signed-off-by: Shah, Karan <[email protected]> * Patch gandlf test Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chages Signed-off-by: yes <[email protected]> * Adding Secure Aggregation Test (#1419) * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Secure Aggregation Tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Handle success and exception messages Signed-off-by: Chaurasiya, Payal <[email protected]> * Format issue Signed-off-by: Chaurasiya, Payal <[email protected]> * Put Secure Agg Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * [tutorial] Fix `KeyError: 'w_old'` (#1425) Signed-off-by: Situ Singh <[email protected]> Signed-off-by: yes <[email protected]> * Moving GaNDLF test to pytest framework (#1423) * Upload artifacts for GanDLF Signed-off-by: noopur <[email protected]> * Moving GaNDLF to pytest Signed-off-by: noopur <[email protected]> * Move papermill import isndie relevant function Signed-off-by: noopur <[email protected]> * Add paramiko install in the step Signed-off-by: noopur <[email protected]> * Use model_name with request.config Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Corrected the marker Signed-off-by: noopur <[email protected]> * Corrected the range loop for collab data Signed-off-by: noopur <[email protected]> * Export workspace after certify steps Signed-off-by: noopur <[email protected]> * Make GaNDLF workflow as E2E Signed-off-by: noopur <[email protected]> * Correction in conditions Signed-off-by: noopur <[email protected]> * More checks at pytest level Signed-off-by: noopur <[email protected]> * Added missing import Signed-off-by: noopur <[email protected]> * Debug stmts Signed-off-by: noopur <[email protected]> * Corrected path for config seg file Signed-off-by: noopur <[email protected]> * Corrected curr work dir Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction due to code rebase Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> * Correction in checkout of gandlf step Signed-off-by: noopur <[email protected]> * Final changes in gandlf.yml Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Cleanup (#1422) * DCE, rename variable Signed-off-by: Shah, Karan <[email protected]> * Move to module-level loggers Signed-off-by: Shah, Karan <[email protected]> * Plan.logger to logger Signed-off-by: Shah, Karan <[email protected]> * Fixup Signed-off-by: Shah, Karan <[email protected]> * Also for workspaces Signed-off-by: Shah, Karan <[email protected]> * Fix again for logger Signed-off-by: Shah, Karan <[email protected]> * Remove run_simulation tests Signed-off-by: Shah, Karan <[email protected]> * Header creation is stateless and common across both Signed-off-by: Shah, Karan <[email protected]> * Remove TASK_REGISTRY code; deprecated with interactive Signed-off-by: Shah, Karan <[email protected]> * More use_delta_updates changes Signed-off-by: Shah, Karan <[email protected]> * lets see if this revert works Signed-off-by: Shah, Karan <[email protected]> * Remove openfl install in notebook Signed-off-by: Shah, Karan <[email protected]> * revert few flags Signed-off-by: Shah, Karan <[email protected]> * please please please this should work Signed-off-by: Shah, Karan <[email protected]> * Migrate stateless function to common Signed-off-by: Shah, Karan <[email protected]> * Cleanup outdated docstrings Signed-off-by: Shah, Karan <[email protected]> * Remove Interactive API RPCs Signed-off-by: Shah, Karan <[email protected]> * Remove set devices call; not used anywhere Signed-off-by: Shah, Karan <[email protected]> * Replace checks with inline assertions Signed-off-by: Shah, Karan <[email protected]> * Unfold one-liner functions Signed-off-by: Shah, Karan <[email protected]> * Remove moot tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * B413 bandit issue ignored (#1427) * B413 bandit issue ignored Signed-off-by: yes <[email protected]> * code formatted Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory) fix (#1424) * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * Bandit B108 medium issue(Probable insecure usage of temp file/directory.) Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> * dummy commit Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * fix(plan reading): use yaml safe_load (#1432) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * [bandit] Ignore B614: Use of unsafe `torch.load` (#1426) * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * added safetensors==0.5.3 Signed-off-by: yes <[email protected]> * testing changes Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * B614 Use of unsafe PyTorch load fix Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * formatted code Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> * revert changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]> * Use Single Function For Logging With Rich Text Handler (#1435) * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * Use rich handler for e2e logger Signed-off-by: noopur <[email protected]> * More params to Rich constructor Signed-off-by: noopur <[email protected]> * Do not run straggler Signed-off-by: noopur <[email protected]> * Use formatter for rich Signed-off-by: noopur <[email protected]> * Missed a comma Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * E2E: use rich text handler Signed-off-by: noopur <[email protected]> * Keep formatter with message only Signed-off-by: noopur <[email protected]> * Use logs file Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Retain same function name Signed-off-by: noopur <[email protected]> * Code formatting fixed Signed-off-by: noopur <[email protected]> * Review comments incorporated Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add collaborator name attribute to `AggregatorGRPCClient` (#1436) * Remove explicit collaborator name arg within RPC calls Signed-off-by: Shah, Karan <[email protected]> * Update tests Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * Bump keras from 3.8.0 to 3.9.0 in /openfl-workspace/keras/2dunet (#1439) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yes <[email protected]> * Deleting AggregatorBasedWorkflow tutorials and examples (#1438) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * updated keras (#1447) Signed-off-by: yes <[email protected]> * updated keras (#1446) Signed-off-by: yes <[email protected]> * updated keras (#1445) Signed-off-by: yes <[email protected]> * Bump keras in /openfl-workspace/keras/tensorflow/mnist (#1444) Bumps [keras](https://github.com/keras-team/keras) from 3.8.0 to 3.9.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](https://github.com/keras-team/keras/compare/v3.8.0...v3.9.0) --- updated-dependencies: - dependency-name: keras dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shailesh Tanwar <[email protected]> Signed-off-by: yes <[email protected]> * Remove the usage of `shell=True` when running commands using `subprocess` (#1443) * fix(bandit b602): remove the usage of shell=True when running commands using subprocess Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): keyword argument name Signed-off-by: Pant, Akshay <[email protected]> * fix(execute): value type in dict Signed-off-by: Pant, Akshay <[email protected]> * fix(dockerize): command args Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Fix typos (#1449) Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: yes <[email protected]> * Upgrading nbdev package version from 2.3.12 to 2.3.37 to fix ImportError (#1456) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E: Remove ANSI escape codes and unwanted patterns from participant log files with LOG_FILE usage (#1455) * Use LOG_FILE for logging to participant log files Signed-off-by: noopur <[email protected]> * Append & for background processes Signed-off-by: noopur <[email protected]> * Do not show date in log files Signed-off-by: noopur <[email protected]> * Param correction Signed-off-by: noopur <[email protected]> * More changes Signed-off-by: noopur <[email protected]> * Using & Signed-off-by: noopur <[email protected]> * Keep bg file running Signed-off-by: noopur <[email protected]> * Upgrade nbdev from 2.3.12 to 2.3.37 Signed-off-by: noopur <[email protected]> * Skip csv from artifact upload Signed-off-by: noopur <[email protected]> * Variable moved up Signed-off-by: noopur <[email protected]> * Move time.sleep before process id check Signed-off-by: noopur <[email protected]> * Remove current round logging Signed-off-by: noopur <[email protected]> * Do not force rich text to terminal Signed-off-by: noopur <[email protected]> * Specific console for conftest Signed-off-by: noopur <[email protected]> * Use single rich handler in conftest Signed-off-by: noopur <[email protected]> * Change log file path for dockerized workspace Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Update `installation.rst` (#1452) * Documentation update to specify: - requirements for python versions compatible with OpenFL - requirements for platforms compatible with the OpenFL docker image Signed-off-by: Lior Malka <[email protected]> * Responding to PR comments Signed-off-by: Lior Malka <[email protected]> * Paraphrasing python supported versions Signed-off-by: Lior Malka <[email protected]> --------- Signed-off-by: Lior Malka <[email protected]> Signed-off-by: yes <[email protected]> * dependency(ray): update to 2.43.0 from 2.9.2 (#1448) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Introducing Task Runner workspace to enable interoperability with Flower workloads (#1433) * enable flwr workspace Signed-off-by: kta-intel <[email protected]> * add tmp dir to minimize how much patching is needed Signed-off-by: kta-intel <[email protected]> * add hash verification to setup data Signed-off-by: kta-intel <[email protected]> * remove patch Signed-off-by: kta-intel <[email protected]> * fix save location Signed-off-by: kta-intel <[email protected]> * docstring Signed-off-by: kta-intel <[email protected]> * remove superfluous edit Signed-off-by: kta-intel <[email protected]> * update readme Signed-off-by: kta-intel <[email protected]> * remove debugger Signed-off-by: kta-intel <[email protected]> * update aggregator client Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * remove todo Signed-off-by: kta-intel <[email protected]> * minor functionality fixes Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update docstrings Signed-off-by: kta-intel <[email protected]> * update README instructions Signed-off-by: kta-intel <[email protected]> * do not add connector to settings unless connector exists Signed-off-by: kta-intel <[email protected]> * check attribute for connector availability Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * code cleanup Signed-off-by: kta-intel <[email protected]> * fix readme Signed-off-by: kta-intel <[email protected]> * update plan to reflect refactoring Signed-off-by: kta-intel <[email protected]> * grammar fix Signed-off-by: kta-intel <[email protected]> * remove __all__ Signed-off-by: kta-intel <[email protected]> * remove superfluous init Signed-off-by: kta-intel <[email protected]> * change flwr home dir Signed-off-by: kta-intel <[email protected]> * remove patch comment, .sort() added in flwr 1.16 Signed-off-by: kta-intel <[email protected]> * update name for grpc protocols and components Signed-off-by: kta-intel <[email protected]> * remove Connector abc Signed-off-by: kta-intel <[email protected]> * refactoring local grpc to interop Signed-off-by: kta-intel <[email protected]> * import fixes Signed-off-by: kta-intel <[email protected]> * remove duplicate self.callback Signed-off-by: kta-intel <[email protected]> * fix hashes Signed-off-by: kta-intel <[email protected]> * move into workspace for relative path installation Signed-off-by: kta-intel <[email protected]> * update readme to remove connector ABC Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Introduce a default `WaitForAllPolicy` for straggler handling (#1461) * feat(straggler handling): add NoPolicy Signed-off-by: Pant, Akshay <[email protected]> * feat(straggler handling): use NoPolicy as the default Signed-off-by: Pant, Akshay <[email protected]> * restructure(straggler handling): rename NoPolicy to WaitForAllPolicy Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * Get current round for non FedEval scenarios (#1463) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * [taskrunner] Secure Aggregation cleanup (#1420) * fix(dependency): remove pycryptodome from openfl setup Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg): removed bootstrap steps from aggregator to helper class Signed-off-by: Pant, Akshay <[email protected]> * restructure(collaborator secagg): move masking step to utility function Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): import utility functions only when used Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): add check for pycrptodome installation when importing Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add future enhancements that are not supported Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): bootstrap class e2e Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): update bootstrap class usage and not masking metrics Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): move masked input vector calculation to collaborator class and remove masking from metrics Signed-off-by: Pant, Akshay <[email protected]> * restructure(secagg): rename methods more appropriately Signed-off-by: Pant, Akshay <[email protected]> * fix(aggregator): remove duplicate callbacklist initialisation Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator secagg setup): remove unnecesary else conditional block Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add missing word Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): use importlib instead of pkg_resources to check module installation Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): check pycrytodome install Signed-off-by: Pant, Akshay <[email protected]> * restructure(aggregator): agg function slection Signed-off-by: Pant, Akshay <[email protected]> * fix: remove repeated line Signed-off-by: Pant, Akshay <[email protected]> * feat(plan): add plan verify to check for incompatible features Signed-off-by: Pant, Akshay <[email protected]> * docs(secagg): add explicit mention for using WaitForAllPolicy with secagg Signed-off-by: Pant, Akshay <[email protected]> * fix(plan): change verify to non-static method Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Added notes for non-TLS in Quickstart documentation (#1462) * Separate section for non-TLS Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Keep non TLS at the top Signed-off-by: noopur <[email protected]> * Optional step Signed-off-by: noopur <[email protected]> * Review comment incorp Signed-off-by: noopur <[email protected]> * Forgot to save the changes earlier Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Security TLS certificate report tests (#1460) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * formatting fix Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * Reduce Aggregator Memory Usage (#1459) * Changes to reduce unnecessary allocations Signed-off-by: Shah, Karan <[email protected]> * Remove compression/decompression for _prepare_trained Signed-off-by: Shah, Karan <[email protected]> * Execute SendLocalTaskResults in a queued call Signed-off-by: Shah, Karan <[email protected]> * Update test with the remove col_name argument Signed-off-by: Shah, Karan <[email protected]> * Use allclose instead of equal Signed-off-by: Shah, Karan <[email protected]> * Revert "Remove compression/decompression for _prepare_trained" This reverts commit cb0adebcf80bc95f47dc2f28c416d2a622bfc039. Signed-off-by: Shah, Karan <[email protected]> * Move lock decorator to common Signed-off-by: Shah, Karan <[email protected]> --------- Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: yes <[email protected]> * code chnages Signed-off-by: yes <[email protected]> * code changes as per comments Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * Skipping GaNDLF run from PR pipeline (#1482) * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> * setuptools upgrade pr Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file (#1479) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Update taskrunner tutorial Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * reverted to torch Signed-off-by: yes <[email protected]> * update README.md and plan.yaml for `flower-app-pytorch` workspace (#1470) * chore: update openfl version to 1.8 (#1464) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1467) Signed-off-by: Pant, Akshay <[email protected]> * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * update README.md Signed-off-by: kta-intel <[email protected]> * 1.8: Raise error if data is not found when running flower-app-pytorch workspace (#1473) * fix(secagg): correct module name to import module (#1466) Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> * raise error if data is not found Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: Akshay Pant <[email protected]> * add fqdn flag to aggregator commands, fix auto_shutdown key, add additional notes to experiment configuration Signed-off-by: kta-intel <[email protected]> * restructure: remove trailing whitespace (#1483) Signed-off-by: Pant, Akshay <[email protected]> --------- Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Remove additional folder permissioning (#1485) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Issue with Gandlf solved (#1486) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * OpenFL v1.8 release notes (#1487) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * revert core components changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * E2E automation for `flower-app-pytorch` workspace (#1494) * Test Flower automation Signed-off-by: noopur <[email protected]> * Run only once Signed-off-by: noopur <[email protected]> * Added Flower as separate job Signed-off-by: noopur <[email protected]> * Extra closing bracket Signed-off-by: noopur <[email protected]> * Removed dependencies installation step Signed-off-by: noopur <[email protected]> * Correction of step name and logging Signed-off-by: noopur <[email protected]> * New marker for flower model Signed-off-by: noopur <[email protected]> * Separate workflow for Flower Signed-off-by: noopur <[email protected]> * Changes for dockerized ws Signed-off-by: noopur <[email protected]> * Modified test name to reflect flower model Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * 1.8: Address Coverity issues related to `flower-app-pytorch` workspace (#1488) (#1490) * bind socket to a specific interface * add input validation and allow list * address filepath coverity issue by creating a function to check path safety using set of allowed characters * pin upper bound to flwr * formatting --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: yes <[email protected]> * Hotfix for PQ pipeline (#1497) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * changes as per comments Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * plan updated Signed-off-by: yes <[email protected]> * Added conda installation instructions (#1508) * added publications Signed-off-by: sarthakpati <[email protected]> * added conda installation instructions Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest-asyncio from 0.25.3 to 0.26.0 (#1509) Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.25.3 to 0.26.0. - [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases) - [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.25.3...v0.26.0) --- updated-dependencies: - dependency-name: pytest-asyncio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * add hippmapp3r (#1498) * add hippmapp3r Signed-off-by: porteratzo <[email protected]> * addresed comments Signed-off-by: porteratzo <[email protected]> * add reference Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Signed-off-by: yes <[email protected]> * Bump ruff from 0.9.9 to 0.11.2 (#1515) Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * Task Runner E2E: Simplify the components' `start` and `stop` processes (#1514) * Run resiliency 3 times Signed-off-by: noopur <[email protected]> * Run 50 rounds Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Extra debugging Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Remove extra logging Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Kill process as a one-liner Signed-off-by: noopur <[email protected]> * Upgraded pytest from 8.3.4 to 8.3.5 Signed-off-by: noopur <[email protected]> * Specific functions to fetch pids and kill processes Signed-off-by: noopur <[email protected]> * Multiple changes done Signed-off-by: noopur <[email protected]> * File changes as part of lint fixing Signed-off-by: noopur <[email protected]> * Minor logging fixes and copyright year correction Signed-off-by: noopur <[email protected]> * Review comments addressed Signed-off-by: noopur <[email protected]> * Revert pytest version upgrade change Signed-off-by: noopur <[email protected]> * Check if start_process is present Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump pytest from 8.3.4 to 8.3.5 (#1510) Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.3.4 to 8.3.5. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/8.3.4...8.3.5) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Noopur <[email protected]> Signed-off-by: yes <[email protected]> * Bump aquasecurity/trivy-action in the github-actions group (#1512) Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action). Updates `aquasecurity/trivy-action` from 0.29.0 to 0.30.0 - [Release notes](https://github.com/aquasecurity/trivy-action/releases) - [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0) --- updated-dependencies: - dependency-name: aquasecurity/trivy-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Signed-off-by: yes <[email protected]> * Change to add time taken for each round in metrics (#1517) * Change to add time taken for each round in metrics Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: yes <[email protected]> * [Workflow Interface] Rename and Refactor WorkspaceExport into NotebookTools (#1364) * Experimental-export-module-refactoring Signed-off-by: refai06 <[email protected]> * Incorporated comments Signed-off-by: refai06 <[email protected]> * Improvements added Signed-off-by: refai06 <[email protected]> * Added testcase Signed-off-by: refai06 <[email protected]> * Code Enhancement Signed-off-by: refai06 <[email protected]> * Docstring & loggers update Signed-off-by: refai06 <[email protected]> * Added NotebookTools module Testcase Signed-off-by: refai06 <[email protected]> * Testcase update Signed-off-by: refai06 <[email protected]> * Review_comments_incorporated Signed-off-by: refai06 <[email protected]> * Fix testcase Signed-off-by: refai06 <[email protected]> * Added review comments and testcase update Signed-off-by: refai06 <[email protected]> * Review comments Signed-off-by: refai06 <[email protected]> * Remove testcase Signed-off-by: refai06 <[email protected]> * Enhance code and incorporate review comments Signed-off-by: refai06 <[email protected]> * Code changes Signed-off-by: refai06 <[email protected]> * revert tutorial change Signed-off-by: refai06 <[email protected]> * Incorporate comments Signed-off-by: refai06 <[email protected]> * Docstring update Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]> Signed-off-by: yes <[email protected]> * PQ Pipeline: Publish `openfl-nightly` package to `pypi` on daily basis (#1504) * PQ Pipeline: Publish openfl-nightly package to pypi on daily basis Signed-off-by: noopur <[email protected]> * Add trufflehog to scan secrets in repo and logs file (#1495) * Add trufflehog to scan secrets in repo and logs file Signed-off-by: payalcha <[email protected]> * title fix Signed-off-by: payalcha <[email protected]> * Review comments Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: payalcha <[email protected]> Signed-off-by: noopur <[email protected]> * Correction Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev (#1503) * Change openfl version from 1.8 to 1.9.0-dev Signed-off-by: noopur <[email protected]> * Change openfl version from 1.8 to 1.9.0.dev Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> * PQ Pipeline: Ensure usage of same commit id across workflow jobs (#1501) * Initial changes for testing Signed-off-by: noopur <[email protected]> * Initial changes for testing Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * All workflows modified Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Correction of job name Signed-off-by: noopur <[email protected]> * Testing one workflow Signed-off-by: noopur <[email protected]> * Final set of changes Signed-off-by: noopur <[email protected]> * Correct the repo URL Signed-off-by: noopur <[email protected]> * Only commit related changes Signed-off-by: noopur <[email protected]> * Correction in trufflehog job name Signed-off-by: noopur <[email protected]> * Added commit_id to SSL wf after rebase Signed-off-by: noopur <[email protected]> * Modified version file as well Signed-off-by: noopur <[email protected]> * Make commit_id optional input Signed-off-by: noopur <[email protected]> * Job name added for Trufflehog Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: noopur <[email protected]> * Run Trivy and Bandit on PQ scheduled run Signed-off-by: noopur <[email protected]> * Syntax correction Signed-off-by: noopur <[email protected]> * Added required permissions for Trivy and Bandit Signed-off-by: noopur <[email protected]> * Remove generic permissions Signed-off-by: noopur <[email protected]> * PQ pipeline - run trivy and bandit for workflow_dispatch as well Signed-off-by: noopur <[email protected]> * Use version from setup.pt Signed-off-by: noopur <[email protected]> * Extra files for lint related fixes Signed-off-by: noopur <[email protected]> * Call publish wf from pq with specific commit id Signed-off-by: noopur <[email protected]> * All jobs to run for scheduler from main branch or manual trigger only Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: payalcha <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Changes in PQ Pipeline & Publish Package workflows for PYPI_API_TOKEN (#1520) * Added dev environment Signed-off-by: noopur <[email protected]> * Added dev environment Signed-off-by: noopur <[email protected]> * Do not continue on error Signed-off-by: noopur <[email protected]> * Added permission for token read write Signed-off-by: noopur <[email protected]> * Use shell commands Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Testing Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Changes in PQ pipeline as well Signed-off-by: noopur <[email protected]> * Comment out jobs fopr quick testing Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass on token Signed-off-by: noopur <[email protected]> * Inherit secret Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Do not pass env from called wf Signed-off-by: noopur <[email protected]> * Revert the testing changes Signed-off-by: noopur <[email protected]> * Renmote printing job Signed-off-by: noopur <[email protected]> * Run publish step for manual trigger as well Signed-off-by: noopur <[email protected]> * Ignore case for Aggregated model validation score Signed-off-by: noopur <[email protected]> * Read 10 lines Signed-off-by: noopur <[email protected]> * For trusted publishing Signed-off-by: noopur <[email protected]> * Revert previous change Signed-off-by: noopur <[email protected]> * Search string in lowercase Signed-off-by: noopur <[email protected]> * Review comments incorp Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Collaborator CLI command for pinging the aggregator (#1516) * Adding a collaborator command for pinging the aggregator, without starting any tasks Signed-off-by: Teodor Parvanov <[email protected]> * Addressing review comments Signed-off-by: Teodor Parvanov <[email protected]> * Additional logging for the "fx collaborator ping" command Signed-off-by: Teodor Parvanov <[email protected]> * Additional documentation on the usage of fx collaborator ping Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Handling for .proto files in openfl-nightly package (#1523) * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Taking latest nbdev as previous giving error Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * Security TLS certificate report tests Signed-off-by: Chaurasiya, Payal <[email protected]> * FedEval fix Signed-off-by: Chaurasiya, Payal <[email protected]> * Small change Signed-off-by: Chaurasiya, Payal <[email protected]> * Logs correction Signed-off-by: Chaurasiya, Payal <[email protected]> * Format changes Signed-off-by: Chaurasiya, Payal <[email protected]> * To streamline Collaborator yaml for torch/mnist with all other cols.yaml file Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * change tp fpdf2 from fpdf Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Fix for openfl-nightly package Signed-off-by: Chaurasiya, Payal <[email protected]> * Revert changes from pre-test Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * Run an E2E test before publishing openfl-nightly package (#1525) * Check wheel file name Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Added test case run Signed-off-by: noopur <[email protected]> * Minor corrections Signed-off-by: noopur <[email protected]> * Github summary step Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Remove extra WHL_FILE assignment Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Verifiable Dataset (#1434) * Verifiable Dataset Signed-off-by: Dar, Efrat <[email protected]> * Pytorch classes for verifaiable dataset loading and verify Signed-off-by: Dar, Efrat <[email protected]> * Move code to openfl.federated.data.sources Signed-off-by: Dar, Efrat <[email protected]> * New validation method (comb of concise and verbose): When create hash, save in memory both root_hash (concise) and all_hashes (verbose) Then when validate the whole dataset, verify against root_hash only, and when validate a single object, verify against relevant hash from all_hashes. When serialize to json, save only root_hash. Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> * Fix: move base_path to LocalDataSource Signed-off-by: Dar, Efrat <[email protected]> * Fixes Signed-off-by: Dar, Efrat <[email protected]> --------- Signed-off-by: Dar, Efrat <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Update HippMapp3r plan, requirments and readme (#1524) * fixes Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> * Update openfl-workspace/keras/hippmapp3r/README.md Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: porteratzo <[email protected]> --------- Signed-off-by: porteratzo <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: yes <[email protected]> * Task callback enhancement / Bugfix (#1469) * task callback support * adjust save model to reflect right tensor key * bump round number after saving model so tasks does not use advanced round number until model is updated * set extra params on the callback object and not function for flexibility based on CR comments * use round as is from persistent store * attempt for lambda filtering to avoid duplicate iterations --------- Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Removed keras/jax/mnist from dockerized ws run (#1532) Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * No-op workspace for TaskRunner API (#1527) * Initializing a no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Refactor no-op task runner and data loader to workspace Signed-off-by: Teodor Parvanov <[email protected]> * Fixed copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Workaround for model initialization for the no-op workspace Signed-off-by: Teodor Parvanov <[email protected]> * Additionally standardizing the copyright headers Signed-off-by: Teodor Parvanov <[email protected]> * Adding a README.md Signed-off-by: Teodor Parvanov <[email protected]> --------- Signed-off-by: Teodor Parvanov <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * Add empty requirements.txt to the no-op workspace template (#1536) Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: yes <[email protected]> * Removing the references of federated-evaluation/aggregator.yaml (#1535) Signed-off-by: yes <[email protected]> * undo round number updates (#1537) Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: yes <[email protected]> * Adding workflow for creating openfl Dependency list (#1530) * create ospdt dependency workflow * dependency list * fixed code format * create ospdt dependency workflow * dependency list * fixed code format * modified filename as release_version-date.xlsx * added format * resolved comments * resolved comments --------- Co-authored-by: Rajith <[email protected]> Signed-off-by: yes <[email protected]> * Run a FedEval testcase in PR pipeline (#1539) * Run FedEval test case in PR pipeline Signed-off-by: noopur <[email protected]> * Added condition for non-draft PR as well Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Fixing bandit issue (#1540) Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: yes <[email protected]> * return from save_model immediately if federation evaluation mode is enabled (#1538) * return from save_model immediately if federation evaluation mode is enabled Signed-off-by: Rahul Garg <[email protected]> * fix the format issue Signed-off-by: Rahul Garg <[email protected]> * Format issue Signed-off-by: Rahul Garg <[email protected]> * Fix the test case and move the location to skip the model save Signed-off-by: Rahul Garg <[email protected]> * remove unnecessary code Signed-off-by: Rahul Garg <[email protected]> * formatting issue fix Signed-off-by: Rahul Garg <[email protected]> * fix linting issue Signed-off-by: Rahul Garg <[email protected]> * addressing comments Signed-off-by: Rahul Garg <[email protected]> --------- Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner FedEval - fetch model score from metric file instead of tensor db (#1543) * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Read accuracy from metric file for FedEval Signed-off-by: noopur <[email protected]> * Variable name correction Signed-off-by: noopur <[email protected]> * Consider only numeric value of the line Signed-off-by: noopur <[email protected]> * File converted to json and some lint fixes Signed-off-by: noopur <[email protected]> * Explicit run of test-requirements.txt in GaNDLF Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Load native model with fx plan initialize (#1528) * enable loading of model path in native format Signed-off-by: kta-intel <[email protected]> * add load_native to runner_xgb.py Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * check pbuf file extension Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * Minor bug fix to `runner_xgb.py` to properly rebuild model during aggregated model validation (#1529) * fix conditional to rebuild model only during aggregated validation Signed-off-by: kta-intel <[email protected]> * remove print statement Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: teoparvanov <[email protected]> Signed-off-by: yes <[email protected]> * TaskRunner E2E - Added missing `needs` for straggler and eden jobs (#1545) * Sort collaborators while displaying Signed-off-by: noopur <[email protected]> * Added needs in straggler and eden jobs Signed-off-by: noopur <[email protected]> * Input selection fix Signed-off-by: noopur <[email protected]> * Syntax fix Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Check last few lines based on no of collabs Signed-off-by: noopur <[email protected]> * Revert sorting logic Signed-off-by: noopur <[email protected]> * Simplified the no of lines logic Signed-off-by: noopur <[email protected]> * Consider all lines if lesser than the index Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * Adding Federation Runtime to Federated Evaluation workflow (#1519) * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Adding Federation Runtime to Federated Evaulation workflow * removed MNIST_FedEvaluation * Update openfl-tutorials/experimental/workflow/FederatedEvaluation/workspace/MNIST_FederatedEvaluation.ipynb Co-authored-by: Copilot <[email protected]> * added doc string --------- Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: yes <[email protected]> * Documentation: minor correction in `MNIST Example: Workflow API` (#1549) * Minor correction in Workflow API doc Signed-off-by: noopur <[email protected]> * Revert some unintended changes Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]> Signed-off-by: yes <[email protected]> * rename analysis to analytics, moved code to src folder Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * changes to remove keras model Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * added callback on_round_end for FA Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: yes <[email protected]> Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: Situ Singh <[email protected]> Signed-off-by: noopur <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Teodor Parvanov <[email protected]> Signed-off-by: Pant, Akshay <[email protected]> Signed-off-by: Yaroslav Halchenko <[email protected]> Signed-off-by: Lior Malka <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: payalcha <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: porteratzo <[email protected]> Signed-off-by: refai06 <[email protected]> Signed-off-by: Dar, Efrat <[email protected]> Signed-off-by: Balakrishnan, Senthil <[email protected]> Signed-off-by: Rahul Garg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Karan Shah <[email protected]> Co-authored-by: Payal Chaurasiya <[email protected]> Co-authored-by: Situ Singh <[email protected]> Co-authored-by: Noopur <[email protected]> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]> Co-authored-by: Yaroslav Halchenko <[email protected]> Co-authored-by: Lior Malka <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: refai06 <[email protected]> Co-authored-by: Efrat1 <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: senthil <[email protected]> Co-authored-by: rahulga1 <[email protected]> Co-authored-by: rajithkrishnegowda <[email protected]> Co-authored-by: Rajith <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-…
This PR moves package management to the user, instead of forcing a pip install during
fx workspace createandfx plan initialize.Reasons:
pipis not necessarily a package manager of choice.Tests: CI.
Misc: This may impact OpenFL-Security launch script. It will be modified accordingly post finalization of the changes here.
Additional context: #73
Documentation: Link
This is part 1 (of n). In an upcoming PR,
data.yamlwill be removed in favor of directly providing--data-pathduring collaborator start.