Issue: the package can't be installed and imported correctly in colab

[brandonHaipas]

Description

When running the !pip install harmonydata cell, this error is displayed:

After that, when I try to import the package this error is dsiplayed:

Environment

Google colab, python 3.11

How to Reproduce

1. Open the harmony walkthrouh notebook in colab.
2. Run the first three cells.
3. Check the output of the cells.

Expected Behaviour

Installing and importing without hassle.

[kunjcr2]

Let me take a look at it !

[brandonHaipas]

I created a fork where I solved this! Also, the minimum python version should be updated to 3.10 due to type notation in file src/matching/matcher.py file at line 148

[kunjcr2]

Gimme a minute
Im just forking it

[kunjcr2]

Ohh shoot
I need to use an older kernal
Cuz newwer ones doesnt suppport Tensorflow

[kunjcr2]

albumentations 2.0.5 requires pydantic>=2.9.2, but you have pydantic 2.8.2 which is incompatible.

harmonydata 1.0.3 requires pydantic==2.8.2; python_version <= "3.13", but you have pydantic 2.9.2 which is incompatible.

You may want to take a look at this
This is creating a basically dependency hell

And of course, I am not sure about how to exactly get out of it
Maybe try switching to some alternatives of this albumentation

[brandonHaipas]

I did it at this fork! I'm running the tests in my machine as we speak to check everything is working fine. The notebook is working as expected but I don't know if my machine will be able to run every test since it's a potato (it has 4GB of RAM).

[kunjcr2]

Ohh shoot
That's something you need to keep in mind
My memory was getting full slowly when I tried running it

Be careful
Try GPU or even TPU on google colab

[brandonHaipas]

Tests worked on my machine! (conda env with python 3.11)

[woodthom2]

thanks so much guys @brandonHaipas @kunjcr2 !!! I really appreciate you taking the time to try this out. Do we need to update anything in the core library? I am unclear what the conclusion is of your investigation.

[brandonHaipas]

Hey @woodthom2 ! I'm sorry for the really late reply. Basically there's a dependency and version error for some features. For instance, for pydantic you're using a notation supported only from python 3.10 onwards, here's the link to pydantic documentation. On the other hand, python 3.13 wasn't a stable version yet, so certain libraries like scipy were broken.

In summary:

• Pydantic has a different type notation for python versions below 3.10.
• There were issues with python 3.13 version dependencies that are not related to your work. I'll test if the package is working now after the bug fixes.

In order to check which versions work for the entire package, it might be best to not specify the version of the packages that harmony relies on, and then do a pip freeze to check which version is working for the python version.

[woodthom2]

Thank you Brandon! Is this something that you could fix by just editing requirements.txt and pyproject.toml? In which case would you please be able to make a pull request with the fix?

[brandonHaipas]

Actually, it has more to do with the typing used in src/harmony/matching/matcher.py, since in all other files the typing is consistent. This file uses typing only available for python version from 3.10 and above as I specified here. I adjusted the requirements.txt, tox.ini, pyproject.toml and src/harmony/matching/matcher.py files so that it passes most tests.

Before changing anything, I pulled the changes from the main branch of this repo and tested the original requirements.txt and pyproject.toml. It wouldn't build on python 3.11. So I set the environments for python versions from 3.8 to 3.11 and tested with tox.

I still have this error for versions 3.9 onwards on a particular test, but the packages itself can be installed now.
Here's the error log now after running tests with tox:

Here's the pull request so you can check the changes.

[brandonHaipas]

Please note that I'm still working on it, I want it to have as little warnings and errors as possible. Feel free to comment in the pull request for any observation.