[SPARKNLP-1215] Updating support for Microsoft Fabric

[danilojsl]

Description

Enhance Microsoft Fabric Support by adding the capability to download models from lakehouse containers in Microsoft Fabric.

Motivation and Context

Currently, Microsoft Fabric raises an error when attempting to download a model that is defined in a lakehouse container via the cache path. This limitation hinders users from managing and retrieving their models efficiently in lakehouse environments.

How Has This Been Tested?

Screenshots (if appropriate):

Microsoft Fabric notebooks with cache_folder i.e.

%%configure -f
{
  "conf": {
    "spark.jsl.settings.pretrained.cache_folder": "abfss://[email protected]/lakehouse_folder.Lakehouse/Files/my_models"
  }
}

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Code improvements with no or little impact
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING page.
• I have added tests to cover my changes.
• All new and existing tests passed.