Skip to content
This repository was archived by the owner on Nov 16, 2023. It is now read-only.

Commit ddc493e

Browse files
dciborowmicrosoft-github-operations[bot]microsoftopensource
authored
Dciborow/aks (#69)
* Initial commit * Adding First Version of Template (#1) * Initial CODE_OF_CONDUCT.md commit * Initial README.md commit * Initial SECURITY.md commit * Initial LICENSE commit * Adding first template files * Adding first template files * Adding first template files * Update azure-pipelines.yml for Azure Pipelines * Adding first template files * Adding first template files * Adding first template files * Adding first template files * Adding first template files * Adding first template files * adding test notebooks * Adding first template files * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Update README.md * updates Co-authored-by: Microsoft Open Source <microsoftopensource@users.noreply.github.com> * Initial Merge into Master (#2) * Initial CODE_OF_CONDUCT.md commit * Initial README.md commit * Initial SECURITY.md commit * Initial LICENSE commit * Adding first template files * Adding first template files * Adding first template files * Update azure-pipelines.yml for Azure Pipelines * Adding first template files * Adding first template files * Adding first template files * Adding first template files * Adding first template files * Adding first template files * adding test notebooks * Adding first template files * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Update README.md * updates * Update environment.yml * Update environment.yml * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration * Adding Pylint configuration Co-authored-by: Microsoft Open Source <microsoftopensource@users.noreply.github.com> * Update sample_workspace_conf.yml * Adding Test Method * Adding Test Method * Adding Test Method * Adding Test Method * Adding Pylint configuration * Update environment.yml * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update azure-pipelines.yml for Azure Pipelines * Updating to new configuration (#3) * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Adding Realtime Deployment Wrappers * Update 02_TrainOnLocal.ipynb * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Trying to get test 2 to pass * Trying to get test 2 to pass * Working on test 4 * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Create deployment_params.yml * Update azure-pipelines-v2.yml * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py * Merge remote-tracking branch 'template/master' # Conflicts: # .ci/azure-pipelines.yml # .gitignore # LICENSE # README.md # environment.yml # notebooks/00_AMLConfiguration.ipynb # notebooks/__init__.py # sample_workspace_conf.yml # tests/test_notebooks.py Co-authored-by: microsoft-github-operations[bot] <55726097+microsoft-github-operations[bot]@users.noreply.github.com> Co-authored-by: Microsoft Open Source <microsoftopensource@users.noreply.github.com>
1 parent 84d4490 commit ddc493e

16 files changed

Lines changed: 1137 additions & 1673 deletions

.ci/azure-pipelines-v2.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,20 @@ pr:
2828
include:
2929
- master
3030

31+
variables:
32+
- template: ./vars/deployment_params.yml
33+
3134
stages:
32-
- template: .ci/stages/deploy_notebooks_stages_v2.yml@aitemplates
35+
- template: .ci/stages/deploy_notebooks_stages_v5.yml@aitemplates
3336
parameters:
3437
Agent: $(Agent_Name)
3538
jobDisplayName: az-ml-realtime-score
36-
DefaultWorkingDirectory: $(System.DefaultWorkingDirectory)
37-
workload_vars: ../vars/az-ml-realtime-score.yml
39+
TridentWorkloadTypeShort: ${{ variables.TridentWorkloadTypeShort }}
40+
DeployLocation: ${{ variables.DeployLocation }}
41+
ProjectLocation: ${{ variables.ProjectLocation }}
42+
conda: ${{ variables.conda }}
43+
post_cleanup: false
44+
3845
flighting_release: false
3946
flighting_preview: false
4047
flighting_master: false

.ci/vars/deployment_params.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
variables:
2+
TridentWorkloadTypeShort: azmlrts
3+
DeployLocation: westus
4+
ProjectLocation: "."
5+
conda: az-ml-realtime-score

.gitignore

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ workspace_conf.yml
44
*.output_ipynb
55
.azureml
66
pylint-results.xml
7+
project.yml
78
.idea
89
score.py
910

@@ -14,16 +15,13 @@ assets/
1415
scripts/assets/
1516
.amlignore
1617
scripts/.amlignore
17-
__pycache__/
1818
scripts/__pycache__/
1919

2020
# Environments
2121
.env
2222

23-
# Jupyter Notebook
24-
.ipynb_checkpoints
25-
2623
# Byte-compiled / optimized / DLL files
24+
__pycache__/
2725
*.py[cod]
2826
*$py.class
2927

@@ -151,3 +149,4 @@ dmypy.json
151149

152150
# Pyre type checker
153151
.pyre/
152+
/project.yml

.pylintrc

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,12 @@ confidence=
6060
# --enable=similarities". If you want to run only the classes checker, but have
6161
# no Warning level messages displayed, use "--disable=all --enable=classes
6262
# --disable=W".
63-
disable=missing-module-docstring,
63+
disable=global-variable-undefined,
64+
global-statement,
65+
too-many-arguments,
66+
too-many-function-args,
67+
pointless-statement,
68+
missing-module-docstring,
6469
trailing-whitespace,
6570
fixme,
6671
print-statement,
@@ -302,7 +307,7 @@ variable-naming-style=snake_case
302307
expected-line-ending-format=
303308

304309
# Regexp for a line that is allowed to be longer than the limit.
305-
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
310+
ignore-long-lines=^\s*(# )?<?https?:\S+>?$|^\s*get_ipython\S+
306311

307312
# Number of spaces of indent required inside a hanging or continued line.
308313
indent-after-paren=4
@@ -454,7 +459,7 @@ signature-mutators=
454459

455460
# List of additional names supposed to be defined in builtins. Remember that
456461
# you should avoid defining new builtins when possible.
457-
additional-builtins=
462+
additional-builtins=get_ipython
458463

459464
# Tells whether unused global variables should be treated as a violation.
460465
allow-global-unused-variables=yes

notebooks/00_AMLConfiguration.ipynb

Lines changed: 18 additions & 127 deletions
Original file line numberDiff line numberDiff line change
@@ -18,92 +18,19 @@
1818
"## Prerequisites\n",
1919
"\n",
2020
"If you have already completed the prerequisites and selected the correct Kernel for this notebook, the AML Python SDK \n",
21-
"is already installed. Let's load the imports and check the AML SDK version."
22-
]
23-
},
24-
{
25-
"cell_type": "code",
26-
"execution_count": null,
27-
"metadata": {
28-
"pycharm": {
29-
"name": "#%%\n"
30-
}
31-
},
32-
"outputs": [],
33-
"source": [
34-
"import json\n",
21+
"is already installed. Let's load the imports and check the AML SDK version.\n",
3522
"\n",
36-
"import azureml.core\n",
37-
"from azure_utils.machine_learning.utils import load_configuration, get_or_create_workspace\n",
38-
"\n",
39-
"print(\"AML SDK Version:\", azureml.core.VERSION)"
40-
]
41-
},
42-
{
43-
"cell_type": "markdown",
44-
"metadata": {},
45-
"source": [
4623
"## Set up your Azure Machine Learning workspace\n",
4724
"## Load Configurations from file\n",
4825
"\n",
49-
"Configurations are loaded from a file, to prevent accident commits of Azure secerts into source control.\n",
50-
"This file name is included in the .gitignore to also prevent accident commits. A template file is included that should\n",
51-
"be copied, and each parameter filled in."
52-
]
53-
},
54-
{
55-
"cell_type": "code",
56-
"execution_count": null,
57-
"metadata": {
58-
"pycharm": {
59-
"name": "#%%\n"
60-
}
61-
},
62-
"outputs": [],
63-
"source": [
64-
"cfg = load_configuration(\"../workspace_conf.yml\")"
65-
]
66-
},
67-
{
68-
"cell_type": "markdown",
69-
"metadata": {
70-
"pycharm": {
71-
"name": "#%% md\n"
72-
}
73-
},
74-
"source": [
75-
"## Load Configurations into Notebook.\n",
26+
"Configurations are loaded by default from a file `project.yml`, to prevent accident commits of Azure secrets into \n",
27+
"source control. This file name is included in the `.gitignore` to also prevent accident commits. A template file \n",
28+
"is included that should be copied, and each parameter filled in.\n",
29+
"\n",
30+
"If the file is not present, and UI Prompt will pop up to insert configurations, and save to the file.\n",
7631
"\n",
77-
"The following cell loads the configurations from the local file, into the notebook memory. The following cell is also\n",
78-
"marked as a parameter cell. When using this notebook with [papermill](https://github.com/nteract/papermill), these\n",
79-
"parameters can be override. See the tests for examples."
80-
]
81-
},
82-
{
83-
"cell_type": "code",
84-
"execution_count": null,
85-
"metadata": {
86-
"pycharm": {
87-
"name": "#%%\n"
88-
},
89-
"tags": [
90-
"parameters"
91-
]
92-
},
93-
"outputs": [],
94-
"source": [
95-
"subscription_id = cfg['subscription_id']\n",
96-
"resource_group = cfg['resource_group']\n",
97-
"workspace_name = cfg['workspace_name']\n",
98-
"workspace_region = cfg['workspace_region']"
99-
]
100-
},
101-
{
102-
"cell_type": "markdown",
103-
"metadata": {},
104-
"source": [
10532
"## Create the workspace\n",
106-
"This cell will create an AML workspace for you in a subscription, provided you have the correct permissions.\n",
33+
"This cell will also create an AML workspace for you in a subscription, provided you have the correct permissions.\n",
10734
"\n",
10835
"This will fail when:\n",
10936
"1. You do not have permission to create a workspace in the resource group\n",
@@ -112,32 +39,9 @@
11239
"subscription\n",
11340
"\n",
11441
"If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to \n",
115-
"provision the required resources. If this cell succeeds, you're done configuring AML!\n"
116-
]
117-
},
118-
{
119-
"cell_type": "code",
120-
"execution_count": null,
121-
"metadata": {
122-
"pycharm": {
123-
"name": "#%%\n"
124-
}
125-
},
126-
"outputs": [],
127-
"source": [
128-
"ws = get_or_create_workspace(workspace_name, subscription_id, resource_group, workspace_region)\n",
129-
"ws_json = ws.get_details()"
130-
]
131-
},
132-
{
133-
"cell_type": "markdown",
134-
"metadata": {
135-
"pycharm": {
136-
"name": "#%% md\n"
137-
}
138-
},
139-
"source": [
140-
"Let's check the details of the workspace."
42+
"provision the required resources. If this cell succeeds, you're done configuring AML!\n",
43+
"\n",
44+
"After creation we will check the details of the workspace."
14145
]
14246
},
14347
{
@@ -150,18 +54,17 @@
15054
},
15155
"outputs": [],
15256
"source": [
153-
"print(json.dumps(ws_json, indent=2))"
57+
"from azure_utils.machine_learning.utils import get_or_create_workspace_from_file\n",
58+
"\n",
59+
"ws = get_or_create_workspace_from_file()"
15460
]
15561
},
15662
{
15763
"cell_type": "markdown",
158-
"metadata": {
159-
"pycharm": {
160-
"name": "#%% md\n"
161-
}
162-
},
64+
"metadata": {},
16365
"source": [
164-
"You are now ready to move on to the [AutoML Local](01_DataPrep.ipynb) notebook."
66+
"We can now move on to [Data Preperation](01_DataPrep.ipynb) notebook to train our model using Azure Machine \n",
67+
"Learning."
16568
]
16669
}
16770
],
@@ -172,25 +75,13 @@
17275
"language": "python",
17376
"name": "az-ml-realtime-score"
17477
},
175-
"language_info": {
176-
"codemirror_mode": {
177-
"name": "ipython",
178-
"version": 3
179-
},
180-
"file_extension": ".py",
181-
"mimetype": "text/x-python",
182-
"name": "python",
183-
"nbconvert_exporter": "python",
184-
"pygments_lexer": "ipython3",
185-
"version": "3.6.2"
186-
},
18778
"pycharm": {
18879
"stem_cell": {
18980
"cell_type": "raw",
190-
"source": [],
19181
"metadata": {
19282
"collapsed": false
193-
}
83+
},
84+
"source": []
19485
}
19586
}
19687
},

0 commit comments

Comments
 (0)