|
| 1 | +.. _child_services: |
| 2 | + |
| 3 | +============== |
| 4 | +Child services |
| 5 | +============== |
| 6 | + |
| 7 | +When a Twine file is written, there is the option to include children (i.e. child services or child digital twins) so |
| 8 | +the main or parent service can communicate with them to ask them "questions". A question is a set of input |
| 9 | +values and/or an input manifest in the form the child's Twine specifies. When a question is asked, the parent can expect |
| 10 | +an answer, which is a set of output values and/or an output manifest, again in the form specified by the child's Twine. |
| 11 | + |
| 12 | +There can be: |
| 13 | + |
| 14 | +- Any number of children |
| 15 | +- Any number of questions asked to each child |
| 16 | + |
| 17 | +Further, a child can have its own children that it asks questions to. There is no limit to this as long as the tree of |
| 18 | +services forms a directed acyclical graph (DAG) - i.e. there are no loops and no children ask their parents any |
| 19 | +questions. |
| 20 | + |
| 21 | + |
| 22 | +------------------------- |
| 23 | +Example usage in your app |
| 24 | +------------------------- |
| 25 | + |
| 26 | +Assuming you have specified which children you would like to use in your ``twine.json`` file (see below for an example), |
| 27 | +you can ask children questions in your ``app.py`` file as follows: |
| 28 | + |
| 29 | +.. code-block:: python |
| 30 | +
|
| 31 | + answer_1 = analysis.children["child_1"].ask(input_values=analysis.input_values, timeout=None) |
| 32 | + answer_2 = analysis.children["child_2"].ask(input_values=analysis.input_values, timeout=None) |
| 33 | +
|
| 34 | + >>> answer_1 |
| 35 | + { |
| 36 | + 'output_values': <output values in form specified by child twine.json>, |
| 37 | + 'output_manifest': <output manifest in form specified by child twine.json> |
| 38 | + } |
| 39 | +
|
| 40 | +
|
| 41 | +A timeout (measured in seconds) can be set for how long you are willing to wait for an answer, but bear in mind that the |
| 42 | +question has to reach the child, the child has to run its own analysis on the inputs sent to it (this most likely |
| 43 | +corresponds to the dominant part of the wait time), and the answer has to be send back to the parent. If you are not |
| 44 | +sure how long a particular analysis might take, it's best to set the timeout to ``None`` or ask the owner/maintainer of |
| 45 | +the child for an estimate. |
| 46 | + |
| 47 | + |
| 48 | +-------- |
| 49 | +Backends |
| 50 | +-------- |
| 51 | + |
| 52 | +The backend specifies which method of communication the child uses (e.g. Google Cloud Pub/Sub), as well as providing |
| 53 | +pointers to the credentials and any other parameters necessary to access it. Each child must have its backend |
| 54 | +specified explicitly, even if all children use the same one. This is to support the use case where each child uses a |
| 55 | +different backend. |
| 56 | + |
| 57 | +To make use of a certain child in ``app.py``, its backend configuration must be specified in ``children.json``. The only |
| 58 | +backend currently supported is ``GCPPubSubBackend``, which uses Google Cloud Platform's publisher/subscriber service. |
| 59 | + |
| 60 | + |
| 61 | +-------------------------- |
| 62 | +Example children.json file |
| 63 | +-------------------------- |
| 64 | + |
| 65 | +To access children in ``app.py``, they must be specified in ``children.json``, which is by default looked for in |
| 66 | +``<data_dir>/configuration/children.json``: |
| 67 | + |
| 68 | +.. code-block:: javascript |
| 69 | +
|
| 70 | + [ |
| 71 | + { |
| 72 | + "key": "wind_speed", |
| 73 | + "id": "7b9d07fa-6bcd-4ec3-a331-69f737a15751", |
| 74 | + "backend": { |
| 75 | + "name": "GCPPubSubBackend", |
| 76 | + "project_name": "<google_cloud_project_name>", |
| 77 | + "credentials_filename": "<absolute/path/to/credentials_file.json>" |
| 78 | + } |
| 79 | + }, |
| 80 | + { |
| 81 | + "key": "elevation", |
| 82 | + "id": "8dgd07fa-6bcd-4ec3-a331-69f737a15332", |
| 83 | + "backend": { |
| 84 | + "name": "GCPPubSubBackend", |
| 85 | + "project_name": "<google_cloud_project_name>", |
| 86 | + "credentials_filename": "<absolute/path/to/credentials_file.json>" |
| 87 | + } |
| 88 | + } |
| 89 | + ] |
| 90 | +
|
| 91 | +
|
| 92 | +------------------------------------------- |
| 93 | +Example children field in a twine.json file |
| 94 | +------------------------------------------- |
| 95 | + |
| 96 | +The children field must also be present in the ``twine.json`` file: |
| 97 | + |
| 98 | +.. code-block:: javascript |
| 99 | +
|
| 100 | + { |
| 101 | + ... |
| 102 | + "children": [ |
| 103 | + { |
| 104 | + "key": "wind_speed", |
| 105 | + "purpose": "A service that returns the average wind speed for a given latitude and longitude.", |
| 106 | + "notes": "Some notes.", |
| 107 | + "filters": "tags:wind_speed" |
| 108 | + }, |
| 109 | + { |
| 110 | + "key": "elevation", |
| 111 | + "purpose": "A service that returns the elevation for a given latitude and longitude.", |
| 112 | + "notes": "Some notes.", |
| 113 | + "filters": "tags:elevation" |
| 114 | + } |
| 115 | + ], |
| 116 | + ... |
| 117 | + } |
| 118 | +
|
| 119 | +
|
| 120 | +------------------------------------ |
| 121 | +Starting a child/service as a server |
| 122 | +------------------------------------ |
| 123 | + |
| 124 | +For a parent to ask a child questions, the child must already be running as a server. The person/organisation |
| 125 | +responsible for the child must start it as a server if it is to be able to answer questions. |
| 126 | + |
| 127 | +To start a service as a server, the command line interface (CLI) can be used: |
| 128 | + |
| 129 | +.. code-block:: bash |
| 130 | +
|
| 131 | + octue-app start \ |
| 132 | + --app-dir=<path/to/app_directory> \ |
| 133 | + --twine=<path/to/twine.json> \ |
| 134 | + --config-dir=<path/to/configuration> \ |
| 135 | + --service-id=<UUID of service> |
| 136 | +
|
| 137 | +You can choose a random UUID for the service ID, but it must be unique across all services. It must also stay the same |
| 138 | +once it has been created so that Scientists and other services can know which service is which and communicate with the |
| 139 | +correct ones. We recommend registering your service with Octue if you want others to be able to use it easily (and, if |
| 140 | +allowed, look it up), and also so that its ID is reserved permanently. |
| 141 | + |
| 142 | +**Note:** We will be automating this process soon. In the meantime, please contact us to register service IDs. |
| 143 | + |
| 144 | + |
| 145 | +-------------------------------------------------------------------------- |
| 146 | +See services communicate in real time: running the child services template |
| 147 | +-------------------------------------------------------------------------- |
| 148 | + |
| 149 | +1. Contact Octue to request a Google Cloud Platform service account credentials file. |
| 150 | + |
| 151 | +2. Save this file locally and create a ``GCP_SERVICE_ACCOUNT`` environment variable whose value is the file's absolute path. This variable must be available to all three terminal windows used to run the template - see below for one method of doing this. **IMPORTANT**: Do not commit this or any other credentials or credentials file to git, GitHub, or any other version control software or website - doing so opens you, your systems and equipment, and our systems and equipment up to hackers and cyber attack. |
| 152 | + |
| 153 | +3. From the repository root, start the elevation service as a server in a terminal window: |
| 154 | + |
| 155 | +.. code-block:: bash |
| 156 | +
|
| 157 | + GCP_SERVICE_ACCOUNT=</absolute/path/to/gcp_credentials.json> octue-app --log-level=debug --show-twined-logs |
| 158 | + start \ |
| 159 | + --app-dir=octue/templates/template-child-services/elevation_service \ |
| 160 | + --twine=octue/templates/template-child-services/elevation_service/twine.json \ |
| 161 | + --config-dir=octue/templates/template-child-services/elevation_service/data/configuration \ |
| 162 | + --service-id=8dgd07fa-6bcd-4ec3-a331-69f737a15332 |
| 163 | +
|
| 164 | +4. In another terminal window, start the wind speeds service as a server: |
| 165 | + |
| 166 | +.. code-block:: bash |
| 167 | +
|
| 168 | + GCP_SERVICE_ACCOUNT=</absolute/path/to/gcp_credentials.json> octue-app --log-level=debug --show-twined-logs \ |
| 169 | + start \ |
| 170 | + --app-dir=octue/templates/template-child-services/wind_speed_service \ |
| 171 | + --twine=octue/templates/template-child-services/wind_speed_service/twine.json \ |
| 172 | + --config-dir=octue/templates/template-child-services/wind_speed_service/data/configuration \ |
| 173 | + --service-id=7b9d07fa-6bcd-4ec3-a331-69f737a15751 |
| 174 | +
|
| 175 | +5. In a third terminal window, run the parent app (don't start it as a server): |
| 176 | + |
| 177 | +.. code-block:: bash |
| 178 | +
|
| 179 | + GCP_SERVICE_ACCOUNT=</absolute/path/to/gcp_credentials.json> octue-app --log-level=debug --show-twined-logs \ |
| 180 | + run \ |
| 181 | + --app-dir=octue/templates/template-child-services/parent_service \ |
| 182 | + --twine=octue/templates/template-child-services/parent_service/twine.json \ |
| 183 | + --data-dir=octue/templates/template-child-services/parent_service/data |
| 184 | +
|
| 185 | +6. Watch the logs to observe the three services communicate with each other via the cloud in real time. When finished, you will find the output values of the parent in ``octue/templates/template-child-services/parent_service/data/output/values.json`` |
0 commit comments