-
Notifications
You must be signed in to change notification settings - Fork 6.7k
Add samples for Cloud Tasks #1068
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
87c721f
507a54b
1a43939
ee920fb
fa4270c
4d1f228
5e7a379
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,7 +33,7 @@ On Compute Engine and Container Engine, authentication credentials will be | |
| automatically detected, but the instances must have been created with the | ||
| necessary scopes. | ||
|
|
||
| In any other environment, for example Compute Engine instance without the | ||
| In any other environment, for example a Compute Engine instance without the | ||
| necessary scopes, you should set `GOOGLE_APPLICATION_CREDENTIALS` environment | ||
| variable to a JSON key file for a service account. | ||
|
|
||
|
|
@@ -42,17 +42,18 @@ for more information. | |
|
|
||
| ## Creating a queue | ||
|
|
||
| To create a queue using the Cloud SDK, use the provided queue.yaml: | ||
| To create a queue using the Cloud SDK, use the following gcloud command: | ||
|
|
||
| gcloud app deploy queue.yaml | ||
| gcloud alpha tasks queues create-app-engine-queue "my-appengine-queue" | ||
|
||
|
|
||
| ## Deploying the App Engine app | ||
|
|
||
| First, vendor the dependencies into the project: | ||
| Note: A newly created queue will route to the default App Engine service and | ||
| version unless configured to do otherwise. Read the online help for the | ||
| `create-app-engine-queue` or the `update-app-engine-queue` commands to learn | ||
| about routing overrides for App Engine queues. | ||
|
|
||
| pip install -r requirements.txt | ||
| ## Deploying the App Engine app | ||
|
|
||
| Next, deploy the App Engine app | ||
| Deploy the App Engine app with gcloud: | ||
|
|
||
| gcloud app deploy | ||
|
|
||
|
|
@@ -61,47 +62,44 @@ Verify the index page is serving: | |
| gcloud app browse | ||
|
|
||
| The App Engine app serves as a target for the push requests. It has an | ||
| endpoint `/set_payload` that that stores the payload from the HTTP POST data in | ||
| Cloud Datastore. The payload can be accessed in your browser at the | ||
| `/get_payload` endpoint with a GET request. | ||
| endpoint `/log_payload` that reads the payload (i.e., the request body) of the | ||
| HTTP POST request and logs it. The log output can be viewed with: | ||
|
|
||
| gcloud app logs read | ||
|
|
||
| ## Running the Samples | ||
|
|
||
| The project ID must be specified either as a command line argument using | ||
| `--project-id`, or by editing `DEFAULT_PROJECT_ID` within `task_snippets.py`. | ||
|
|
||
| Set the environment variables: | ||
| Set environment variables: | ||
|
|
||
| First, your project ID: | ||
|
|
||
| export PROJECT_ID=my-project-id | ||
| export LOCATION_ID=us-central1 | ||
| export QUEUE_ID=my-appengine-queue # From queue.yaml | ||
|
|
||
| View all queues: | ||
| Then the queue ID, as specified at queue creation time. Queue IDs already | ||
| created can be listed with `gcloud alpha tasks queue list`. | ||
|
||
|
|
||
| python app_engine_queue_snippets.py --api_key=$API_KEY list-queues --project_id=$PROJECT_ID --location_id=$LOCATION_ID | ||
| export QUEUE_ID=my-appengine-queue | ||
|
|
||
| Set the queue name as an environment variable: | ||
| And finally the location ID, which can be discovered with | ||
| `gcloud alpha tasks queue describe $QUEUE_ID`, with the location embedded in the | ||
|
||
| "name" value (for instance, if the name is | ||
| "projects/my-project/locations/us-central1/queues/my-appengine-queue", then the | ||
| location is "us-central1"). | ||
|
|
||
| export QUEUE_NAME=projects/$PROJECT_ID/locations/$LOCATION_ID/queues/$QUEUE_ID | ||
| export LOCATION_ID=us-central1 | ||
|
|
||
| Create a task, targeted at the `set_payload` endpoint with a payload specified: | ||
|
||
|
|
||
| python app_engine_queue_snippets.py --api_key=$API_KEY create-task --queue_name=$QUEUE_NAME --payload=hello | ||
|
|
||
| Now view that the payload was received and verify the count and payload: | ||
|
|
||
| http://your-app-id.appspot.com/get_payload | ||
|
|
||
| Create a task that will be scheduled for a few seconds in the future using | ||
| the `--in_seconds` flag: | ||
|
|
||
| python app_engine_queue_snippets.py --api_key=$API_KEY create-task --queue_name=$QUEUE_NAME --payload=hello --in_seconds=30 | ||
| python create_app_engine_queue_task.py --project=$PROJECT_ID --queue=QUEUE_ID --location=LOCATION_ID --payload=hello | ||
|
||
|
|
||
| Since `--in_seconds` was set to 30, it will take 30 seconds for the new | ||
| payload to be pushed to the `/get_payload` endpoint, which can then be viewed at: | ||
| Now view that the payload was received and verify the payload: | ||
|
|
||
| http://your-app-id.appspot.com/get_payload | ||
| gcloud app logs read | ||
|
|
||
| It might also be helpful to view the request logs of your App Engine app: | ||
| Create a task that will be scheduled for a time in the future using the | ||
| `--in_seconds` flag: | ||
|
|
||
| https://console.cloud.google.com/logs | ||
| python create_app_engine_queue_task.py --project=$PROJECT_ID --queue=QUEUE_ID --location=LOCATION_ID --payload=hello --in_seconds=30 | ||
|
||
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # Copyright 2017 Google Inc. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from __future__ import print_function | ||
|
|
||
| import argparse | ||
| import base64 | ||
| import datetime | ||
| import pprint | ||
|
|
||
| from googleapiclient import discovery | ||
|
|
||
|
|
||
| def seconds_from_now_to_rfc3339_datetime(seconds): | ||
| """Return an RFC 3339 datetime string for a number of seconds from now.""" | ||
| d = datetime.datetime.utcnow() + datetime.timedelta(seconds=seconds) | ||
| return d.isoformat('T') + 'Z' | ||
|
|
||
|
|
||
| def create_task(project, queue, location, payload=None, in_seconds=None): | ||
| """Create a task for a given queue with an arbitrary payload.""" | ||
| client = get_client() | ||
|
|
||
| url = '/log_payload' | ||
| body = { | ||
| 'task': { | ||
| 'app_engine_task_target': { | ||
| 'http_method': 'POST', | ||
| 'relative_url': url | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if payload is not None: | ||
| # Payload is a string (unicode), so | ||
| body['task']['app_engine_task_target']['payload'] = base64.b64encode( | ||
| payload.encode()).decode() | ||
|
|
||
| if in_seconds is not None: | ||
| scheduled_time = seconds_from_now_to_rfc3339_datetime(in_seconds) | ||
| body['task']['schedule_time'] = scheduled_time | ||
|
|
||
| queue_name = 'projects/{}/locations/{}/queues/{}'.format( | ||
| project, location, queue) | ||
|
|
||
| print('Sending task {}'.format(pprint.pformat(body))) | ||
|
||
|
|
||
| response = client.projects().locations().queues().tasks().create( | ||
| parent=queue_name, body=body).execute() | ||
|
|
||
| # By default CreateTaskRequest.responseView is BASIC, so not all | ||
| # information is retrieved by default because some data, such as payloads, | ||
| # might be desirable to return only when needed because of its large size | ||
| # or because of the sensitivity of data that it contains. | ||
| print('Created task {}'.format(response['name'])) | ||
| return response | ||
|
|
||
|
|
||
| def get_client(): | ||
|
||
| """Build an http client.""" | ||
| DISCOVERY_URL = 'https://cloudtasks.googleapis.com/$discovery/rest?version=v2beta2' | ||
|
||
| client = discovery.build('cloudtasks', 'v2beta2', | ||
| discoveryServiceUrl=DISCOVERY_URL) | ||
| return client | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| parser = argparse.ArgumentParser( | ||
| description=create_task.__doc__, | ||
| formatter_class=argparse.RawDescriptionHelpFormatter) | ||
|
|
||
| parser.add_argument( | ||
| '--project', | ||
| help='Project of the queue to add the task to.' | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--queue', | ||
| help='ID (short name) of the queue to add the task to.' | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--location', | ||
| help='Location of the queue to add the task to.' | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--payload', | ||
| help='Optional payload to attach to the push queue.' | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--in_seconds', | ||
| help='The number of seconds from now to schedule task attempt.' | ||
| ) | ||
|
|
||
| args = parser.parse_args() | ||
|
|
||
| create_task( | ||
| args.project, args.queue, args.location, | ||
| args.payload, args.in_seconds) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above