-
Notifications
You must be signed in to change notification settings - Fork 2k
Add PostgreSQL connection pooling sample. #926
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
93836b1
6fc9cdc
39e788e
d783314
9f191b1
15bca85
b7d2e8a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Format: //devtools/kokoro/config/proto/build.proto | ||
|
|
||
| # Set the folder in which the tests are run | ||
| env_vars: { | ||
| key: "PROJECT" | ||
| value: "cloudsql/postgres/knex" | ||
| } | ||
|
|
||
| # Tell the trampoline which build file to use. | ||
| env_vars: { | ||
| key: "TRAMPOLINE_BUILD_FILE" | ||
| value: "github/nodejs-docs-samples/.kokoro/build.sh" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # Connecting to Cloud SQL - PostgreSQL | ||
|
|
||
| ## Before you begin | ||
|
|
||
| 1. If you haven't already, set up a Node.js Development Environment by following the [Node.js setup guide](https://cloud.google.com/nodejs/docs/setup) and | ||
| [create a project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project). | ||
|
|
||
| 2. Create a Cloud SQL for PostgreSQL instance by following these | ||
| [instructions](https://cloud.google.com/sql/docs/postgres/create-instance). Note the instance name that you create, | ||
| and password that you specify for the default 'postgres' user. | ||
|
|
||
| * If you don't want to use the default user to connect, [create a user](https://cloud.google.com/sql/docs/postgres/create-manage-users#creating). | ||
|
|
||
| 3. Create a database for your application by following these [instructions](https://cloud.google.com/sql/docs/postgres/create-manage-databases). Note the database name. | ||
|
|
||
| 4. Create a service account with the 'Cloud SQL Client' permissions by following these | ||
| [instructions](https://cloud.google.com/sql/docs/mysql/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account). | ||
| Download a JSON key to use to authenticate your connection. | ||
|
|
||
|
|
||
| 5. Use the information noted in the previous steps: | ||
| ```bash | ||
| export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json | ||
| export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>' | ||
| export DB_USER='my-db-user' | ||
| export DB_PASS='my-db-pass' | ||
| export DB_NAME='my_db' | ||
| export DB_PORT='5432' | ||
| ``` | ||
| Note: Saving credentials in environment variables is convenient, but not secure - consider a more | ||
| secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe. | ||
|
|
||
| ## Running locally | ||
|
|
||
| 1. To run this application locally, download and install the `cloud_sql_proxy` by | ||
| following the instructions [here](https://cloud.google.com/sql/docs/postgres/sql-proxy#install). | ||
|
|
||
| Once the proxy is ready, use the following command to start the proxy in the | ||
| background: | ||
| ```bash | ||
| ./cloud_sql_proxy -dir=/cloudsql -instances=$CLOUD_SQL_CONNECTION_NAME=tcp:$DB_PORT -credential_file=$GOOGLE_APPLICATION_CREDENTIALS | ||
| ``` | ||
| Note: Make sure to run the command under a user with write access in the | ||
| `/cloudsql` directory. This proxy will use this folder to create a unix socket | ||
| the application will use to connect to Cloud SQL. | ||
|
|
||
| 2. Next, install the Node.js packages necessary to run the app locally by running the following command: | ||
|
|
||
| ``` | ||
| npm install | ||
| ``` | ||
|
|
||
| 3. Run `createTable.js` to create the database table the app requires and to ensure that the database is properly configured. | ||
| With the Cloud SQL proxy running, run the following command to create the sample app's table in your Cloud SQL PostgreSQL database: | ||
|
|
||
| ``` | ||
| node createTable.js | ||
| ``` | ||
|
|
||
| 4. Run the sample app locally with the following command: | ||
|
|
||
| ``` | ||
| npm start | ||
| ``` | ||
|
|
||
| Navigate towards `http://127.0.0.1:8080` to verify your application is running correctly. | ||
|
|
||
| ## Deploy to Google App Engine Standard | ||
|
|
||
| 1. To allow your app to connect to your Cloud SQL instance when the app is deployed, add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.standard.yaml` file. The deployed application will connect via unix sockets. | ||
|
|
||
| ``` | ||
| env_variables: | ||
| DB_USER: MY_DB_USER | ||
| DB_PASS: MY_DB_PASSWORD | ||
| DB_NAME: MY_DATABASE | ||
| # e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
| CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| ``` | ||
|
|
||
| 2. To deploy to App Engine Standard, run the following command: | ||
|
|
||
| ``` | ||
| gcloud app deploy app.standard.yaml | ||
| ``` | ||
|
|
||
| 3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command: | ||
|
|
||
| ``` | ||
| gcloud app browse | ||
| ``` | ||
|
|
||
| ## Deploy to Google App Engine Flexible | ||
|
|
||
| 1. Add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.flexible.yaml` file. The deployed application will connect via unix sockets. | ||
|
|
||
| ``` | ||
| env_variables: | ||
| DB_USER: MY_DB_USER | ||
| DB_PASS: MY_DB_PASSWORD | ||
| DB_NAME: MY_DATABASE | ||
| # e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
| CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| ``` | ||
|
|
||
| 2. To deploy to App Engine Node.js Flexible Environment, run the following command: | ||
|
|
||
| ``` | ||
| gcloud app deploy app.flexible.yaml | ||
| ``` | ||
|
|
||
| 3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command: | ||
|
|
||
| ``` | ||
| gcloud app browse | ||
| ``` | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # Copyright 2018, Google, Inc. | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| runtime: nodejs | ||
| env: flex | ||
|
|
||
| # [START gae_flex_postgres_env] | ||
|
||
| # The following env variables may contain sensitive information that grants | ||
| # anyone access to your database. Do not add this file to your source control. | ||
| env_variables: | ||
| DB_USER: MY_DB_USER | ||
| DB_PASS: MY_DB_PASSWORD | ||
| DB_NAME: MY_DATABASE | ||
| # e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
| CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| # [END gae_flex_postgres_env] | ||
|
|
||
| beta_settings: | ||
| # The connection name of your instance, available by using | ||
| # 'gcloud beta sql instances describe [INSTANCE_NAME]' or from | ||
| # the Instance details page in the Google Cloud Platform Console. | ||
| cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Copyright 2018, Google, Inc. | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| runtime: nodejs8 | ||
|
|
||
| # [START gae_postgres_env] | ||
|
||
| # The following env variables may contain sensitive information that grants | ||
| # anyone access to your database. Do not add this file to your source control. | ||
| env_variables: | ||
| DB_USER: MY_DB_USER | ||
| DB_PASS: MY_DB_PASSWORD | ||
| DB_NAME: MY_DATABASE | ||
| # e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
| CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| # [END gae_postgres_env] | ||
|
|
||
| beta_settings: | ||
| # The connection name of your instance, available by using | ||
| # 'gcloud beta sql instances describe [INSTANCE_NAME]' or from | ||
| # the Instance details page in the Google Cloud Platform Console. | ||
| cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,52 @@ | ||||||
| /** | ||||||
| * Copyright 2018, Google, Inc. | ||||||
|
||||||
| * Copyright 2018, Google, Inc. | |
| * Copyright 2018 Google LLC. |
(same for the other files)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| { | ||
| "name": "appengine-cloudsql-postgres", | ||
| "description": "Node.js PostgreSQL sample for Cloud SQL on App Engine.", | ||
| "version": "0.0.1", | ||
| "private": true, | ||
| "license": "Apache-2.0", | ||
| "author": "Google Inc.", | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git" | ||
| }, | ||
| "engines": { | ||
| "node": ">=8" | ||
| }, | ||
| "scripts": { | ||
| "unit-test": "ava --verbose test/*.test.js", | ||
| "start-proxy": "! pgrep cloud_sql_proxy > /dev/null && cloud_sql_proxy -instances=$CLOUD_SQL_INSTANCE_NAME=tcp:$DB_PORT &", | ||
| "system-test": "repo-tools test app -- server.js", | ||
| "system-test-proxy": "npm run start-proxy; npm run system-test", | ||
| "all-test": "npm run unit-test && npm run system-test", | ||
| "test": "repo-tools test run --cmd npm -- run all-test" | ||
| }, | ||
| "dependencies": { | ||
| "async": "2.6.0", | ||
| "body-parser": "1.18.2", | ||
| "express": "4.16.2", | ||
| "knex": "0.14.4", | ||
| "pg": "7.4.1", | ||
| "prompt": "1.0.0", | ||
| "pug": "2.0.0-rc.4", | ||
| "@google-cloud/logging-winston": "^0.10.2", | ||
| "winston": "^3.1.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@google-cloud/nodejs-repo-tools": "3.0.0", | ||
| "ava": "0.25.0", | ||
| "proxyquire": "^2.1.0", | ||
| "supertest": "^3.3.0", | ||
| "sinon": "^7.1.1" | ||
| }, | ||
| "cloud-repo-tools": { | ||
| "requiresKeyFile": true, | ||
| "requiresProjectId": true, | ||
| "test": { | ||
| "app": { | ||
| "requiredEnvVars": [ | ||
| "DB_USER", | ||
| "DB_PASS", | ||
| "DB_NAME", | ||
| "DB_PORT", | ||
| "CLOUD_SQL_INSTANCE_NAME" | ||
| ], | ||
| "args": [ | ||
| "server.js" | ||
| ] | ||
| }, | ||
| "build": { | ||
| "requiredEnvVars": [ | ||
| "DB_USER", | ||
| "DB_PASS", | ||
| "DB_NAME", | ||
| "DB_PORT", | ||
| "CLOUD_SQL_INSTANCE_NAME" | ||
| ] | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I didn't catch this before - you don't need both
=tcp:$DB_PORTand-dir=/cloudsql, since the first specifies using TCP and the second is using a UNIX socket. Since AppEngine uses the Unix Socket in/cloudsql/, we should show the user how to use the proxy to create the same environment.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.