From 6e0ab945e9807683f757910a0868d63832f59374 Mon Sep 17 00:00:00 2001 From: Andre Date: Mon, 13 Jun 2022 12:15:30 -0300 Subject: [PATCH] fixing typos in docs --- docs/docs/concepts/processor.md | 2 +- docs/docs/concepts/recipe.md | 16 ++++++++-------- docs/docs/concepts/sink.md | 6 +++--- docs/docs/concepts/source.md | 2 +- docs/docs/guides/1_list_Plugins.md | 2 +- docs/docs/guides/2_manage_recipes.md | 6 +++--- docs/docs/guides/4_deployment.md | 2 +- docs/docs/guides/introduction.md | 4 ++-- docs/docs/reference/commands.md | 12 ++++++------ docs/docs/reference/configuration.md | 2 +- docs/docs/reference/extractors.md | 2 +- docs/docs/reference/metadata_models.md | 14 +++++++------- docs/docs/reference/sinks.md | 2 +- 13 files changed, 36 insertions(+), 36 deletions(-) diff --git a/docs/docs/concepts/processor.md b/docs/docs/concepts/processor.md index c562c2ac1..4de20904c 100644 --- a/docs/docs/concepts/processor.md +++ b/docs/docs/concepts/processor.md @@ -1,6 +1,6 @@ # Processor -A recipe can have none or many processors registered, depending upon the way user want metadata to processed. A processor is basically a function that: +A recipe can have none or many processors registered, depending upon the way the user wants metadata to be processed. A processor is basically a function that: * expects a list of data * processes the list diff --git a/docs/docs/concepts/recipe.md b/docs/docs/concepts/recipe.md index e2356b94e..edb3a2634 100644 --- a/docs/docs/concepts/recipe.md +++ b/docs/docs/concepts/recipe.md @@ -1,10 +1,10 @@ # Recipe -A recipe is a set of instructions and configurations defined by user, and in Meteor they are used to define how a particular job will be performed. It should contain instruction about the `source` from which the metadata will be fetched, information about metadata `processors` and the destination is to be defined as `sinks` of metadata. +A recipe is a set of instructions and configurations defined by the user, and in Meteor they are used to define how a particular job will be performed. It should contain instructions about the `source` from which the metadata will be fetched, information about metadata `processors` and the destination is to be defined as `sinks` of metadata. -The recipe should contain about **only one** `source` since we wish to have a seperate job for different extractors, hence keeping them isolated. Should have **atleast one** destination of metadata mentioned in `sinks`, and the `processors` field is optional but can have multiple processors. +The recipe should contain about **only one** `source` since we wish to have a seperate job for different extractors, hence keeping them isolated. Should have **at least one** destination of metadata mentioned in `sinks`, and the `processors` field is optional but can have multiple processors. -Recipe is a yaml file, follows a structure as shown below and needs to passed as a individual file or as a bunch of recipes contained in a directory as shown in [sample usage](recipe.md#sample-usage). +Recipe is a yaml file, follows a structure as shown below and needs to be passed as an individual file or as a bunch of recipes contained in a directory as shown in [sample usage](recipe.md#sample-usage). ## Writing a Recipe for Meteor @@ -32,19 +32,19 @@ processors: # optional - metadata processors ### Glossary Table -Contains details about the ingridients of our recipe. The `config` of each source, sinks and processors differs as different data source required different kinds of credentials, please refer more about them in further reference section. +Contains details about the ingredients of our recipe. The `config` of each source, sinks and processors differs as different data sources require different kinds of credentials, please refer more about them in further reference section. | Key | Description | Requirement | further reference | | :--- | :--- | :--- | :--- | | `name` | **unique** recipe name, will be used as ID for job | required | N/A | | `version` | Specify the version of recipe being used | required | N/A | | `source` | contains details about the source of metadata extraction | required | [source](source.md) | -| `sinks` | defines the final destination's of extracted and processed metadata | required | [sink](sink.md) | +| `sinks` | defines the final destination of extracted and processed metadata | required | [sink](sink.md) | | `processors` | used process the metadata before sinking | optional | [processor](processor.md) | ## Dynamic recipe value -Meteor reads recipe using [go template](https://golang.org/pkg/text/template/), which means you can put a variable instead of static value in a recipe. +Meteor reads recipe using [go template](https://golang.org/pkg/text/template/), which means you can put a variable instead of a static value in a recipe. Environment variables with prefix `METEOR_`, such as `METEOR_MONGODB_PASS`, will be used as the template data for the recipe. This is to allow you to skip creating recipes containing the credentials of datasource. @@ -78,9 +78,9 @@ sinks: ## Support to pass env variables with --var flag -Meteor allows you to maintain a `.yaml` file as well which can be used as a template data for recipe. +Meteor allows you to maintain a `.yaml` file as well which can be used as template data for recipes. The variables here should not contain a `METEOR_` prefix and should be as normal as any other `config` file. -Meteor reads both local environment variables as well as the ones from `yaml` file and in case of conflict prefers the one mentioned in `yaml` file. +Meteor reads both local environment variables as well as the ones from the `yaml` file and in case of conflict prefers the one mentioned in the `yaml` file. * _sample-config.yaml_ diff --git a/docs/docs/concepts/sink.md b/docs/docs/concepts/sink.md index 3595982cc..2191b7cf5 100644 --- a/docs/docs/concepts/sink.md +++ b/docs/docs/concepts/sink.md @@ -1,6 +1,6 @@ # Sink -`sinks` are used to define the medium of consuming the metadata being extracted. You need to specify **atleast one** sink or can specify multiple sinks in a recipe, this will prevent you from having to create duplicate recipes for the same job. The given examples shows you its correct usage if your sink is `http` and `kafka`. +`sinks` are used to define the medium of consuming the metadata being extracted. You need to specify **at least one** sink or can specify multiple sinks in a recipe, this will prevent you from having to create duplicate recipes for the same job. The given examples show you its correct usage if your sink is `http` and `kafka`. ## Writing `sinks` part of your recipe @@ -60,9 +60,9 @@ By default, metadata would be serialized into JSON format before sinking. To sen ## Custom Sink -Meteor has built-in sinks like Kafka and HTTP which users can just utilise directly. We will also allow creating custom sinks for DRY purposes. +Meteor has built-in sinks like Kafka and HTTP which users can just use directly. We will also allow creating custom sinks for DRY purposes. -It will be useful if you can find yourself sinking multiple metadata source to one place. +It will be useful if you can find yourself sinking multiple metadata sources to one place. ### Sample Custom Sink diff --git a/docs/docs/concepts/source.md b/docs/docs/concepts/source.md index 9983f2f74..64e5fba04 100644 --- a/docs/docs/concepts/source.md +++ b/docs/docs/concepts/source.md @@ -1,6 +1,6 @@ # Source -When the source field is defined, Meteor will extract data from a metadata source using the details defined in the field. `type` field should define the name of Extractor you want, you can use one from this list [here](../reference/extractors.md). `config` of a extractor can be different for different Extractor and needs you to provide details to setup a connection between meteor and your source. To determine the required configurations you can visit README of each Extractor [here](https://github.com/odpf/meteor/tree/cb12c3ecf8904cf3f4ce365ca8981ccd132f35d0/plugins/extractors/README.md). +When the source field is defined, Meteor will extract data from a metadata source using the details defined in the field. `type` field should define the name of Extractor you want, you can use one from this list [here](../reference/extractors.md). `config` of an extractor can be different for different Extractor and needs you to provide details to set up a connection between meteor and your source. To determine the required configurations you can visit README of each Extractor [here](https://github.com/odpf/meteor/tree/cb12c3ecf8904cf3f4ce365ca8981ccd132f35d0/plugins/extractors/README.md). ## Writing source part of your recipe diff --git a/docs/docs/guides/1_list_Plugins.md b/docs/docs/guides/1_list_Plugins.md index 4dd45cb79..c6209103b 100644 --- a/docs/docs/guides/1_list_Plugins.md +++ b/docs/docs/guides/1_list_Plugins.md @@ -5,7 +5,7 @@ Before getting started we expect you went through the [prerequisites](./introduc Meteor follows a plugin driven approach and hence includes basically three kinds of plugins for the metadata orchestration: extractors (source), processors, and sinks (destination). Some details on these 3 are: -- **Extractors** are the set of plugins that are source of our metadata and include databases, dashboards, users, etc. +- **Extractors** are the set of plugins that are the source of our metadata and include databases, dashboards, users, etc. - **Processors** are the set of plugins that perform the enrichment or data processing for the metadata after extraction. diff --git a/docs/docs/guides/2_manage_recipes.md b/docs/docs/guides/2_manage_recipes.md index 3ba580fc4..8f61ae6d5 100644 --- a/docs/docs/guides/2_manage_recipes.md +++ b/docs/docs/guides/2_manage_recipes.md @@ -1,6 +1,6 @@ # Recipes - Creation and linting -A recipe is a set of instructions and configurations defined by user, and in Meteor they are used to define how a particular job will be performed. +A recipe is a set of instructions and configurations defined by the user, and in Meteor they are used to define how a particular job will be performed. Thus, for the entire set of orchestration all you will need to provide will be recipe\(s\) for all the jobs you want meteor to do. Read more about the concepts of Recipe [here](../concepts/recipe.md). @@ -9,7 +9,7 @@ A sample recipe can be generated using the `meteor new` command mentioned [below One can also generate multiple recipes with similar configurations using the `meteor gen` command mentioned [below](#generating-multiple-recipes-from-a-template). -After making the necessary changes to the source, and sinks as per your local setup, you can validate the sample-recipe using steps mentioed [here](#linting-recipes). +After making the necessary changes to the source, and sinks as per your local setup, you can validate the sample-recipe using steps mentioned [here](#linting-recipes). ## Generating new Sample recipe\(s\) @@ -22,7 +22,7 @@ $ meteor new recipe sample -e -s $ meteor new recipe sample -e bigquery -s compass,kafka # for the tour you can use a single console sink -# extracor(-e) as postgres, sink(-s) and enrich processor(-p) +# extractor(-e) as postgres, sink(-s) and enrich processor(-p) # save the generated recipe to a recipe.yaml $ meteor new recipe sample -e postgres -s console -p enrich > recipe.yaml diff --git a/docs/docs/guides/4_deployment.md b/docs/docs/guides/4_deployment.md index 154227e34..95173d5fe 100644 --- a/docs/docs/guides/4_deployment.md +++ b/docs/docs/guides/4_deployment.md @@ -1,6 +1,6 @@ # Deployment -After we are done with running and veryfing that the recipes works with the data-source and sink you have mentioned. +After we are done with running and verifying that the recipes works with the data-source and sink you have mentioned. You may want to automate the process of metadata collection on some regular basis as a cron job. One can setup user for the same. diff --git a/docs/docs/guides/introduction.md b/docs/docs/guides/introduction.md index f34e8a774..552de1e11 100644 --- a/docs/docs/guides/introduction.md +++ b/docs/docs/guides/introduction.md @@ -9,9 +9,9 @@ The tour takes you through how you can use meteor as an end user and is a great - Install Homebrew (for macOS) or Docker (if using docker image). -- You are required to have atleast one Database, Dashboard or any other Data source and their proper credentials ready. +- You are required to have at least one Database, Dashboard or any other Data source and their proper credentials ready. For e.g. Postgres, MySQL, MSSQL, MongoDB, GitHub, etc. - For sink we will be using console, if you are planning to use compass or Kafka please make necessary changes accordingly. -Once done with the pre-requisites Get started with [Installation](./0_installation.md). +Once done with the prerequisites Get started with [Installation](./0_installation.md). diff --git a/docs/docs/reference/commands.md b/docs/docs/reference/commands.md index cbbf0dbe8..1eb545135 100644 --- a/docs/docs/reference/commands.md +++ b/docs/docs/reference/commands.md @@ -2,13 +2,13 @@ Meteor currently supports the following commands and these can be utilised after the installation: -* completion: generate the autocompletion script for the specified shell +* completion: generate the auto completion script for the specified shell * [gen](#creating-sample-recipes): The recipe will be printed on standard output. Specify recipe name with the first argument without extension. -Use commma to separate multiple sinks and processors. +Use comma to separate multiple sinks and processors. -* [help](#get-help-on-commands-when-stuck): to help user with meteor. +* [help](#get-help-on-commands-when-stuck): to help the user with meteor. * [info](#getting-information-about-plugins): Info command is used to get suitable information about various plugins. Specify the type of plugin as extractor, sink or processor. @@ -19,7 +19,7 @@ Helps in avoiding any failure during running the meteor due to invalid recipe fo * [list](#listing-all-the-plugins): used to state all the plugins of a certain type. -* [run](#running-recipes): the command is used for running the metadata extraction as per the instructions in recipe. +* [run](#running-recipes): the command is used for running the metadata extraction as per the instructions in the recipe. Can be used to run a single recipe, a directory of recipes or all the recipes in the current directory. ## Listing all the plugins @@ -59,7 +59,7 @@ $ meteor info extractor postgres ## Generating Sample recipe\(s\) Since recipe is the main resource of Meteor, we first need to create it before anything else. -You can create a sample recipe usin the gen command. +You can create a sample recipe using the gen command. ```bash # generate a sample recipe @@ -69,7 +69,7 @@ $ meteor gen recipe sample -e bigquery -s console # generate recipe with multiple sinks $ meteor gen recipe sample -e bigquery -s compass,kafka -# extracor(-e) as postgres, multiple sinks(-s) and enrich processor(-p) +# extractor(-e) as postgres, multiple sinks(-s) and enrich processor(-p) # save the generated recipe to a recipe.yaml meteor gen recipe sample -e postgres -s compass,kafka -p enrich > recipe.yaml ``` diff --git a/docs/docs/reference/configuration.md b/docs/docs/reference/configuration.md index 6ddb2b1c3..8efed29d5 100644 --- a/docs/docs/reference/configuration.md +++ b/docs/docs/reference/configuration.md @@ -1,6 +1,6 @@ # Configuration -This page contains reference for all the application configurations for Meteor. +This page contains references for all the application configurations for Meteor. ## Table of Contents diff --git a/docs/docs/reference/extractors.md b/docs/docs/reference/extractors.md index 6ba238168..d1889b255 100644 --- a/docs/docs/reference/extractors.md +++ b/docs/docs/reference/extractors.md @@ -1,6 +1,6 @@ # Extractors -Meteor currently support metadata extraction on these data sources. To perform extraction on any of these you need to create a recipe file with instructions as mentioned [here](../concepts/recipe.md). In the `sample-recipe.yaml` add `source` information such as `type` from the table below and `config` for that particular extractor can be found by visiting the link in `type` field. +Meteor currently supports metadata extraction on these data sources. To perform extraction on any of these you need to create a recipe file with instructions as mentioned [here](../concepts/recipe.md). In the `sample-recipe.yaml` add `source` information such as `type` from the table below and `config` for that particular extractor can be found by visiting the link in the `type` field. ## Extractors Feature Matrix diff --git a/docs/docs/reference/metadata_models.md b/docs/docs/reference/metadata_models.md index c434669e5..0e246ad1e 100644 --- a/docs/docs/reference/metadata_models.md +++ b/docs/docs/reference/metadata_models.md @@ -9,27 +9,27 @@ We are currently using the following metadata models: Currently, Meteor provides a metadata extractor for the buckets mentioned [here](../reference/extractors.md) - [Dashboard](https://github.com/odpf/proton/blob/main/odpf/assets/dashboard.proto): - Dashboards are essential part of data analysis and are used to track, analyse and visualization. + Dashboards are an essential part of data analysis and are used to track, analyze and visualize. These Dashboard metadata model includes some basic fields like `urn` and `source`, etc and a list of `Chart`. There are multiple dashboards that are essential for Data Analysis such as metabase, grafana, tableau, etc. - Please refer the list of Dashboards meteor currently supports [here](../reference/extractors.md). + Please refer to the list of Dashboards meteor currently supports [here](../reference/extractors.md). - [Chart](https://github.com/odpf/proton/blob/main/odpf/assets/chart.proto): - Charts are included in all the Dashboard and are result of certain queries in a Dashboard. + Charts are included in all the Dashboard and are the result of certain queries in a Dashboard. Information about them includes the information of the query and few similar details. - [User](https://github.com/odpf/proton/blob/main/odpf/assets/user.proto): This metadata model is used for defining the output of extraction on Users accounts. - Some of these source can be GitHub, Workday, Google Suite, LDAP. - Please refer the list of user meteor currently supports [here](../reference/extractors.md). + Some of these sources can be GitHub, Workday, Google Suite, LDAP. + Please refer to the list of user meteor currently supports [here](../reference/extractors.md). - [Table](https://github.com/odpf/proton/blob/main/odpf/assets/table.proto): This metadata model is being used by extractors based around `databases` or for the ones that store data in tabular format. - It contains various fields that includes `schema` of the table and other access related information. + It contains various fields that include `schema` of the table and other access related information. - [Job](https://github.com/odpf/proton/blob/main/odpf/assets/job.proto): Most of the data is being streamed as queues by kafka or other stack in DE pipeline. - And hence Job is a metadata model build for this purpose. + And hence Job is a metadata model built for this purpose. `Proto` has been used to define these metadata models. To check their implementation please refer [here](https://github.com/odpf/proton/tree/main/odpf/assets). diff --git a/docs/docs/reference/sinks.md b/docs/docs/reference/sinks.md index dd06dd35b..2b9b2b603 100644 --- a/docs/docs/reference/sinks.md +++ b/docs/docs/reference/sinks.md @@ -17,7 +17,7 @@ sinks: `compass` -Upload metadata to a given `type` in [Compass](https://github.com/odpf/meteor/tree/cb12c3ecf8904cf3f4ce365ca8981ccd132f35d0/docs/reference/github.com/odpf/compass/README.md). Request will be send via HTTP to given host. +Upload metadata to a given `type` in [Compass](https://github.com/odpf/meteor/tree/cb12c3ecf8904cf3f4ce365ca8981ccd132f35d0/docs/reference/github.com/odpf/compass/README.md). Request will be sent via HTTP to a given host. ### Sample usage of compass sink