Updating Bigtable docs.

Addressing review comments, fixing broken links and making
style changes.

Author: dhermes

docs/bigtable-client-intro.rst

@@ -27,10 +27,12 @@ Authorization
 - For an overview of authentication in ``gcloud-python``,
   see :doc:`gcloud-auth`.
 
-- In addition to any authentication configuration, you should also set the
+- In addition to any authentication configuration, you can also set the
   :envvar:`GCLOUD_PROJECT` environment variable for the project you'd like
   to interact with. If you are Google App Engine or Google Compute Engine
-  this will be detected automatically.
+  this will be detected automatically. (Setting this environment
+  variable is not required, you may instead pass the ``project`` explicitly
+  when constructing a :class:`Client <gcloud.storage.client.Client>`).
 
 - After configuring your environment, create a
   :class:`Client <gcloud.storage.client.Client>`
@@ -59,7 +61,7 @@ API requests, you'll need to pass the ``admin`` argument:
 
 .. code:: python
 
-    client = Client(admin=True)
+    client = bigtable.Client(admin=True)
 
 Read-Only Mode
 --------------
@@ -69,7 +71,7 @@ you can pass the ``read_only`` argument:
 
 .. code:: python
 
-    client = Client(read_only=True)
+    client = bigtable.Client(read_only=True)
 
 This will ensure that the
 :data:`READ_ONLY_SCOPE <gcloud.bigtable.client.READ_ONLY_SCOPE>` is used

docs/bigtable-cluster-api.rst

@@ -30,7 +30,7 @@ with :meth:`Client.list_zones() <gcloud.bigtable.client.Client.list_zones>`:
 You can choose a :class:`string <str>` from among the result to pass to
 the :class:`Cluster <gcloud.bigtable.cluster.Cluster>` constructor.
 
-As of right now, the available zones are
+The available zones (as of February 2016) are
 
 .. code:: python
 
@@ -50,7 +50,8 @@ To create a :class:`Cluster <gcloud.bigtable.cluster.Cluster>` object:
 
 Both ``display_name`` and ``serve_nodes`` are optional. When not provided,
 ``display_name`` defaults to the ``cluster_id`` value and ``serve_nodes``
-defaults to the minimum allowed: ``3``.
+defaults to the minimum allowed:
+:data:`DEFAULT_SERVE_NODES <gcloud.bigtable.cluster.DEFAULT_SERVE_NODES>`.
 
 Even if this :class:`Cluster <gcloud.bigtable.cluster.Cluster>` already
 has been created with the API, you'll want this object to use as a
@@ -69,7 +70,9 @@ with :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`:
     cluster.display_name = 'My very own cluster'
     cluster.create()
 
-If you would like more than the minimum number of nodes (``3``) in your cluster:
+If you would like more than the minimum number of nodes
+(:data:`DEFAULT_SERVE_NODES <gcloud.bigtable.cluster.DEFAULT_SERVE_NODES>`)
+in your cluster:
 
 .. code:: python
 
@@ -104,8 +107,9 @@ by making a `GetOperation`_ request with
 .. note::
 
     Once an :class:`Operation <gcloud.bigtable.cluster.Operation>` object
-    has returned :data:`True` from ``finished()``, the object should not
-    be re-used. Subsequent calls to
+    has returned :data:`True` from
+    :meth:`finished() <gcloud.bigtable.cluster.Operation.finished>`, the
+    object should not be re-used. Subsequent calls to
     :meth:`finished() <gcloud.bigtable.cluster.Operation.finished>`
     will result in a :class:`ValueError <exceptions.ValueError>`.
 

docs/bigtable-column-family.rst

@@ -10,7 +10,8 @@ during periodic garbage collection (which executes opportunistically in the
 background).
 
 The types
-:class:`GarbageCollectionRule <gcloud.bigtable.column_family.GarbageCollectionRule>`,
+:class:`MaxAgeGCRule <gcloud.bigtable.column_family.MaxAgeGCRule>`,
+:class:`MaxVersionsGCRule <gcloud.bigtable.column_family.MaxVersionsGCRule>`,
 :class:`GarbageCollectionRuleUnion <gcloud.bigtable.column_family.GarbageCollectionRuleUnion>` and
 :class:`GarbageCollectionRuleIntersection <gcloud.bigtable.column_family.GarbageCollectionRuleIntersection>`
 can all be used as the optional ``gc_rule`` argument in the

docs/bigtable-data-api.rst

@@ -7,8 +7,10 @@ column families, you are ready to store and retrieve data.
 Cells vs. Columns vs. Column Families
 +++++++++++++++++++++++++++++++++++++
 
-* As we saw before, a table can have many column families.
-* As we'll see below, a table also has many rows (specified by row keys).
+* As explained in the :doc:`table overview <bigtable-table-api>`, tables can
+  have many column families.
+* As described below, a table can also have many rows which are
+  specified by row keys.
 * Within a row, data is stored in a cell. A cell simply has a value (as
   bytes) and a timestamp. The number of cells in each row can be
   different, depending on what was stored in each row.
@@ -51,7 +53,7 @@ methods.
 * The **conditional** way is via `CheckAndMutateRow`_. This method
   first checks if some filter is matched in a a given row, then
   applies one of two sets of mutations, depending on if a match
-  occurred or not. (These mutations sets are called the "true
+  occurred or not. (These mutation sets are called the "true
   mutations" and "false mutations".)
 * The **append** way is via `ReadModifyWriteRow`_. This simply
   appends (as bytes) or increments (as an integer) data in a presumed
@@ -142,7 +144,7 @@ Direct mutations can be added via one of four methods
 Conditional Mutations
 ---------------------
 
-Making **conditional** conditional modifications is essentially identical
+Making **conditional** modifications is essentially identical
 to **direct** modifications, but we need to specify a filter to match
 against in the row:
 
@@ -157,13 +159,15 @@ The only other difference from **direct** modifications are that each mutation
 added must specify a ``state``: will the mutation be applied if the filter
 matches or if it fails to match.
 
-For example
+For example:
 
 .. code:: python
 
     row.set_cell(column_family_id, column, value,
                  timestamp=timestamp, state=True)
 
+will add to the set of true mutations.
+
 .. note::
 
     If ``state`` is passed when no ``filter_`` is set on a
@@ -176,15 +180,15 @@ Append Mutations
 
 Append mutations can be added via one of two methods
 
-* :meth:`append_cell_value <gcloud.bigtable.row.Row.append_cell_value>` appends
-  a bytes value to an existing cell:
+* :meth:`append_cell_value() <gcloud.bigtable.row.Row.append_cell_value>`
+  appends a bytes value to an existing cell:
 
   .. code:: python
 
       row.append_cell_value(column_family_id, column, bytes_value)
 
-* :meth:`increment_cell_value <gcloud.bigtable.row.Row.increment_cell_value>` increments
-  an integer value in an existing cell:
+* :meth:`increment_cell_value() <gcloud.bigtable.row.Row.increment_cell_value>`
+  increments an integer value in an existing cell:
 
   .. code:: python
 
@@ -228,7 +232,33 @@ To make a `ReadRows`_ API request for a single row key, use
 
 .. code:: python
 
-    row_data = table.read_row(row_key)
+    >>> row_data = table.read_row(row_key)
+    >>> row_data.cells
+    {
+        u'fam1': {
+            b'col1': [
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+            ],
+            b'col2': [
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+            ],
+        },
+        u'fam2': {
+            b'col3': [
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+                <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>,
+            ],
+        },
+    }
+    >>> cell = row_data.cells[u'fam1'][b'col1'][0]
+    >>> cell
+    <gcloud.bigtable.row_data.Cell at 0x7f80d150ef10>
+    >>> cell.value
+    b'val1'
+    >>> cell.timestamp
+    datetime.datetime(2016, 2, 27, 3, 41, 18, 122823, tzinfo=<UTC>)
 
 Rather than returning a :class:`Row <gcloud.bigtable.row.Row>`, this method
 returns a :class:`PartialRowData <gcloud.bigtable.row_data.PartialRowData>`
@@ -257,10 +287,6 @@ To make a `ReadRows`_ API request for a stream of rows, use
     row_data = table.read_rows()
 
 Using gRPC over HTTP/2, a continual stream of responses will be delivered.
-We have a custom
-:class:`PartialRowsData <gcloud.bigtable.row_data.PartialRowsData>`
-class to allow consuming and parsing these streams as they come.
-
 In particular
 
 * :meth:`consume_next() <gcloud.bigtable.row_data.PartialRowsData.consume_next>`

docs/bigtable-row.rst

@@ -1,11 +1,66 @@
 Bigtable Row
 ============
 
-A
+It is possible to use a :class:`RowFilter <gcloud.bigtable.row.RowFilter>`
+when adding mutations to a :class:`Row <gcloud.bigtable.row.Row>` and when
+reading row data with :meth:`read_row() <gcloud.bigtable.table.Table.read_row>`
+:meth:`read_rows() <gcloud.bigtable.table.Table.read_rows>`.
+
+As laid out in the `RowFilter definition`_, the following basic filters
+are provided:
+
+* :class:`SinkFilter <.row.SinkFilter>`
+* :class:`PassAllFilter <.row.PassAllFilter>`
+* :class:`BlockAllFilter <.row.BlockAllFilter>`
+* :class:`RowKeyRegexFilter <.row.RowKeyRegexFilter>`
+* :class:`RowSampleFilter <.row.RowSampleFilter>`
+* :class:`FamilyNameRegexFilter <.row.FamilyNameRegexFilter>`
+* :class:`ColumnQualifierRegexFilter <.row.ColumnQualifierRegexFilter>`
+* :class:`TimestampRangeFilter <.row.TimestampRangeFilter>`
+* :class:`ColumnRangeFilter <.row.ColumnRangeFilter>`
+* :class:`ValueRegexFilter <.row.ValueRegexFilter>`
+* :class:`ValueRangeFilter <.row.ValueRangeFilter>`
+* :class:`CellsRowOffsetFilter <.row.CellsRowOffsetFilter>`
+* :class:`CellsRowLimitFilter <.row.CellsRowLimitFilter>`
+* :class:`CellsColumnLimitFilter <.row.CellsColumnLimitFilter>`
+* :class:`StripValueTransformerFilter <.row.StripValueTransformerFilter>`
+* :class:`ApplyLabelFilter <.row.ApplyLabelFilter>`
+
+In addition, these filters can be combined into composite filters with
+
+* :class:`RowFilterChain <.row.RowFilterChain>`
+* :class:`RowFilterUnion <.row.RowFilterUnion>`
+* :class:`ConditionalRowFilter <.row.ConditionalRowFilter>`
+
+These rules can be nested arbitrarily, with a basic filter at the lowest
+level. For example:
+
+.. code:: python
+
+    # Filter in a specified column (matching any column family).
+    col1_filter = ColumnQualifierRegexFilter(b'columnbia')
+
+    # Create a filter to label results.
+    label1 = u'label-red'
+    label1_filter = ApplyLabelFilter(label1)
+
+    # Combine the filters to label all the cells in columnbia.
+    chain1 = RowFilterChain(filters=[col1_filter, label1_filter])
+
+    # Create a similar filter to label cells blue.
+    col2_filter = ColumnQualifierRegexFilter(b'columnseeya')
+    label2 = u'label-blue'
+    label2_filter = ApplyLabelFilter(label2)
+    chain2 = RowFilterChain(filters=[col2_filter, label2_filter])
+
+    # Bring our two labeled columns together.
+    row_filter = RowFilterUnion(filters=[chain1, chain2])
 
 ----
 
 .. automodule:: gcloud.bigtable.row
   :members:
   :undoc-members:
   :show-inheritance:
+
+.. _RowFilter definition: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/1ff247c2e3b7cd0a2dd49071b2d95beaf6563092/bigtable-protos/src/main/proto/google/bigtable/v1/bigtable_data.proto#L195

docs/bigtable-table-api.rst

@@ -14,7 +14,9 @@ If you want a comprehensive list of all existing tables in a cluster, make a
 
 .. code:: python
 
-    tables = cluster.list_tables()
+    >>> cluster.list_tables()
+    [<gcloud.bigtable.table.Table at 0x7ff6a1de8f50>,
+     <gcloud.bigtable.table.Table at 0x7ff6a1de8350>]
 
 Table Factory
 -------------
@@ -107,8 +109,7 @@ or similar):
 This rule helps the backend determine when and how to clean up old cells
 in the column family.
 
-See the :doc:`Column Families doc <bigtable-column-family>`
-for more information about
+See :doc:`bigtable-column-family` for more information about
 :class:`GarbageCollectionRule <gcloud.bigtable.column_family.GarbageCollectionRule>`
 and related classes.
 

docs/index.rst

@@ -65,7 +65,6 @@
   bigtable-table-api
   bigtable-data-api
   Client <bigtable-client>
-  bigtable-client
   bigtable-cluster
   bigtable-table
   bigtable-column-family

gcloud/bigtable/client.py

@@ -27,6 +27,8 @@
 """
 
 
+from pkg_resources import get_distribution
+
 from grpc.beta import implementations
 
 from gcloud.bigtable._generated import bigtable_cluster_data_pb2 as data_pb2
@@ -75,7 +77,8 @@
 DEFAULT_TIMEOUT_SECONDS = 10
 """The default timeout to use for API requests."""
 
-DEFAULT_USER_AGENT = 'gcloud-bigtable-python'
+DEFAULT_USER_AGENT = 'gcloud-python/{0}'.format(
+    get_distribution('gcloud').version)
 """The default user agent for API requests."""
 
 

gcloud/bigtable/cluster.py

@@ -34,7 +34,6 @@
 _OPERATION_NAME_RE = re.compile(r'^operations/projects/([^/]+)/zones/([^/]+)/'
                                 r'clusters/([a-z][-a-z0-9]*)/operations/'
                                 r'(?P<operation_id>\d+)$')
-_DEFAULT_SERVE_NODES = 3
 _TYPE_URL_BASE = 'type.googleapis.com/google.bigtable.'
 _ADMIN_TYPE_URL_BASE = _TYPE_URL_BASE + 'admin.cluster.v1.'
 _CLUSTER_CREATE_METADATA = _ADMIN_TYPE_URL_BASE + 'CreateClusterMetadata'
@@ -46,6 +45,9 @@
     _UNDELETE_CREATE_METADATA: messages_pb2.UndeleteClusterMetadata,
 }
 
+DEFAULT_SERVE_NODES = 3
+"""Default number of nodes to use when creating a cluster."""
+
 
 def _prepare_create_request(cluster):
     """Creates a protobuf request for a CreateCluster request.
@@ -216,11 +218,11 @@ class Cluster(object):
 
     :type serve_nodes: int
     :param serve_nodes: (Optional) The number of nodes in the cluster.
-                        Defaults to 3 (``_DEFAULT_SERVE_NODES``).
+                        Defaults to :data:`DEFAULT_SERVE_NODES`.
     """
 
     def __init__(self, zone, cluster_id, client,
-                 display_name=None, serve_nodes=_DEFAULT_SERVE_NODES):
+                 display_name=None, serve_nodes=DEFAULT_SERVE_NODES):
         self.zone = zone
         self.cluster_id = cluster_id
         self.display_name = display_name or cluster_id
@@ -259,8 +261,10 @@ def from_pb(cls, cluster_pb, client):
         :rtype: :class:`Cluster`
         :returns: The cluster parsed from the protobuf response.
         :raises: :class:`ValueError <exceptions.ValueError>` if the cluster
-                 name does not match :data:`_CLUSTER_NAME_RE` or if the parsed
-                 project ID does not match the project ID on the client.
+                 name does not match
+                 ``projects/{project}/zones/{zone}/clusters/{cluster_id}``
+                 or if the parsed project ID does not match the project ID
+                 on the client.
         """
         match = _CLUSTER_NAME_RE.match(cluster_pb.name)
         if match is None:
@@ -277,9 +281,8 @@ def from_pb(cls, cluster_pb, client):
     def copy(self):
         """Make a copy of this cluster.
 
-        Copies the local data stored as simple types but does not copy the
-        current state of any operations with the Cloud Bigtable API. Also
-        copies the client attached to this instance.
+        Copies the local data stored as simple types and copies the client
+        attached to this instance.
 
         :rtype: :class:`.Cluster`
         :returns: A copy of the current cluster.

gcloud/bigtable/column_family.py

@@ -157,7 +157,7 @@ def __eq__(self, other):
         return other.rules == self.rules
 
     def to_pb(self):
-        """Converts the union into a single gc rule as a protobuf.
+        """Converts the union into a single GC rule as a protobuf.
 
         :rtype: :class:`.data_pb2.GcRule`
         :returns: The converted current object.
@@ -183,7 +183,7 @@ def __eq__(self, other):
         return other.rules == self.rules
 
     def to_pb(self):
-        """Converts the intersection into a single gc rule as a protobuf.
+        """Converts the intersection into a single GC rule as a protobuf.
 
         :rtype: :class:`.data_pb2.GcRule`
         :returns: The converted current object.