2.8.1-dev.1

Bugfixes

• Fixed a bug causing topics to be erroneously reported as present on Topic
• Fixed noisy logging from consumer shutdown

Miscellaneous

• Updated dependencies: gevent, xxhash, pytest, lz4

2.8.0

Minor Version Features

• Added a deserializer kwarg to consumer components to facilitate unicode support
• Added a reset_offset_on_fetch kwarg to consumer components to support read-only
  consumption
• Changed the expected type of the consumer's consumer_group kwarg to str from bytes
• Changed the expected type of TopicDict.__getitem__'s parameter to str from bytes
• Added a pending_timeout_ms kwarg to Producer.__init__ to allow delivery report
  wait timeouts
• Added a serializer kwarg to Producer.__init__ to facilitate unicode support
• Deprecated the generation_id and consumer_id parameters on SimpleConsumer
• Added a partition_offsets kwarg to consumers' commit_offsets method to decouple
  the notions of "committed" vs "consumed" messages
• Added an attempts kwarg to Broker.connect that controls retries during broker
  connection
• Added a queue_empty_timeout_ms kwarg to Producer.__init__ that creates an "empty
  wait" state in the Producer when no messages are available to produce
• Added the zookeeper_hosts kwarg to BalancedConsumer to standardize kwarg naming
• Implemented versioning for ListOffsetRequest
• Changed the behavior of integer arguments passed to reset_offsets

Bugfixes

• Changed consumers to handle valid ascii strings for consumer group names instead of
  bytes
• Handled NoNodeException during consumer ZK node releases
• Used api_versions to select the version-appropriate implementation for
  OffsetFetchRequest
• Adjusted synchronous production logic to avoid infinite blocking when delivery report
  is lost
• Fixed a bug in FetchResponseV1 causing throttle_time to be returned as a tuple
  instead of an integer
• Implemented support for all current versions of OffsetFetchRequest and
  OffsetFetchResponse
• Updated some cli.print_managed_consumer_groups to be Py3 compatible
• Updated the topic creation/deletion CLI to avoid trying to talk to 0.10.0 brokers
• Improved error handling in Cluster.get_group_coordinator
• Added retry logic to BrokerConnection.connect
• Handled some nuisance errors when shutting down in handlers.py
• Added a threading.Event instance to Producer indicating the presence of at least
  one message in the queue to enable nonbusy "empty waiting"
• Added logic to SimpleConsumer.commit_offsets and
  OwnedPartition.build_offset_commit_request that handles user-specified offset
  information and sends it in requests
• Fixed the internal logic of reset_offsets to be more predictable and user-friendly,
  and to better handle the case where the topic has a single log segment
• Standardized the handling of offsets_before arguments across the API
• Added cluster update attempts to produce() retries
• Added a more descriptive error message on certain broker version mismatch errors

Miscellaneous

• Used logging.NullHandler to remove nuisance logs
• Added stock unicode serde to utils
• Added README to pypi info
• Updated version of Kafka used in Travis tests to 1.0.1
• Added usage guide section on connection loss
• Updated test harness to allow simulated killing of brokers
• Added a giant explanatory docstring to Topic.fetch_offset_limits clarifying how the
  ListOffsets API works
• Pinned gevent dependency to avoid breaking change in kazoo, which depends on it
• Added tests for retrying broker connections
• Added tests for user-specified offset commits
• Added usage example on consuming the last N messages from a topic
• Deprecated the zookeeper_connect kwarg on BalancedConsumer
• Split the protocol.py file into multiple smaller files via the protocol module
• Changed the lag monitor CLI to avoid resetting offsets
• Added InvalidTopic to the list of supported exceptions
• Updated requirement versions: lz4, pytest, xxhash
• Removed hacky test-skipping logic from test suite
• xfail test_update_cluster, since it occasionally fails

2.8.0-rc.1

This is a multi-tagging of 2.8.0-dev.5 for the purpose of indicating that this release is slated to become 2.8.0 after a bit of testing.

Changes since 2.7.0

Minor Version Features

• Added a deserializer kwarg to consumer components to facilitate unicode support
• Added a reset_offset_on_fetch kwarg to consumer components to support read-only
  consumption
• Changed the expected type of the consumer's consumer_group kwarg to str from bytes
• Changed the expected type of TopicDict.__getitem__'s parameter to str from bytes
• Added a pending_timeout_ms kwarg to Producer.__init__ to allow delivery report
  wait timeouts
• Added a serializer kwarg to Producer.__init__ to facilitate unicode support
• Deprecated the generation_id and consumer_id parameters on SimpleConsumer
• Added a partition_offsets kwarg to consumers' commit_offsets method to decouple
  the notions of "committed" vs "consumed" messages
• Added an attempts kwarg to Broker.connect that controls retries during broker
  connection
• Added a queue_empty_timeout_ms kwarg to Producer.__init__ that creates an "empty
  wait" state in the Producer when no messages are available to produce
• Added the zookeeper_hosts kwarg to BalancedConsumer to standardize kwarg naming
• Implemented versioning for ListOffsetRequest
• Changed the behavior of integer arguments passed to reset_offsets

Bugfixes

• Changed consumers to handle valid ascii strings for consumer group names instead of
  bytes
• Handled NoNodeException during consumer ZK node releases
• Used api_versions to select the version-appropriate implementation for
  OffsetFetchRequest
• Adjusted synchronous production logic to avoid infinite blocking when delivery report
  is lost
• Fixed a bug in FetchResponseV1 causing throttle_time to be returned as a tuple
  instead of an integer
• Implemented support for all current versions of OffsetFetchRequest and
  OffsetFetchResponse
• Updated some cli.print_managed_consumer_groups to be Py3 compatible
• Updated the topic creation/deletion CLI to avoid trying to talk to 0.10.0 brokers
• Improved error handling in Cluster.get_group_coordinator
• Added retry logic to BrokerConnection.connect
• Handled some nuisance errors when shutting down in handlers.py
• Added a threading.Event instance to Producer indicating the presence of at least
  one message in the queue to enable nonbusy "empty waiting"
• Added logic to SimpleConsumer.commit_offsets and
  OwnedPartition.build_offset_commit_request that handles user-specified offset
  information and sends it in requests
• Fixed the internal logic of reset_offsets to be more predictable and user-friendly,
  and to better handle the case where the topic has a single log segment
• Standardized the handling of offsets_before arguments across the API
• Added cluster update attempts to produce() retries
• Added a more descriptive error message on certain broker version mismatch errors

Miscellaneous

• Used logging.NullHandler to remove nuisance logs
• Added stock unicode serde to utils
• Added README to pypi info
• Updated version of Kafka used in Travis tests to 1.0.1
• Added usage guide section on connection loss
• Updated test harness to allow simulated killing of brokers
• Added a giant explanatory docstring to Topic.fetch_offset_limits clarifying how the
  ListOffsets API works
• Pinned gevent dependency to avoid breaking change in kazoo, which depends on it
• Added tests for retrying broker connections
• Added tests for user-specified offset commits
• Added usage example on consuming the last N messages from a topic
• Deprecated the zookeeper_connect kwarg on BalancedConsumer
• Split the protocol.py file into multiple smaller files via the protocol module
• Changed the lag monitor CLI to avoid resetting offsets
• Added InvalidTopic to the list of supported exceptions
• Updated requirement versions: lz4, pytest, xxhash
• Removed hacky test-skipping logic from test suite
• xfail test_update_cluster, since it occasionally fails

2.8.0-dev.5

(Duplicate of 2.8.0-dev.4)

2.8.0-dev.4

Bugfixes

• Added a more descriptive error message on certain broker version mismatch errors

Miscellaneous

• xfail test_update_cluster, since it occasionally fails

2.8.0-dev.3

Minor Version Features

• Added the zookeeper_hosts kwarg to BalancedConsumer to standardize kwarg naming
• Implemented versioning for ListOffsetRequest
• Changed the behavior of integer arguments passed to reset_offsets

Bugfixes

• Fixed the internal logic of reset_offsets to be more predictable and user-friendly,
  and to better handle the case where the topic has a single log segment
• Standardized the handling of offsets_before arguments across the API
• Added cluster update attempts to produce() retries

Miscellaneous

• Added usage example on consuming the last N messages from a topic
• Deprecated the zookeeper_connect kwarg on BalancedConsumer
• Split the protocol.py file into multiple smaller files via the protocol module
• Changed the lag monitor CLI to avoid resetting offsets
• Added InvalidTopic to the list of supported exceptions
• Updated requirement versions: lz4, pytest, xxhash
• Removed hacky test-skipping logic from test suite

2.8.0-dev.2

Minor Version Features

• Added a partition_offsets kwarg to consumers' commit_offsets method to decouple
  the notions of "committed" vs "consumed" messages
• Added an attempts kwarg to Broker.connect that controls retries during broker
  connection
• Added a queue_empty_timeout_ms kwarg to Producer.__init__ that creates an "empty
  wait" state in the Producer when no messages are available to produce

Bugfixes

• Updated some cli.print_managed_consumer_groups to be Py3 compatible
• Updated the topic creation/deletion CLI to avoid trying to talk to 0.10.0 brokers
• Improved error handling in Cluster.get_group_coordinator
• Added retry logic to BrokerConnection.connect
• Handled some nuisance errors when shutting down in handlers.py
• Added a threading.Event instance to Producer indicating the presence of at least
  one message in the queue to enable nonbusy "empty waiting"
• Added logic to SimpleConsumer.commit_offsets and
  OwnedPartition.build_offset_commit_request that handles user-specified offset
  information and sends it in requests

Miscellaneous

• Updated version of Kafka used in Travis tests to 1.0.1
• Added usage guide section on connection loss
• Updated test harness to allow simulated killing of brokers
• Added a giant explanatory docstring to Topic.fetch_offset_limits clarifying how the
  ListOffsets API works
• Pinned gevent dependency to avoid breaking change in kazoo, which depends on it
• Added tests for retrying broker connections
• Added tests for user-specified offset commits

2.8.0-dev.1

Minor Version Features

• Added a deserializer kwarg to consumer components to facilitate unicode support
• Added a reset_offset_on_fetch kwarg to consumer components to support read-only
  consumption
• Changed the expected type of the consumer's consumer_group kwarg to str from bytes
• Changed the expected type of TopicDict.__getitem__'s parameter to str from bytes
• Added a pending_timeout_ms kwarg to Producer.__init__ to allow delivery report
  wait timeouts
• Added a serializer kwarg to Producer.__init__ to facilitate unicode support
• Deprecated the generation_id and consumer_id parameters on SimpleConsumer

Bugfixes

• Changed consumers to handle valid ascii strings for consumer group names instead of
  bytes
• Handled NoNodeException during consumer ZK node releases
• Used api_versions to select the version-appropriate implementation for
  OffsetFetchRequest
• Adjusted synchronous production logic to avoid infinite blocking when delivery report
  is lost
• Fixed a bug in FetchResponseV1 causing throttle_time to be returned as a tuple
  instead of an integer
• Implemented support for all current versions of OffsetFetchRequest and
  OffsetFetchResponse

Miscellaneous

• Used logging.NullHandler to remove nuisance logs
• Added stock unicode serde to utils
• Added README to pypi info

2.7.0

Minor Version Features

• Added a broker_version kwarg to Broker.__init__ for the purpose of setting
  api_version in FetchResponse
• Added a topic_name argument to Broker.join_group for use in protocol metadata,
  visible via the Administrative API
• Added a function print_managed_consumer_groups to the CLI
• Added a timestamp kwarg to Producer.produce to pass on messages when the broker
  supports newer message formats
• Changed Producer.produce to return the produced Message instance
• Added protocol_version and timestamp kwargs to Message
• Added support for the fetch_error_backoff_ms kwarg on SimpleConsumer
• Added an unblock_event kwarg to SimpleConsumer.consume used to notify the consumer
  that its parent BalancedConsumer is in the process of rebalancing
• Added a general-purpose cleanup function to SimpleConsumer
• Added a membership_protocol kwarg to BalancedConsumer that allows switchable and
  user-defined membership protocols to be used
• Implemented GroupMembershipProtocol objects for the two standard partition assignment
  strategies
• Added an api_versions kwarg to Broker to facilitate switchable API protocol versions
• Added support for all versions of the MetadataRequest to Broker
• Added the controller_broker attribute to Cluster
• Added create_topics and delete_topics to Broker
• Added fetch_api_versions to Broker and Cluster
• Added a CLI for creating and deleting topics on the cluster to kafka_tools
• Added support for LZ4 compression to the Producer and SimpleConsumer

Bug Fixes

• Added an Event that notifies the internal SimpleConsumer of a BalancedConsumer
  that a rebalance is in progress, fixing a bug causing partitions to be unreleased
• Fixed a bug causing busywaiting in the BalancedConsumer when there are no partitions
  available
• Updated the protocol implementation to send non-empty GroupMembershipProtocol
  objects and become compatible with the Administrative API
• Fixed a bytestring bug causing kafka_tools.reset_offsets not to work in python 3
• Added a separate retry limit on connections to the offset manager
• Improved logging on socket errors
• Fixed a bug causing API version not to be passed on certain requests
• Handled new MessageSet compression scheme in API v1
• Fixed a bug in rdkafka.SimpleConsumer causing exceptions not to be raised from worker
  threads
• Fixed a bug causing fetch_offsets not to raise exceptions under certain conditions
  when it should
• Adjusted Cluster to become aware of supported API versions immediately upon
  instantiation
• Refactored code in Cluster related to metadata requests to make logic reusable for
  pre-bootstrap communication with the cluster
• Added the ability to pass arguments to protocol.Response instances when waiting
  on a future
• Adjusted the RandomPartitioner to avoid actually calling random.choice to improve
  performance
• Removed some calls in Producer.procuce to isinstance to improve performance
• Simplified retry logic in SimpleConsumer.fetch_offsets

Miscellaneous

• Separated gevent tests from other builds in Travis
• Made dependency on gevent optional
• Added a convenient CLI entry point via __main__
• Fixed exception naming convention to align with naming in the broker
• Avoided building the rdkafka extension on platforms that don't support it
• Fixed a bug in test harness causing some tests not to be inherited from parent classes
• Used sudo: required to get around dead Travis machines
• Upgraded Travis tests to use Kafka 1.0.0
• Added Code of Conduct
• Documented release process
• Made PyKafka available via conda-forge
• Fleshed out the beginning of the usage guide
• Made kafka_instance fetch its binary from archive.apache.org instead of
  mirror.reverse.net because the latter removed old versions of Kafka

2.7.0-dev.2

Minor Version Features

• Added a membership_protocol kwarg to BalancedConsumer that allows switchable and
  user-defined membership protocols to be used
• Implemented GroupMembershipProtocol objects for the two standard partition assignment
  strategies
• Added an api_versions kwarg to Broker to facilitate switchable API protocol versions
• Added support for all versions of the MetadataRequest to Broker
• Added the controller_broker attribute to Cluster
• Added create_topics and delete_topics to Broker
• Added fetch_api_versions to Broker and Cluster
• Added a CLI for creating and deleting topics on the cluster to kafka_tools
• Added support for LZ4 compression to the Producer and SimpleConsumer

Bug Fixes

• Adjusted Cluster to become aware of supported API versions immediately upon
  instantiation
• Refactored code in Cluster related to metadata requests to make logic reusable for
  pre-bootstrap communication with the cluster
• Added the ability to pass arguments to protocol.Response instances when waiting
  on a future
• Adjusted the RandomPartitioner to avoid actually calling random.choice to improve
  performance
• Removed some calls in Producer.procuce to isinstance to improve performance
• Simplified retry logic in SimpleConsumer.fetch_offsets

Miscellaneous

• Used sudo: required to get around dead Travis machines
• Upgraded Travis tests to use Kafka 1.0.0
• Added Code of Conduct
• Documented release process
• Made PyKafka available via conda-forge
• Fleshed out the beginning of the usage guide
• Made kafka_instance fetch its binary from archive.apache.org instead of
  mirror.reverse.net because the latter removed old versions of Kafka