This repository was archived by the owner on Mar 24, 2021. It is now read-only.
Releases: Parsely/pykafka
Releases · Parsely/pykafka
2.3.0
Minor Version Features
- Added the
ManagedBalancedConsumerclass, which performs balanced consumption
using the Kafka 0.9 Group Membership API - Added the
managedkeyword argument toTopic.get_balanced_consumerto access
ManagedBalancedConsumer - Added a
compacted_topickwarg toBalancedConsumerto make it smarter about
offset ordering for compacted topics - Added methods to
Brokerthat use the Group Membership API - Changed the terminology "offset manager" to "group coordinator" to match updated
Kafka jargon - Added new exception types from Kafka 0.9
- Added
auto_startkeyword argument toProducerto match the consumer interface - Added
max_request_sizekeyword argument toProducerto catch large messages
before they're sent to Kafka - Added protocol functions for the Group Membership API
- New
SimpleConsumerkeyword arguments:compacted_topic,generation_id,
consumer_id
Bug Fixes
- Fixed a bug in Travis config causing tests not to run against Kafka 0.9
- Upgraded to non-beta gevent version
- Allowed a single
Brokerinstance to maintain multiple connections to a broker
(useful when multiple consumers are sharing the sameKafkaClient) - Allowed switchable socket implementations when using gevent
- Handled
TypeErrorduring worker thread shutdown to avoid nuisance messages - Limited
Producer.min_queued_messagesto 1 whensync=True - Monkeypatched a bug in py.test causing tests to be erroneously skipped
Miscellaneous
- Added an issue template
2.2.1
Bug Fixes
- Fixed Travis issues related to PyPy testing
- Fixed deprecated dependency on gevent.coros
- Enabled caching in Travis for pip, librdkafka, and kafka installations
- Fixed a crash during metadata updating related to zookeeper fallback
- Unified connection retry logic in
Cluster - Raised an exception if consumer offset reset fails after maximum retries
- Fixed a bug allowing
get_delivery_reportto indefinitely blockproduce() - Fixed a bug causing producers to drop
to_retrymessages onstop() - Added retry logic to offset limit fetching
2.2.0
Minor Version Features
- Added support for gevent-based concurrency in pure cpython
- Allowed ZooKeeper hosts to be specified directly to KafkaClient instead of
being treated as a fallback
Bug Fixes
- Fixed a bug causing
RLock-related crashes in Python 3 - Used the more stable sha1 hash function as the default for
hashing_partitioner - Fixed a bug in the meaning of
linger_msin the producer
2.1.2
Features
- Allowed consumers to run with no partitions
Bug Fixes
- Fixed a bug causing consumers to hold outdated partition sets
- Handled some previously uncaught error codes in
SimpleConsumer - Fixed an off-by-one bug in message set fetching
- Made
consume()stricter about message ordering and duplication
2.1.1
Features
- Improved unicode-related error reporting in several components
- Removed the ZooKeeper checker thread from the
BalancedConsumer - Added a test consumer CLI to
kafka_tools
Bug Fixes
- Fixed a memory leak in the rdkafka-based consumer
- Fixed offset committing to work against Kafka 0.9
- Improved the reliability of the Kafka test harness
Miscellaneous
- Simplified the Travis test matrix to handle testing against multiple Kafka versions
2.1.0
Features
- Addded an optional C extension making use of librdkafka for enhanced producer and
consumer performance - Added a delivery report queue to the
Producerallowing per-message errors
to be handled - Added a callback indicating that the
BalancedConsumeris in the process of rebalancing
Bug Fixes
- Fixed a longstanding issue causing certain tests to hang on Travis
- Fixed a bug causing the default error handles in the consumer to mask unknown error
codes - Moved the
Messageclass to using__slots__to minimize its memory footprint
2.0.4
Features
- Allowed discovery of Kafka brokers via a ZooKeeper connect string supplied to
KafkaClient
Bug Fixes
- Made
BalancedConsumer's ZooKeeper watches close quietly on consumer exit - Disconnect sockets in response to any socket-level errors
- Fixed
HashingPartitionerfor python 3
2.0.3
Features
- Raise exceptions from worker threads to the main thread in
BalancedConsumer - Call
stop()whenBalancedConsumeris finalized to minimize zombie threads
Bug Fixes
- Use weak references in
BalancedConsumerworkers to avoid zombie threads creating
memory leaks - Stabilize
BalancedConsumer.start() - Fix a bug in
TopicDict.values()causing topics to be listed asNone - Handle
IOErrorinBrokerConnectionandsocket.recvall_into - Unconditionally update partitions' leaders after metadata requests
- Fix thread-related memory leaks in
Producer - Handle connection errors during offset commits
- Fix an interpreter error in
SimpleConsumer
2.0.2
Features
- Switched the
BalancedConsumerto using ZooKeeper as the single source of truth
about which partitions are held - Made
BalancedConsumerresilient to ZooKeeper failure - Made the consumer resilient to broker failure
Bug Fixes
- Fixed a bug in
BrokerConnectioncausing the message length field to
occasionally be corrupted - Fixed a bug causing
RequestHandlerworker threads to sometimes abort
before the request was completed - Fixed a bug causing
SimpleConsumerto hang when the number of brokers in
the cluster goes below the replication factor
2.0.1
Features
- Added support for python 3.5
- Added iteration to the
BalancedConsumer - Disallowed
min_queued_messages<1inProducer - Made
SimpleConsumercommit offsets onstop() - Supported
Nonein message values
Bug Fixes
- Improved
BalancedConsumer's handling of an externalKazooClientinstance - Fixed
kafka_tools.pyfor Python 3 - Removed the unused
use_greenletskwarg fromKafkaClient - Improved
Cluster's ability to reconnect to brokers during metadata updates - Fixed an interpreter error in
conncection.py - Fixed failure case in
Producerwhenrequired_acks==0 - Fixed a bug causing
SimpleConsumerto leave zombie threads after disconnected brokers - Improved
SimpleConsumer's worker thread exception reporting - Simplified
SimpleConsumer's partition locking logic duringfetchby usingRLock - Fixed
SimpleConsumeroff-by-one error causing lag to never reach 0
Miscellaneous
- Switched from Coveralls to Codecov for converage tracking