[Derivation Pipeline] Handling of failed batches #283
Description
Overview
If the derivation pipeline fails to process a batch, we should handle this case more gracefully to ensure that we do not accumulate too many pending batches in the derivation pipeline.