Automatic Packet Downloader endpoint/action #1079
Description
We need a way to retrigger retrieval of packets from the POC. We have had numerous situations now where we aren't getting packets through our automated system due to delays in packet arrival or out of order issues with timestamps. When this happens, we can generally go and get the packets again later and it magically works. So it would be nice to have a few things here:
- A more robust way of downloading packets
- A way of automatically requesting packets to be re-ingested for a given date/instrument (API endpoint?)
- A way to compare two packet files (or potential date ranges) to see if we need to version bump a file or not, i.e. if more packets have arrived or not
- A way of tracking gaps in packet sequences and ids to know if we got the data we were expecting to get