Fixup CSG reports to batch-send; Tweak services definition for better…

[ZachCMP]

… fallback

Please squash merge this PR

Description

• Updates CSG Engage export to batch send households
• Fixup service report component to have a better fallback if it can't create the service type (seems like it never can)

Type of change

• New feature (adds functionality)

Checklist before requesting review

• I have performed a self-review of my code
• I have run the code that is being changed under ideal conditions, and it doesn't fail
• My code includes comments and/or descriptive variable names to help other engineers understand the intent (or not applicable)
• My code follows the style guidelines of this project (rubocop)
• I have updated the documentation (or not applicable)
• If it's not obvious how to test this change, I have provided testing instructions in this PR or the related issue

[eanders]

@ZachCMP I think this looks good, as long as the receiving end can handle receiving partial data sets. With the previous version it would fail atomically, now it might fail after some number of successful batches, but that's probably fine.

Is there any way to batch load data? I think part of what makes it slow is that it is pulling a query for each client from the database.

[ZachCMP]

@eanders Grant confirmed that the API can accept partial uploads. Ideally we shouldn't send duplicate records, but the API will handle it by just returning an error for any duplicates. I think this will be better, since I think the hanging is caused by trying to serialize and upload so many records at once. And yeah, I agree it'd be best to batch load it. One consideration is that we have to load all the HOH enrollments first, then all the enrollment records for that household, including client records, services, incomes, etc. If you have a good strategy for pre-loading all the information, I'd be happy to implement it.

[eanders]

Thanks @ZachCMP. What we've done for batching in the past is to pluck the HouseholdIDs, and then loop over batches of those pulling all related data. Sometimes those batches need to be a bit smaller (say 250 instead of 1,000) since they bring along multiple clients, but it's worked pretty well.

[ZachCMP]

@eanders I adjusted the batching to more or less do what you describe. Take a look and let me know if you like that better. Also Grant asked that we just send an integer for the income amount field, so I made that change as well. Let me know if you need anything else from me to get this deployed.