Publish answers to question topic
Summary
This pull request removes the use of answer topics by publishing answer messages to the service revision (formerly known as question) topic and filtering subscriptions to only receive a) questions or b) response messages to a specific question. This speeds up the question asking process, reduces cloud infrastructure requirements and the permissions surface, and allows us to avoid topic number limits.
Also added is validation of all messages and their attributes against a new publicly available schema. This ensures services are communicating as they should and opens up the possibility of writing services in other languages and creating emulators.
As this, by itself, constitutes an inter-service communication breaking change, we've taken the opportunity to reduce the complexity of the codebase by removing backwards compatibility patches for service communication (i.e. we've grouped multiple breaking changes together into one).
Contents (#603)
IMPORTANT: There are 7 breaking changes.
New features
- Validate messages and their attributes against new service communication schema (see #614 for changelog - it was merged into this branch)
- Allow diagnostics (formerly known as crash diagnostics) to always be switched on for a service
Enhancements
- 💥 BREAKING CHANGE: Publish responses to questions to the service revision (question) topic instead of creating a separate answer topic
- 💥 BREAKING CHANGE: Store message number in message attributes instead of in message data
- 💥 BREAKING CHANGE: Remove question UUID from log record message body
- 💥 BREAKING CHANGE: Remove inter-service communication backwards compatibility code
- 💥 BREAKING CHANGE: Make input and output values and manifest optional
- 💥 BREAKING CHANGE: Replace boolean
allow_save_diagnostics_data_on_crashargument with string/enumsave_diagnosticsargument inService.askand related methods - Add ability to filter subscriptions
- Add question UUID attribute to all messages
- Send more possible errors to parent in
Service.answer - Add
kindfield to question messages - Add
sender_typeattribute to all messages - Add ability to instantiate
Runnerfrom service/app configurations
Fixes
- Stop double-JSON-encoding output manifests
Dependencies
- Update
octueversion in template apps' dependencies
Refactoring
- 💥 BREAKING CHANGE: Rename crash diagnostics to diagnostics
- Group message attributes in
Service._send_messageandMockMessageunder explicitattributesargument - Make
OrderedMessageHandler._waiting_messagesattribute public - Rename various message attributes
Testing
- Store mock pub/sub messages against subscriptions instead of topics
- Add missing
typefield to emulated Pub/Sub questions
Operations
- Fix
add-issues-to-octue-boardworkflow - Stop automatically building docker images for registry in
releaseworkflow - Add ReadTheDocs config file to fix documentation building
Upgrade instructions
💥 Update all Octue services in your network to use this version of octue so they're still able to communicate. Postpone upgrading until you can upgrade all services simultaneously.
💥 Replace allow_save_diagnostics_data_on_crash with save_diagnostics set to one of these values: "SAVE_DIAGNOSTICS_OFF", "SAVE_DIAGNOSTICS_ON_CRASH", or "SAVE_DIAGNOSTICS_ON"
💥 Crash diagnostics rename:
- Use the
octue get-diagnosticsCLI command instead of theoctue get-crash-diagnosticscommand - Rename
crash_diagnostics_cloud_pathin your service configurations todiagnostics_cloud_path