Add Core Subtopologies: ComFprime, ComCcsds, DataProducts, FileHandling (#3768)

* Added led-blinker workflow for aarch64-linux platform

* Update runner label

* Added Tags to RPI self hosted workflows, sparse-checkout-cone-mode false

* Add subtopologies to Svc/ restructing of Ref to include CDH subtopology

* CDHCore Subtopology added under Svc and integrated into Ref

* Health instance within CDHCore references CDHCoreConfig for queue size

* Update metadata

check-spelling run (push) for add-subtopologies

Signed-off-by: check-spelling-bot <[email protected]>
on-behalf-of: @check-spelling <[email protected]>

* Added AssertFatalAdapter Version PassiveTextLogger to Subtopology, config clarity improvements

* Comms subtopology without cmdSeq created, integrated in Ref

* Added cmdSeq to Comms Subtopology, integrated into Ref

* Add FileHandling Subtopology, initial structure

* Fixed Spelling, Flie -> File

* Update metadata

check-spelling run (push) for add-subtopologies

Signed-off-by: check-spelling-bot <[email protected]>
on-behalf-of: @check-spelling <[email protected]>

* Added working FileHandling, DataProducts subtopology, integrated into Ref

* prmDb part of FileHandling, fatalHandler part of CDHCore

* Update metadata

check-spelling run (push) for add-subtopologies

Signed-off-by: check-spelling-bot <[email protected]>
on-behalf-of: @check-spelling <[email protected]>

* Update comment for clarity

* Initial move to phasing components

* Fixing Phasing: Adding initial Phasing for CDHCore, DataProducts, FileHandling

* Renaming topologydefs.hpp, phasing fully working on all four subtopologies, needs cleaning up

* Cleanup with Ports_ComPacketQueue, definition only in Comms.fpp now

* Base project with all 4 subtopologies, integrated into Ref, and passing CI

* Initial Commit with all Subtopologies in Svc/Subtopologies

* Added posixtime back in to Ref Deployment (Accidentally Deleted)

* Updates to match new subtopology structure, integrate CCSDS into Ref

* Changes to CommCCSDS that go with last changes

* Standardize subtopology structure and naming to PascalCase

* Namespace fixe for ComFprime

* CDHCore->Cdhcore for consistency

* Rename CDHCore to CdhCore for consistency

* Update metadata

check-spelling run (pull_request_target) for add-subtopologies

Signed-off-by: check-spelling-bot <[email protected]>
on-behalf-of: @check-spelling <[email protected]>

* Reorder on teardown phases for Com Subtopolgies

* Cpp check style fix

* Fix: Add missing tearDownComponents cleanup for DataProducts dpBufferManager

* Delete old comments, test using same mallocator for all subtopologies

* fix: Deallocated DataProducts BufferMgr

* Fix DataProducts subtopology memory cleanup

* Fix: Dataproducts subtopology teardown order

* Added tearDownComponents(state)

* Consolidate all cleanup to teardowncomponents

* Fix memory leaks: restore teardown phases

* Removal of redundant teardowncomponents()

* Revert "Removal of redundant teardowncomponents()"

This reverts commit 29d4ff8.

* Fix formatting, remove commented out code

* Swap hardcoded numbers to config constants in subtopologies

* register_fprime_config() in config modules, isolated ComDriver into a separate config fpp file

* Added explicit CMake Module Names and Depends between Subtopologies and their Configs

* Fixed CCSDS case, more config constants

* Added explicit depends for each subtopology config

* added mallocator config, subtopology state structure

* Update subtopology config names for clarity, fixes #3571

* Fix reference to dpBuffer config Constants

---------

Signed-off-by: check-spelling-bot <[email protected]>
Co-authored-by: Moises Mata <[email protected]>

Author: moisesmata

.github/actions/spelling/expect.txt

@@ -68,7 +68,7 @@ CCB
 CComponent
 ccsds
 ccsparc
-CDH
+cdh
 CDHCORE
 cerrno
 CFDP
@@ -95,6 +95,10 @@ CODEFILE
 COLORSTYLE
 colorwheel
 COMBUFFER
+COMCCSDS
+COMCCSDSSUBTOPOLOGY
+COMFPRIME
+COMFPRIMESUBTOPOLOGY
 comlogger
 COMMANDDISPATCHERIMPL
 COMMANDDISPATCHERIMPLCFG
@@ -133,6 +137,8 @@ ctu
 culates
 cuz
 CYCLEOUT
+DATAPRODUCTS
+DATAPRODUCTSSUBTOPOLOGY
 DATAROOTDIR
 DDDTHH
 Debian
@@ -213,11 +219,12 @@ ffff
 Ffs
 FILEDOWNLINK
 FILEDOWNLINKCFG
+FILEHANDLING
+FILEHANDLINGSUBTOPOLOGY
 FILEID
 FILEOPENERROR
 FILEWRITEERROR
 fio
-FLDP
 FNDELAY
 FONTPATH
 foodoodie
@@ -390,7 +397,6 @@ multitool
 mutexattr
 Mutexed
 mycompany
-NACI
 nasafprime
 nbits
 ncsl
@@ -451,7 +457,6 @@ PASSIVERATEGROUP
 PASSIVERATEGROUPIMPLTESTER
 patsubst
 pdflatex
-PEB
 penv
 PERLMOD
 PINGENTRIES

Ref/Main.cpp

@@ -82,8 +82,8 @@ int main(int argc, char* argv[]) {
     }
     // Object for communicating state to the reference topology
     Ref::TopologyState inputs;
-    inputs.hostname = hostname;
-    inputs.port = port_number;
+    inputs.comCcsds.hostname = hostname;
+    inputs.comCcsds.port = port_number;
 
     // Setup program shutdown via Ctrl-C
     signal(SIGINT, signalHandler);

Ref/Top/RefPackets.fppi

@@ -1,39 +1,44 @@
 telemetry packets RefPackets {
 
   packet CDH id 1 group 1 {
-    CDHCore.cmdDisp.CommandsDispatched
+    CdhCore.cmdDisp.CommandsDispatched
+
+    FileHandling.fileUplink.FilesReceived
+    FileHandling.fileUplink.PacketsReceived
+    FileHandling.fileDownlink.FilesSent
+    FileHandling.fileDownlink.PacketsSent
+    FileHandling.fileManager.CommandsExecuted
+
+    ComCcsds.cmdSeq.CS_LoadCommands
+    ComCcsds.cmdSeq.CS_CancelCommands
+    ComCcsds.cmdSeq.CS_CommandsExecuted
+    ComCcsds.cmdSeq.CS_SequencesCompleted
+    ComCcsds.comQueue.comQueueDepth
+    ComCcsds.comQueue.buffQueueDepth
+    ComCcsds.commsBufferManager.TotalBuffs
+    ComCcsds.commsBufferManager.CurrBuffs
+    ComCcsds.commsBufferManager.HiBuffs
+    #ComCcsds.tlmSend.SendLevel
+
     Ref.rateGroup1Comp.RgMaxTime
     Ref.rateGroup2Comp.RgMaxTime
     Ref.rateGroup3Comp.RgMaxTime
-    Ref.cmdSeq.CS_LoadCommands
-    Ref.cmdSeq.CS_CancelCommands
-    Ref.cmdSeq.CS_CommandsExecuted
-    Ref.cmdSeq.CS_SequencesCompleted
-    Ref.fileUplink.FilesReceived
-    Ref.fileUplink.PacketsReceived
-    Ref.commsBufferManager.TotalBuffs
-    Ref.commsBufferManager.CurrBuffs
-    Ref.commsBufferManager.HiBuffs
-    Ref.fileDownlink.FilesSent
-    Ref.fileDownlink.PacketsSent
-    Ref.fileManager.CommandsExecuted
-    Ref.comQueue.comQueueDepth
-    Ref.comQueue.buffQueueDepth
-    #CDHCore.tlmSend.SendLevel
   }
 
   packet CDHErrors id 2 group 1 {
+    CdhCore.$health.PingLateWarnings
+
+    FileHandling.fileUplink.Warnings
+    FileHandling.fileDownlink.Warnings
+    FileHandling.fileManager.Errors
+
+    ComCcsds.cmdSeq.CS_Errors
+    ComCcsds.commsBufferManager.NoBuffs
+    ComCcsds.commsBufferManager.EmptyBuffs
+
     Ref.rateGroup1Comp.RgCycleSlips
     Ref.rateGroup2Comp.RgCycleSlips
     Ref.rateGroup3Comp.RgCycleSlips
-    Ref.cmdSeq.CS_Errors
-    Ref.fileUplink.Warnings
-    Ref.fileDownlink.Warnings
-    CDHCore.$health.PingLateWarnings
-    Ref.fileManager.Errors
-    Ref.commsBufferManager.NoBuffs
-    Ref.commsBufferManager.EmptyBuffs
-    Ref.fileManager.Errors
   }
 
   packet DriveTlm id 3 group 1 {
@@ -175,94 +180,97 @@ telemetry packets RefPackets {
   }
 
   packet DataProducts id 21 group 3 {
-    Ref.dpCat.CatalogDps
-    Ref.dpCat.DpsSent
-    Ref.dpMgr.NumSuccessfulAllocations
-    Ref.dpMgr.NumFailedAllocations
-    Ref.dpMgr.NumDataProducts
-    Ref.dpMgr.NumBytes
-    Ref.dpWriter.NumBuffersReceived
-    Ref.dpWriter.NumBytesWritten
-    Ref.dpWriter.NumSuccessfulWrites
-    Ref.dpWriter.NumFailedWrites
-    Ref.dpWriter.NumErrors
-    Ref.dpBufferManager.TotalBuffs
-    Ref.dpBufferManager.CurrBuffs
-    Ref.dpBufferManager.HiBuffs
-    Ref.dpBufferManager.NoBuffs
-    Ref.dpBufferManager.EmptyBuffs
+    DataProducts.dpCat.CatalogDps
+    DataProducts.dpCat.DpsSent
+
+    DataProducts.dpMgr.NumSuccessfulAllocations
+    DataProducts.dpMgr.NumFailedAllocations
+    DataProducts.dpMgr.NumDataProducts
+    DataProducts.dpMgr.NumBytes
+
+    DataProducts.dpWriter.NumBuffersReceived
+    DataProducts.dpWriter.NumBytesWritten
+    DataProducts.dpWriter.NumSuccessfulWrites
+    DataProducts.dpWriter.NumFailedWrites
+    DataProducts.dpWriter.NumErrors
+
+    DataProducts.dpBufferManager.TotalBuffs
+    DataProducts.dpBufferManager.CurrBuffs
+    DataProducts.dpBufferManager.HiBuffs
+    DataProducts.dpBufferManager.NoBuffs
+    DataProducts.dpBufferManager.EmptyBuffs
   }
 
   packet Version1 id 22 group 2 {
-    CDHCore.version.FrameworkVersion
-    CDHCore.version.ProjectVersion
+    CdhCore.version.FrameworkVersion
+    CdhCore.version.ProjectVersion
   }
 
   packet Version_Library1 id 23 group 2 {
-    CDHCore.version.LibraryVersion01
-    CDHCore.version.LibraryVersion02
+    CdhCore.version.LibraryVersion01
+    CdhCore.version.LibraryVersion02
   }
 
   packet Version_Library2 id 24 group 2 {
-    CDHCore.version.LibraryVersion03
-    CDHCore.version.LibraryVersion04
+    CdhCore.version.LibraryVersion03
+    CdhCore.version.LibraryVersion04
   }
 
   packet Version_Library3 id 25 group 2 {
-    CDHCore.version.LibraryVersion05
-    CDHCore.version.LibraryVersion06
+    CdhCore.version.LibraryVersion05
+    CdhCore.version.LibraryVersion06
   }
 
   packet Version_Library4 id 26 group 2 {
-    CDHCore.version.LibraryVersion07
-    CDHCore.version.LibraryVersion08
+    CdhCore.version.LibraryVersion07
+    CdhCore.version.LibraryVersion08
   }
 
   packet Version_Library5 id 27 group 2 {
-    CDHCore.version.LibraryVersion09
-    CDHCore.version.LibraryVersion10
+    CdhCore.version.LibraryVersion09
+    CdhCore.version.LibraryVersion10
   }
 
   packet Version_Custom1 id 28 group 2 {
-    CDHCore.version.CustomVersion01
+    CdhCore.version.CustomVersion01
   }
 
   packet Version_Custom2 id 29 group 2 {
-    CDHCore.version.CustomVersion02
+    CdhCore.version.CustomVersion02
   }
 
   packet Version_Custom3 id 30 group 2 {
-    CDHCore.version.CustomVersion03
+    CdhCore.version.CustomVersion03
   }
 
   packet Version_Custom4 id 31 group 2 {
-    CDHCore.version.CustomVersion04
+    CdhCore.version.CustomVersion04
   }
 
   packet Version_Custom5 id 32 group 2 {
-    CDHCore.version.CustomVersion05
+    CdhCore.version.CustomVersion05
   }
 
   packet Version_Custom6 id 33 group 2 {
-    CDHCore.version.CustomVersion06
+    CdhCore.version.CustomVersion06
   }
 
   packet Version_Custom7 id 34 group 2 {
-    CDHCore.version.CustomVersion07
+    CdhCore.version.CustomVersion07
   }
 
   packet Version_Custom8 id 35 group 2 {
-    CDHCore.version.CustomVersion08
+    CdhCore.version.CustomVersion08
   }
 
   packet Version_Custom9 id 36 group 2 {
-    CDHCore.version.CustomVersion09
+    CdhCore.version.CustomVersion09
   }
 
   packet Version_Custom10 id 37 group 2 {
-    CDHCore.version.CustomVersion10
+    CdhCore.version.CustomVersion10
   }
 
 } omit {
-  CDHCore.cmdDisp.CommandErrors
-}
+  CdhCore.cmdDisp.CommandErrors
+}