speech interactive Virtual First Officer in web ui / LNM data access API in web ui for arbitrary extensions

[u-an-i]

Virtual First Officer: creation configuration for everyone

|
v

based on:

|
v

• static data
  • (flight plan, ..., elevation of ground above sealevel (for any coordinate), ..., aircraft call sign, ...)
• dynamic data
  • (position, ... , weather, ..., angle in space (if exposed), ...)
• delta data = diff(s) between static data and dynamic data
  • (position to closest projected position on flightplan, ..., heeading to bearing, ..., distance over ground, ..., angle over ground (if ground slope exposed), ... angle of attack (if wind direction exposed), ..., fuel burnt since take-off, ... fuel remaining til tank empty, ...)
• rate data = diff of delta data at time interval boundaries divided by duration of that time interval
  • (speed across ground, ..., fuel consumption rate, ..., vertical speed, ...)

some data might need Type definitions ie are objects with subproperties eg position = {x: __, y: __, z: __} or = {lat: __, lng: __} which a VFO creator can rely on

|
v

(C++) data to JSON

|
v

author of a VFO:

hooks = (JSON array of objects:)

{
"custom registration identifier": "(string)",
"data property(/ies) name": ["(string)"],
"condition(s) for data property value(s) (with custom names as multiple severities)": {
  "severity1": [
    [["[0][\"x\"] (string to operate on data properties)", "> (string of predefined operators)", "1 (number, string or data property operator string"], "AND condition (array of condition items)"],
    ["OR conditions (array of AND conditions)"],
    ["OR conditions (array of AND conditions)"]
  ]
},
"dependency on history of (other) custom registration identifier got triggered including [max time since last, minimum amount in past time, severity of last]": {
  "other custom registration identifier having gotten triggered": {
    "max time since last": "(number)",
    "minimum amount in passed time": [
      "minimum amount (number)",
      "passed time (number)"
    ],
    "severity of last": "severityMyCustomName (string)"
  }
},
"text to speak (per severity including [pitch, speed and loudness])": {
  "severity1": ["(string)", "(number)", "(number)", "(number)"]
},
"repeat timeout per severity if condition not becomes false / out of range during timeout": {
  "severity1": "(number)"
},
"predefined poll frequency level per severity": {
  "severity1": "low"
},
"text to listen for": "(string)",
"array of possible reactions to be spoken (random pick)": ["(string)", "(string)"]
}

// a passive hook could be created by having text to speak = "" when hooking **weather at destination airport** for example and listening for "what's the weather at our destination" reacting with "The weather is ${data}[i][j]" where ${data} is a placeholder for the data gotten.
// Weather is an example of a property which rather needs to be detailed in several different properties;
// passivity-like behaviour would here be realised with the "active" configuration, a dedicated "passive hook type" will likely be less energy consuming

|
v

(JS logic:)

function board(hooks) {}
function hook(aHook) {}
function poll() {} // request (hooked) data properties from server with poll frequency level mapping possibly overriden (per level) by user of VFO
function evaluateConditions() {}
function speak(text) {}
function listenForReactionAndActUpon() {}

[albar965]

A bit out of scope for LNM IMO.
Maybe we should head for a JS plugin interface. This would allow a lot of customization.

[u-an-i]

right, i got over my head when thinking of what a JS plugin interface could offer

[KOKAProduktion]

Hi @u-an-i,

very cool idea!

I'm joining you guys, that this would probably be a separate application which connects and queries data from LNM through the Web API. So it would be interesting to outline the calls this application may require from it.

Cheers

[albar965]

I scheduled this ticket for 2.8: #397
No worries about the timeframe. I do not expect 2.8 this year. 🙂