Orquesta - Unable to serialize list objects

[nmaludy]

SUMMARY

When running an Orquesta workflow and trying to use the to_yaml_string filter with a list object, an error is raised even though the object is valid JSON.

ISSUE TYPE

• Bug Report

STACKSTORM VERSION

$ st2 --version
st2 3.0.0, on Python 2.7.5

OS / ENVIRONMENT / INSTALL METHOD

OS = Centos 7.6
Install = puppet-st2

STEPS TO REPRODUCE

/opt/stackstorm/packs/default/actions/orquesta_list_yaml_error.yaml

---
name: orquesta_list_yaml_error
description: A basic sequential workflow.
runner_type: orquesta
entry_point: workflows/orquesta_list_yaml_error.yaml
enabled: true
parameters: {}

/opt/stackstorm/packs/default/actions/workflows/orquesta_list_yaml_error.yaml

version: 1.0
  
description: A basic sequential workflow.

vars:
  - blah:
      - a
      - b
      - c

tasks:
  demo:
    action: core.local
    input:
      cmd: "echo '{{ ctx().blah | to_yaml_string }}'"

EXPECTED RESULTS

Workflow runs successfully

ACTUAL RESULTS

$ st2 run default.orquesta_list_yaml_error
.
id: 5cf5df6b9387ef5a16b63f71
action.ref: default.orquesta_list_yaml_error
parameters: None
status: failed
start_timestamp: Tue, 04 Jun 2019 03:03:07 UTC
end_timestamp: Tue, 04 Jun 2019 03:03:07 UTC
result: 
  errors:
  - message: 'JinjaEvaluationException: Unable to evaluate expression ''{{ ctx().blah | to_yaml_string }}''. RepresenterError: (''cannot represent an object'', [u''a'', u''b'', u''c''])'
    route: 0
    task_id: demo                                                                                                            
    type: error                                                                                                              
  output: null 

[nmaludy]

After doing LOTS of digging (had to add debug statements to the YAML module) that the data type of the list is not actually a list but instead a mongoengine.base.datastructures.BaseList. For some reason these lists aren't be deserialized properly.

$ st2 run default.orquesta_list_yaml_error                                                        
.                                                                                                                            
id: 5cf5df929387ef5a16b63f74                                                                                                 
action.ref: default.orquesta_list_yaml_error                                                                                 
parameters: None                                                                                                             
status: failed                                                                                                               
start_timestamp: Tue, 04 Jun 2019 03:03:46 UTC                                                                               
end_timestamp: Tue, 04 Jun 2019 03:03:46 UTC                                                                                 
result:                                                                                                                      
  errors:                                                                                                                    
  - message: 'JinjaEvaluationException: Unable to evaluate expression ''{{ ctx().blah | to_yaml_string }}''. RepresenterError: ("cannot represent an object data is type=<class ''mongoengine.base.datastructures.BaseList''>", [u''a'', u''b'', u''c''])'
    route: 0
    task_id: demo
    type: error
  output: null

I added the debug statement to /opt/stackstorm/st2/lib/python2.7/site-packages/yaml/representer.py line 251

    def represent_undefined(self, data):
        raise RepresenterError("cannot represent an object data is type={}".format(type(data)), data)

@m4dcoder as another helpful thing, it would be great if errors raised during execution (or even Jinja rendering) actually returned stack traces or at least printed them in the log files, it took me forever to hunt down where this was being thrown from

[m4dcoder]

@nmaludy Please open a separate issue on logging stack traces. This needs to be addressed at the orquesta library and workflow service. We should not mix it w/ this bug fix.