Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
5fb78d2
st2 API and CLI command added for actions/workflows clone operation
ashwini-orchestral Aug 27, 2021
411f38f
Clone api for actions/workflows- adding openapi.yaml file
ashwini-orchestral Aug 27, 2021
89a94a4
Modifications done as per code review comments
ashwini-orchestral Sep 14, 2021
ba17481
Adding RBAC unit tests for clone action API
ashwini-orchestral Sep 14, 2021
0dedd4a
Refactoring clone action function and unit tests
ashwini-orchestral Sep 15, 2021
4ca98b6
Merge branch 'master' into clone_action_api
ashwini-orchestral Sep 15, 2021
89b71c9
Merge branch 'master' into clone_action_api
ashwini-orchestral Sep 28, 2021
5a1f1a8
Refactored clone_actions function to create workflows directory if do…
ashwini-orchestral Sep 29, 2021
ef43f5b
Added new unit test and refactored existing unit tests related to clo…
ashwini-orchestral Sep 29, 2021
63395cc
Refactored action clone API method to reuse existing create and delet…
ashwini-orchestral Sep 29, 2021
d344448
Minor fixes in existing unit tests
ashwini-orchestral Sep 29, 2021
0eb5dce
Minor fix in unit test related to clone response error code
ashwini-orchestral Sep 29, 2021
d5003a4
Update st2client/st2client/commands/action.py to deduplicate `self.ma…
ashwini-orchestral Oct 4, 2021
9750ebc
Merge branch 'master' into clone_action_api
ashwini-orchestral Oct 4, 2021
0ff59a1
Move local generic body class to public GenericRequestParam
m4dcoder Oct 5, 2021
d32b1d0
For options (i.e. `remove_files` flag) using GenericRequestParam clas…
mahesh-orch Oct 8, 2021
b9484c7
Refactoring unit tests regarding cleanup partially cloned actions and…
mahesh-orch Oct 8, 2021
7a55c94
Added functions related to backup destination action files in case of…
mahesh-orch Oct 8, 2021
57eb5fe
Added unit tests for newly added function for restoring destination a…
mahesh-orch Oct 8, 2021
83591f3
Deleted st2api/tests/base.py as it was temporarily added here for RBA…
mahesh-orch Oct 8, 2021
3c0f50e
Deleted test_actions_rbac.py file as these clone action RBAC unit tes…
mahesh-orch Oct 8, 2021
67adadc
Merge branch 'master' into clone_action_api
mahesh-orch Oct 8, 2021
b1815a2
Created new function `_restore_action` to reuse the common code and d…
mahesh-orch Oct 11, 2021
f795daf
Adding check for `actions` directory in pack to create it if doesn't …
mahesh-orch Oct 11, 2021
305d50e
Added unit tests for exceptions for `temp_backup_action_files` and `r…
mahesh-orch Oct 11, 2021
6a250b8
Refactored test_packs.py to fix failing CI unit test chunk 1
mahesh-orch Oct 11, 2021
d41453a
Minor fix in test_packs.py, removed extra variable name `SOURCE_WORKF…
mahesh-orch Oct 11, 2021
0ea4b9a
Refactored unit test `test_clone_overwrite_exception_destination_reco…
mahesh-orch Oct 13, 2021
6d736c1
Minor grammar fix in comment in `test_actions.py`
mahesh-orch Oct 13, 2021
76bebb8
Refactoring unit test `test_clone_overwrite` to use ACTION_16["parame…
mahesh-orch Oct 13, 2021
34326d5
Minor change to add comment line in unit test in `test_actions.py`
mahesh-orch Oct 13, 2021
ab45818
Added comment for source workflow path required for related unit tests
mahesh-orch Oct 14, 2021
7085db1
Fix related to entry_point directories creatinon in `/tmp/<uuid>` rel…
mahesh-orch Oct 14, 2021
d9d27ae
Minor fix in clone api function for removing cloned action database e…
mahesh-orch Oct 14, 2021
02bac39
Refactoring unit tests for newly updated clone api function for remov…
mahesh-orch Oct 14, 2021
44fb50e
Deleted unnecessary file __init__.py
mahesh-orch Oct 19, 2021
a009f8c
Fixing failing unit tests with `FileNotFoundError`. Creating empty fo…
mahesh-orch Oct 19, 2021
5a8f693
Fixing black/flake error - removed blank line
mahesh-orch Oct 19, 2021
d9a43b3
Fixing black/flake check - added Apache license header and copyright …
mahesh-orch Oct 19, 2021
f27900a
Simplified logic of creating entry point folder in /tmp/<uuid> directory
mahesh-orch Oct 19, 2021
64db892
Merge branch 'master' into clone_action_api
mahesh-orch Oct 19, 2021
c125cd1
Merge branch 'master' into clone_action_api
mahesh-orch Oct 26, 2021
97251ae
Merge branch 'master' into clone_action_api
mahesh-orch Oct 27, 2021
91e4c2d
Minor fix for running CIs to fix timeout error in ci/circleci: packages
mahesh-orch Oct 27, 2021
9ad36c4
Minor fix in comment for creation actions directory under packs.py
mahesh-orch Oct 27, 2021
b2cef24
Adding newly added functions in __all__ list
mahesh-orch Oct 28, 2021
91029eb
Adding newly added functions in __all__ list in misc.py
mahesh-orch Oct 28, 2021
f9ec53e
Merge branch 'master' into clone_action_api
m4dcoder Dec 1, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions st2api/st2api/controllers/v1/actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
from st2api.controllers import resource
from st2api.controllers.v1.action_views import ActionViewsController
from st2common import log as logging
from st2common.bootstrap.actionsregistrar import register_actions
from st2common.constants.triggers import ACTION_FILE_WRITTEN_TRIGGER
from st2common.exceptions.action import InvalidActionParameterException
from st2common.exceptions.apivalidation import ValueValidationException
Expand All @@ -39,10 +40,12 @@
from st2common.router import abort
from st2common.router import Response
from st2common.validators.api.misc import validate_not_part_of_system_pack
from st2common.validators.api.misc import validate_not_part_of_system_pack_by_name
from st2common.content.utils import get_pack_base_path
from st2common.content.utils import get_pack_resource_file_abs_path
from st2common.content.utils import get_relative_path_to_pack_file
from st2common.services.packs import delete_action_files_from_pack
from st2common.services.packs import clone_action
from st2common.transport.reactor import TriggerDispatcher
from st2common.util.system_info import get_host_info
import st2common.validators.api.action as action_validator
Expand Down Expand Up @@ -271,6 +274,88 @@ def delete(self, ref_or_id, requester_user):
LOG.audit("Action deleted. Action.id=%s" % (action_db.id), extra=extra)
return Response(status=http_client.NO_CONTENT)

def clone(
self,
source_pack,
source_action,
dest_pack,
dest_action,
requester_user,
overwrite=False,
):
"""
Clone an action.
Handles requests:
POST /actions/{sorce_pack}/{source_action}/{dest_pack}/{dest_action}
"""

source_ref = "%s.%s" % (source_pack, source_action)
source_action_db = self._get_by_ref(resource_ref=source_ref)
msg = "The requested source for cloning operation doesn't exists"
if not source_action_db:
abort(http_client.BAD_REQUEST, six.text_type(msg))

permission_type = PermissionType.ACTION_CREATE
rbac_utils = get_rbac_backend().get_utils_class()
rbac_utils.assert_user_has_resource_db_permission(
user_db=requester_user,
resource_db=source_action_db,
permission_type=permission_type,
)

LOG.debug(
"POST /actions/ lookup with ref=%s found object: %s",
source_ref,
source_action_db,
)

source_pack = source_action_db["pack"]
source_entry_point = source_action_db["entry_point"]
source_metadata_file = source_action_db["metadata_file"]
source_pack_base_path = get_pack_base_path(pack_name=source_pack)
dest_pack_base_path = get_pack_base_path(pack_name=dest_pack)
if not os.path.isdir(dest_pack_base_path):
raise ValueError('Destination pack "%s" doesn\'t exist' % (dest_pack))

dest_ref = "%s.%s" % (dest_pack, dest_action)
dest_action_db = self._get_by_ref(resource_ref=dest_ref)

try:
validate_not_part_of_system_pack_by_name(dest_pack)
except ValueValidationException as e:
abort(http_client.BAD_REQUEST, six.text_type(e))

if dest_action_db:
if overwrite:
pass
else:
msg = "The requested destination action already exists"
abort(http_client.BAD_REQUEST, six.text_type(msg))
try:
clone_action(
source_pack_base_path=source_pack_base_path,
source_metadata_file=source_metadata_file,
source_entry_point=source_entry_point,
dest_pack_base_path=dest_pack_base_path,
dest_pack=dest_pack,
dest_action=dest_action,
)
register_actions(pack_dir=dest_pack_base_path)
except Exception as e:
LOG.error(
"Exception encountered during cloning resource." "Exception was %s",
e,
)
abort(http_client.INTERNAL_SERVER_ERROR, six.text_type(e))
return

dest_action_db = self._get_by_ref(resource_ref=dest_ref)
extra = {"cloned_acion_db": dest_action_db}
LOG.audit("Action cloned. Action.id=%s" % (dest_action_db.id), extra=extra)
cloned_action_api = ActionAPI.from_model(dest_action_db)

return Response(json=cloned_action_api, status=http_client.CREATED)

def _handle_data_files(self, pack_ref, data_files):
"""
Method for handling action data files.
Expand Down
95 changes: 95 additions & 0 deletions st2client/st2client/commands/resource.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ def __init__(
"delete": ResourceDeleteCommand,
"enable": ResourceEnableCommand,
"disable": ResourceDisableCommand,
"clone": ResourceCloneCommand,
}
for cmd, cmd_class in cmd_map.items():
if cmd not in commands:
Expand All @@ -116,6 +117,7 @@ def __init__(
self.commands["create"] = commands["create"](*args)
self.commands["update"] = commands["update"](*args)
self.commands["delete"] = commands["delete"](*args)
self.commands["clone"] = commands["clone"](*args)

if has_disable:
self.commands["enable"] = commands["enable"](*args)
Expand Down Expand Up @@ -758,6 +760,99 @@ class ContentPackResourceDeleteCommand(ResourceDeleteCommand):
pk_argument_name = "ref_or_id"


class ResourceCloneCommand(ResourceCommand):
pk_source_pack = "source_pack"
pk_source_action = "source_action"
pk_dest_pack = "destination_pack"
pk_dest_action = "destination_action"

def __init__(self, resource, *args, **kwargs):
super(ResourceCloneCommand, self).__init__(
resource,
"clone",
"Clone a new %s." % resource.get_display_name().lower(),
*args,
**kwargs,
)

pk_list = [
self.pk_source_pack,
self.pk_source_action,
self.pk_dest_pack,
self.pk_dest_action,
]

for var in pk_list:
metavar = self._get_metavar_for_argument(argument=var)
helparg = self._get_help_for_argument(resource=resource, argument=var)
self.parser.add_argument(var, metavar=metavar, help=helparg)

self.parser.add_argument(
"-f",
"--force",
action="store_true",
dest="force",
help="Auto yes flag to overwrite action files on disk if destination exists.",
)

@add_auth_token_to_kwargs_from_cli
def run(self, args, **kwargs):
source_pack = getattr(args, self.pk_source_pack, None)
source_action = getattr(args, self.pk_source_action, None)
dest_pack = getattr(args, self.pk_dest_pack, None)
dest_action = getattr(args, self.pk_dest_action, None)
source_ref = "%s.%s" % (source_pack, source_action)
instance = self.get_resource(source_ref, **kwargs)
msg = 'Action with name "%s" has been successfully cloned in "%s" pack.' % (
dest_action,
dest_pack,
)

dest_ref = "%s.%s" % (dest_pack, dest_action)
try:
dest_instance = self.get_resource(dest_ref, **kwargs)
except ResourceNotFoundError:
dest_instance = None

if dest_instance:
user_input = ""
if not args.force:
user_input = input(
"The destination action already exists. Do you want to overwrite? (y/n): "
)
if args.force or user_input.lower() == "y" or user_input.lower() == "yes":
self.manager.clone(
instance,
source_pack,
source_action,
dest_pack,
dest_action,
overwrite=True,
**kwargs,
)
print(msg)
else:
print("Action is not cloned.")
else:
self.manager.clone(
instance,
source_pack,
source_action,
dest_pack,
dest_action,
overwrite=False,
**kwargs,
)
print(msg)

def run_and_print(self, args, **kwargs):
resource_id = getattr(args, self.pk_source_pack)
try:
self.run(args, **kwargs)
except ResourceNotFoundError:
self.print_not_found(resource_id)


def load_meta_file(file_path):
if not os.path.isfile(file_path):
raise Exception('File "%s" does not exist.' % file_path)
Expand Down
25 changes: 25 additions & 0 deletions st2client/st2client/models/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,31 @@ def delete(self, instance, **kwargs):

return True

@add_auth_token_to_kwargs_from_env
def clone(
self,
instance,
source_pack,
source_action,
dest_pack,
dest_action,
overwrite,
**kwargs,
):
url = "/%s/%s/%s/%s/%s?overwrite=%s" % (
self.resource.get_url_path_name(),
source_pack,
source_action,
dest_pack,
dest_action,
overwrite,
)
response = self.client.post(url, instance.serialize(), **kwargs)
if response.status_code != http_client.OK:
self.handle_error(response)
instance = self.resource.deserialize(parse_api_response(response))
return instance

@add_auth_token_to_kwargs_from_env
def delete_by_id(self, instance_id, **kwargs):
url = "/%s/%s" % (self.resource.get_url_path_name(), instance_id)
Expand Down
49 changes: 49 additions & 0 deletions st2common/st2common/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,55 @@ paths:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/api/v1/actions/{source_pack}/{source_action}/{dest_pack}/{dest_action}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow... This is very long and unintuitive design for the API endpoint. Can we make this /api/v1/actions/{ref_or_id}/clone and then pass the destination pack/action and other required args as json in the body?

post:
operationId: st2api.controllers.v1.actions:actions_controller.clone
description: |
Clone one action.
parameters:
- name: source_pack
in: path
description: Source pack name
type: string
required: true
- name: source_action
in: path
description: Source action name
type: string
required: true
- name: dest_pack
in: path
description: Destination pack name
type: string
required: true
- name: dest_action
in: path
description: Destination action name
type: string
required: true
- name: overwrite
in: query
description: Force clone action if destination already exists
type: boolean
default: false
x-parameters:
- name: user
in: context
x-as: requester_user
description: User performing the operation.
responses:
'201':
description: Single action being cloned
schema:
$ref: '#/definitions/Action'
examples:
application/json:
ref: 'core.local'
# and stuff
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/api/v1/actions/views/parameters/{action_id}:
get:
operationId: st2api.controllers.v1.action_views:parameters_view_controller.get_one
Expand Down
49 changes: 49 additions & 0 deletions st2common/st2common/openapi.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,55 @@ paths:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/api/v1/actions/{source_pack}/{source_action}/{dest_pack}/{dest_action}:
post:
operationId: st2api.controllers.v1.actions:actions_controller.clone
description: |
Clone one action.
parameters:
- name: source_pack
in: path
description: Source pack name
type: string
required: true
- name: source_action
in: path
description: Source action name
type: string
required: true
- name: dest_pack
in: path
description: Destination pack name
type: string
required: true
- name: dest_action
in: path
description: Destination action name
type: string
required: true
- name: overwrite
in: query
description: Force clone action if destination already exists
type: boolean
default: false
x-parameters:
- name: user
in: context
x-as: requester_user
description: User performing the operation.
responses:
'201':
description: Single action being cloned
schema:
$ref: '#/definitions/Action'
examples:
application/json:
ref: 'core.local'
# and stuff
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/api/v1/actions/views/parameters/{action_id}:
get:
operationId: st2api.controllers.v1.action_views:parameters_view_controller.get_one
Expand Down
Loading