software_center_download: Add option to search for latest packages #28

marcelmamula · 2025-03-21T15:49:00Z

Description

Managing SAP Software lists is difficult because of frequent changes file changes, new patches and file retirements.
#24 was raised to add new parameter that will drive search logic to find new packages, if requested package is not available anymore.

Changes

Deprecate softwarecenter_search_query and introduce new clearer parameter search_query. Deprecated variable is still compatible.
Deprecate dest and introduce clearer parameter download_path. dest is not backwards compatible, as download_path is required parameter. Change reverted by later PR #34, prior to release
New parameter search_alternatives as boolean that will drive logic for searching using search_query
Fuzzy search logic enhanced to:
- Cover known SAP product names and their structure to allow for better keyword filtering.
- Allow download of latest package for same SPS if it exists, or for latest SPS (for files that contain SPS).
Add function to fix invalid download filenames for SAP CD Media, where S4CORE105_INST_EXPORT_1.zip can be downloaded as 19118000000000004323.
Enhance Error messages for better understanding what happened.
Enhanced dry_run code with better output messages.

Logic for search_alternatives:

Search for file using search_query and filter using Title or Description (unchanged)
Trigger search_alternatives block instead of failing which will:
- Execute fuzzy search using ID (e.g. 10012328 inside of IMDB_AFL100_010P_41-10012328.SAR which is unique combination of Product and Platform, but it can span different versions and revisions)
  NOTE SAP CD Media does not contain ID, so we cannot run fuzzy search for latest as it would change actual Product, therefore failure is expected.
Attempt to filter fuzzy search results:
- First using package name tied to SPS (where applicable): e.g. IMDB_SERVER20_06 for IMDB_SERVER20_067_4-80002046.SAR which is SPS06
- Second using loose search that can find latest SPS.
Sort resulting list (descending order) using new function for processing integer to resolve issue with string sorting, where 700 is higher than 3500.
Return [0] from sorted list back for downloading.

Test Example

New module was tested across wide range of packages, with most of them were adjusted to fail to trigger new functionality.

    - name: Execute Ansible Module with venv Python to download SAP installation media
      community.sap_launchpad.software_center_download:
        suser_id: "{{ sap_id_user | d('user') }}"
        suser_password: "{{ sap_id_user_password | d('password') }}"
        dest: "/software"
        search_query: "{{ item }}"
        search_alternatives: true
      loop:
        - 'SAPCAR_*-70006178.EXE'
        - 'SAPCAR_1115-70006178.EXE'
        - 'SAPCAR_1115-7000617'
        - 'SAPCAR_999-70006178.EXE'  # Invalid

        - 'SYBCTRL_1436-80002616.SAR'  # Invalid
        - 'SYBCTRL_1440-80002616.SAR'

        - 'SUM20SP21_0-80002456.SAR'  # Invalid
        - 'SUM11SP01_1-80006800.SAR'  # Invalid

        - 'SAPHOSTAGENT62_62-80004822.SAR'
        - 'SAPHOSTAGENT99-80004822.SAR'
        - 'SAPHOSTAGENT25_11-80004831.SAR'
        - 'SAPHOSTAGENT61_11-80004831.SAR'
        - 'SAPHOSTAGENT61_61-80004831.SAR'

        - 'SWPM20SP19_1-80003426.SAR' # Invalid
        - '70SWPM10SP20_1-20009701.sar' # Invalid
        - 'SWPM10SP20_2-20009701.SAR' # Invalid

        - 'IMDB_SERVER20_020_2-80002046.SAR' # Invalid
        - 'IMDB_SERVER20_071_1-80002046.SAR' # Invalid
        - 'IMDB_SERVER20_099_1-80002046.SAR' # Invalid

        - 'IMDB_LCAPPS_122P_100-20010426.SAR' # Invalid
        - 'IMDB_LCAPPS_110P_100-20010426.SAR' # Invalid
        - 'IMDB_LCAPPS_2061_400-80002183.SAR' # Invalid

        - 'IMDB_AFL100_010P_41-10012328.SAR' # Invalid
        - 'IMDB_AFL100_102P_01-10012328.SAR' # Invalid
        - 'IMDB_AFL20_067P_400-80002045.SAR' # Invalid

        - 'IMDB_CLIENT20_021_15-80002095.SAR' # Invalid
        - 'IMDB_CLIENT10_011_31-80002095.SAR' # Invalid
        - 'IMDB_CLIENT20_020_10-80002095.SAR' # Invalid

        - 'SAPEXE_100-80005509.SAR'
        - 'SAPEXE_99-80005509.SAR'  # Invalid 
        - 'SAPEXEDB_100-80005508.SAR'

        - 'S4CORE105_INST_EXPORT_1.zip'
        - 'S4CORE105_INST_EXPORT_10.zip'
        - 'S4CORE155_INST_EXPORT_1.zip'  # Invalid

        - 'igsexe_4-70005446.sar' # IGS 7.81
        - 'igsexe_99-70005446.sar'  # Invalid
        - 'igshelper_17-10010245.sar'
        - 'igshelper_99-10010245.sar'  # Invalid

        - 'S4HANAOP105_ERP_LANG_EN.SAR'
        - 'S4HANAOP19_ERP_LANG_EN.SAR'  # Invalid
      loop_control:
        label: "{{ item }} : {{ download_task.msg | d('') }}"
      register: download_task
      retries: 0
      until: download_task is not failed

Test results

Test results show handling for packages above.

TASK [Execute Ansible Module with venv Python to download SAP installation media] ****************************************************************************************************************************************************************
changed: [nwas-pas] => (item=SAPCAR_*-70006178.EXE : Successfully downloaded alternative SAP software: SAPCAR_1115-70006178.EXE - original file SAPCAR_*-70006178.EXE is not available to download)
skipping: [nwas-pas] => (item=SAPCAR_1115-70006178.EXE : File already exists: SAPCAR_1115-70006178.EXE)
changed: [nwas-pas] => (item=SAPCAR_1115-7000617 : Successfully downloaded alternative SAP software: SAPCAR_1115-70006173.EXE - original file SAPCAR_1115-7000617 is not available to download)
skipping: [nwas-pas] => (item=SAPCAR_999-70006178.EXE : Alternative file already exists: SAPCAR_1115-70006178.EXE - original file SAPCAR_999-70006178.EXE is not available to download)
changed: [nwas-pas] => (item=SYBCTRL_1436-80002616.SAR : Successfully downloaded alternative SAP software: SYBCTRL_1440-80002616.SAR - original file SYBCTRL_1436-80002616.SAR is not available to download)
skipping: [nwas-pas] => (item=SYBCTRL_1440-80002616.SAR : File already exists: SYBCTRL_1440-80002616.SAR)
changed: [nwas-pas] => (item=SUM20SP21_0-80002456.SAR : Successfully downloaded alternative SAP software: SUM20SP22_3-80002456.SAR - original file SUM20SP21_0-80002456.SAR is not available to download)
changed: [nwas-pas] => (item=SUM11SP01_1-80006800.SAR : Successfully downloaded alternative SAP software: SUM11SP04_2-80006800.SAR - original file SUM11SP01_1-80006800.SAR is not available to download)
changed: [nwas-pas] => (item=SAPHOSTAGENT62_62-80004822.SAR : Successfully downloaded SAP software: SAPHOSTAGENT62_62-80004822.SAR)
changed: [nwas-pas] => (item=SAPHOSTAGENT99-80004822.SAR : Successfully downloaded alternative SAP software: SAPHOSTAGENT66_66-80004822.SAR - original file SAPHOSTAGENT99-80004822.SAR is not available to download)
changed: [nwas-pas] => (item=SAPHOSTAGENT25_11-80004831.SAR : Successfully downloaded alternative SAP software: SAPHOSTAGENT66_66-80004831.SAR - original file SAPHOSTAGENT25_11-80004831.SAR is not available to download)
changed: [nwas-pas] => (item=SAPHOSTAGENT61_11-80004831.SAR : Successfully downloaded alternative SAP software: SAPHOSTAGENT61_61-80004831.SAR - original file SAPHOSTAGENT61_11-80004831.SAR is not available to download)
skipping: [nwas-pas] => (item=SAPHOSTAGENT61_61-80004831.SAR : File already exists: SAPHOSTAGENT61_61-80004831.SAR)
changed: [nwas-pas] => (item=SWPM20SP19_1-80003426.SAR : Successfully downloaded alternative SAP software: SWPM20SP20_1-80003426.SAR - original file SWPM20SP19_1-80003426.SAR is not available to download)
changed: [nwas-pas] => (item=70SWPM10SP20_1-20009701.sar : Successfully downloaded alternative SAP software: 70SWPM10SP43_2-20009701.sar - original file 70SWPM10SP20_1-20009701.sar is not available to download)
changed: [nwas-pas] => (item=SWPM10SP20_2-20009701.SAR : Successfully downloaded alternative SAP software: SWPM10SP43_2-20009701.SAR - original file SWPM10SP20_2-20009701.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_SERVER20_020_2-80002046.SAR : Successfully downloaded alternative SAP software: IMDB_SERVER20_024_10-80002046.SAR - original file IMDB_SERVER20_020_2-80002046.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_SERVER20_071_1-80002046.SAR : Successfully downloaded alternative SAP software: IMDB_SERVER20_079_3-80002046.SAR - original file IMDB_SERVER20_071_1-80002046.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_SERVER20_099_1-80002046.SAR : Successfully downloaded alternative SAP software: IMDB_SERVER20_059_17-80002046.SAR - original file IMDB_SERVER20_099_1-80002046.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_LCAPPS_122P_100-20010426.SAR : Successfully downloaded alternative SAP software: IMDB_LCAPPS_122P_3500-20010426.SAR - original file IMDB_LCAPPS_122P_100-20010426.SAR is not available to download)
skipping: [nwas-pas] => (item=IMDB_LCAPPS_110P_100-20010426.SAR : Alternative file already exists: IMDB_LCAPPS_122P_3500-20010426.SAR - original file IMDB_LCAPPS_110P_100-20010426.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_LCAPPS_2061_400-80002183.SAR : Successfully downloaded alternative SAP software: IMDB_LCAPPS_2067P_400-80002183.SAR - original file IMDB_LCAPPS_2061_400-80002183.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_AFL100_010P_41-10012328.SAR : Successfully downloaded alternative SAP software: IMDB_AFL100_122P_3500-10012328.SAR - original file IMDB_AFL100_010P_41-10012328.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_AFL100_102P_01-10012328.SAR : Successfully downloaded alternative SAP software: IMDB_AFL100_102P_61-10012328.SAR - original file IMDB_AFL100_102P_01-10012328.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_AFL20_067P_400-80002045.SAR : Successfully downloaded SAP software: IMDB_AFL20_067P_400-80002045.SAR)
changed: [nwas-pas] => (item=IMDB_CLIENT20_021_15-80002095.SAR : Successfully downloaded alternative SAP software: IMDB_CLIENT20_021_31-80002095.SAR - original file IMDB_CLIENT20_021_15-80002095.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_CLIENT10_011_31-80002095.SAR : Successfully downloaded alternative SAP software: IMDB_CLIENT20_004_202-80002095.SAR - original file IMDB_CLIENT10_011_31-80002095.SAR is not available to download)
changed: [nwas-pas] => (item=IMDB_CLIENT20_020_10-80002095.SAR : Successfully downloaded alternative SAP software: IMDB_CLIENT20_020_23-80002095.SAR - original file IMDB_CLIENT20_020_10-80002095.SAR is not available to download)
changed: [nwas-pas] => (item=SAPEXE_100-80005509.SAR : Successfully downloaded SAP software: SAPEXE_100-80005509.SAR)
changed: [nwas-pas] => (item=SAPEXE_99-80005509.SAR : Successfully downloaded alternative SAP software: SAPEXE_400-80005509.SAR - original file SAPEXE_99-80005509.SAR is not available to download)
changed: [nwas-pas] => (item=SAPEXEDB_100-80005508.SAR : Successfully downloaded SAP software: SAPEXEDB_100-80005508.SAR)
changed: [nwas-pas] => (item=S4CORE105_INST_EXPORT_1.zip : Successfully downloaded SAP software: S4CORE105_INST_EXPORT_1.zip)
changed: [nwas-pas] => (item=S4CORE105_INST_EXPORT_10.zip : Successfully downloaded SAP software: S4CORE105_INST_EXPORT_10.zip)
failed: [nwas-pas] (item=S4CORE155_INST_EXPORT_1.zip : An exception has occurred - File S4CORE155_INST_EXPORT_1.zip is not available to download and has no alternatives) =>
    alternative: false
    ansible_loop_var: item
    changed: false
    filename: S4CORE155_INST_EXPORT_1.zip
    item: S4CORE155_INST_EXPORT_1.zip
    msg: An exception has occurred - File S4CORE155_INST_EXPORT_1.zip is not available
        to download and has no alternatives
changed: [nwas-pas] => (item=igsexe_4-70005446.sar : Successfully downloaded SAP software: igsexe_4-70005446.sar)
changed: [nwas-pas] => (item=igsexe_99-70005446.sar : Successfully downloaded alternative SAP software: igsexe_5-70005446.sar - original file igsexe_99-70005446.sar is not available to download)
changed: [nwas-pas] => (item=igshelper_17-10010245.sar : Successfully downloaded SAP software: igshelper_17-10010245.sar)
skipping: [nwas-pas] => (item=igshelper_99-10010245.sar : Alternative file already exists: igshelper_17-10010245.sar - original file igshelper_99-10010245.sar is not available to download)
changed: [nwas-pas] => (item=S4HANAOP105_ERP_LANG_EN.SAR : Successfully downloaded SAP software: S4HANAOP105_ERP_LANG_EN.SAR)
failed: [nwas-pas] (item=S4HANAOP19_ERP_LANG_EN.SAR : An exception has occurred - File S4HANAOP19_ERP_LANG_EN.SAR is not available to download and has no alternatives) =>
    alternative: false
    ansible_loop_var: item
    changed: false
    filename: S4HANAOP19_ERP_LANG_EN.SAR
    item: S4HANAOP19_ERP_LANG_EN.SAR
    msg: An exception has occurred - File S4HANAOP19_ERP_LANG_EN.SAR is not available
        to download and has no alternatives

sean-freeman · 2025-03-26T16:07:41Z

LGTM - I receive identical results from the test data above 👍

feat: query_latest module feature

6590b7f

marcelmamula added the enhancement New feature or request label Mar 21, 2025

marcelmamula requested review from rhmk and sean-freeman March 21, 2025 15:49

marcelmamula self-assigned this Mar 21, 2025

marcelmamula mentioned this pull request Mar 24, 2025

feat: Add detection using transaction ID, not just name #25

Closed

marcelmamula added 4 commits March 25, 2025 10:28

feat: enhanced dry_run, updated naming of variables and parameters

b9392cd

feat: enhance search logic for existing files

71b2bd8

new return parameters for role consumption and improved result output

181dcb3

rename dest argument to download_path

ff4c332

sean-freeman approved these changes Mar 26, 2025

View reviewed changes

marcelmamula merged commit 031ca2c into sap-linuxlab:dev Mar 26, 2025
2 checks passed

This was referenced Mar 27, 2025

feat: Add search for latest SAP media #24

Closed

maintenance_planner modules: Add option to use Display ID instead of name #31

Merged

marcelmamula mentioned this pull request Apr 4, 2025

merge dev to main for release 1.2.0 #37

Merged

marcelmamula deleted the latest branch April 7, 2025 07:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

software_center_download: Add option to search for latest packages #28

software_center_download: Add option to search for latest packages #28

Uh oh!

marcelmamula commented Mar 21, 2025 •

edited by sean-freeman

Loading

Uh oh!

sean-freeman commented Mar 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

software_center_download: Add option to search for latest packages #28

software_center_download: Add option to search for latest packages #28

Uh oh!

Conversation

marcelmamula commented Mar 21, 2025 • edited by sean-freeman Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Logic for search_alternatives:

Test Example

Test results

Uh oh!

sean-freeman commented Mar 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

marcelmamula commented Mar 21, 2025 •

edited by sean-freeman

Loading