New otc-metadata-rework #104
Conversation
YustinaKvr
commented
Jul 11, 2025
YustinaKvr
commented
- Switch to new otc-metadata-rework structure
YustinaKvr
requested review from
LukasCuperDT,
RusselSand,
SebastianGode,
anton-sidelnikov,
sgmv,
tischrei and
vladimirhasko
scripts/eod_1_otc_services_dict.py
Outdated
|
|
||
| main(BASE_DIR_REGULAR, BASE_RTC_TABLE, BASE_DOC_TABLE, STYRING_URL_REGULAR) | ||
| main(BASE_DIR_SWISS, f"{BASE_RTC_TABLE}_swiss", f"{BASE_DOC_TABLE}_swiss", STYRING_URL_SWISS) | ||
| main(BASE_DIR_UNIFIED, BASE_RTC_TABLE, BASE_DOC_TABLE, STYRING_URL_REGULAR, target_cloud="eu_de") |
There was a problem hiding this comment.
instead of hardcoding can you dynamically query it from otc-metadata to avoid that in future you need to change it with every new env?
There was a problem hiding this comment.
I don't see need to use var here, because in case of changes, we anyway need to change it, here or in var. But using vars could potentially complicate and confuse either changes or debugging, because implies and rely on using one more tool here. So since this is not a sensitive content and not not changes very often, I'd prefer use explicit path here
There was a problem hiding this comment.
no, if you change target_cloud=TARGET_CLOUD, you can loop TARGET_CLOUD from fetched values from otc-metadata-rework fetch, in such way you always dynamically refer to values which are defined in otc-metadata
scripts/eod_1_otc_services_dict.py
Outdated
|
|
||
| from config import Database, EnvVariables, Timer, setup_logging | ||
|
|
||
| BASE_URL = "https://gitea.eco.tsi-dev.otc-service.com/api/v1" |
There was a problem hiding this comment.
use this as ENV variable and it should be provided by the configuration file in config map.
Don't hardcode urls here
scripts/eod_1_otc_services_dict.py
Outdated
|
|
||
| main(BASE_DIR_REGULAR, BASE_RTC_TABLE, BASE_DOC_TABLE, STYRING_URL_REGULAR) | ||
| main(BASE_DIR_SWISS, f"{BASE_RTC_TABLE}_swiss", f"{BASE_DOC_TABLE}_swiss", STYRING_URL_SWISS) | ||
| main(BASE_DIR_UNIFIED, BASE_RTC_TABLE, BASE_DOC_TABLE, STYRING_URL_REGULAR, target_cloud="eu_de") |
There was a problem hiding this comment.
no, if you change target_cloud=TARGET_CLOUD, you can loop TARGET_CLOUD from fetched values from otc-metadata-rework fetch, in such way you always dynamically refer to values which are defined in otc-metadata
config/classes.py
Outdated
| import psycopg2 | ||
| import requests | ||
|
|
||
| GITEA_API_ENDPOINT = "https://gitea.eco.tsi-dev.otc-service.com/api/v1" |
There was a problem hiding this comment.
should be read from env variable, not hardcoded here
scripts/eod_1_otc_services_dict.py
Outdated
| max_pages = 50 | ||
| page = 1 | ||
|
|
||
| if rtc_table == "repo_title_category": |
There was a problem hiding this comment.
this condition doesn't seem to me logical, we should discuss
scripts/eod_1_otc_services_dict.py
Outdated
| "Other", "service_type": "cdn", "squad": "Other", "environment": "hidden"}, | ||
| {"service_uri": "data-admin-service", "service_title": "Data Admin Service", "service_category": "Other", | ||
| "service_type": "das", "squad": "Other", "environment": "hidden"} | ||
| "Other", "service_type": "cdn", "squad": "Other", "target_visibility": "hidden"} |
There was a problem hiding this comment.
repositories which are archived, should be skipped automatically
scripts/eod_1_otc_services_dict.py
Outdated
| else: | ||
| rtctable = f"{base_rtctable}_{env_name}" if env_name != "eu_de" else base_rtctable | ||
| doctable = f"{base_doctable}_{env_name}" if env_name != "eu_de" else base_doctable | ||
| styring_url = (f"{BASE_GITEA_URL}/repos/infra/gitstyring/contents/data/github/orgs/opentelekomcloud-docs/da" |
There was a problem hiding this comment.
again, headers should be used?
scripts/eod_1_otc_services_dict.py
Outdated
| logging.info(f"Found environments: {list(env_data.keys())}") | ||
|
|
||
| for env_name, services_list in env_data.items(): | ||
| if env_name == "swiss": |
There was a problem hiding this comment.
let's discuss this hardcoding
