Proxy features: REX by evgeni · Pull Request #309 · theforeman/foremanctl

evgeni · 2025-11-19T12:51:55Z

No description provided.

src/roles/foreman_proxy/defaults/main.yaml

src/roles/foreman_proxy/handlers/main.yml

src/roles/foreman_proxy/tasks/feature.yaml

Gauravtalreja1 · 2025-11-20T15:36:47Z

src/roles/foreman_proxy/tasks/feature/remote_execution_ssh.yaml

+    name: foreman-proxy-remote_execution_ssh-ssh-key
+    path: /root/foreman-proxy-ssh
+  notify:
+    - Restart Foreman Proxy
+    - Refresh Foreman Proxy
+
+- name: Create SSH Pub podman secret
+  containers.podman.podman_secret:
+    state: present
+    name: foreman-proxy-remote_execution_ssh-ssh-pub
+    path: /root/foreman-proxy-ssh.pub


These two tasks can be consolidated into a loop to create the SSH key/pub secrets, which will also ensure the handler runs only once, OR just run the handler when SSH secrets are mounted to the container

Handlers run only once, no matter how often they are notified.

src/roles/foreman_proxy/tasks/feature/remote_execution_ssh.yaml

adamruzicka · 2025-12-03T11:45:09Z

theforeman/smart_proxy_remote_execution_ssh#128 dropped ssh-async mode , would you include this patch to keep the config file in sync 0001-Adjust-sp-rex-ssh-config-file-to-async-ssh-removal.patch ?

adamlazik1 · 2026-01-05T11:52:58Z

theforeman/smart_proxy_remote_execution_ssh#126 added new settings for SSH certs, here is a patch that adds the new settings:

diff --git a/src/roles/foreman_proxy/templates/settings.d/remote_execution_ssh.yml.j2 b/src/roles/foreman_proxy/templates/settings.d/remote_execution_ssh.yml.j2
index b7d6402..76ad344 100644
--- a/src/roles/foreman_proxy/templates/settings.d/remote_execution_ssh.yml.j2
+++ b/src/roles/foreman_proxy/templates/settings.d/remote_execution_ssh.yml.j2
@@ -11,6 +11,15 @@
 # Mode of operation, one of ssh, pull, pull-mqtt
 :mode: ssh
 
+# Enables the use of SSH certificate for smart proxy authentication
+# The file should contain an SSH CA public key that the SSH public key of smart proxy is signed by
+# :ssh_user_ca_public_key_file:
+
+# Enables the use of SSH host certificates for host authentication
+# The file should contain a list of trusted SSH CA authorities that the host certs can be signed by
+# Example file content: @cert-authority * <SSH CA public key>
+# :ssh_ca_known_hosts_file:
+
 # Defines how often (in seconds) should the runner check
 # for new data leave empty to use the runner's default
 # :runner_refresh_interval: 1

evgeni · 2026-01-12T13:24:31Z

@adamlazik1 thanks, applied

adamruzicka · 2026-01-12T13:48:31Z

What remains to be done here until this can be undrafted?

src/filter_plugins/foremanctl.py

evgeni · 2026-02-16T11:39:32Z

What remains to be done here until this can be undrafted?

This PR was my PoC to figure out what we need to put into our feature metadata (see #372).

This PR also has two dependencies that can be reviewed/merged independently:

Once those are settled (and the "temp" commits dropped), we can continue here.

tests/foreman_proxy_test.py

arvind4501 · 2026-02-16T14:35:38Z

So, while adding feature remote_execution it adds dynflow as dependency but when i remove the same feature it disabled remote_execution config enabled: false in remote_execution_ssh.yml but dynflow.yml still has :enabled: true . are we planning for disabling the dependencies once main feature is removed?

evgeni · 2026-03-06T10:19:31Z

There is still one "temp" commit here:

Subject: [temp] flush handlers after registration

this is suboptimal, as it means we do an additional restart, but it
ensures the refresh works

But otherwise I think this can be finally undrafted.

evgeni · 2026-03-06T14:16:55Z

Okay, and cleaned up the last temp commit.

LET'S GO!

src/features.yaml

.github/workflows/test.yml

ehelms · 2026-03-06T15:59:48Z

From the user's perspective this feature is remote_execution -- how does the user specify the type of REX they want? (i.e. SSH, Ansible, pull-mode)

src/roles/foreman_proxy/defaults/main.yaml

evgeni · 2026-03-06T16:10:43Z

Ansible is a standalone feature, given it's an own gem and comes with a lot more data.

For ssh vs mqtt I'd do a dedicated option, like today?

At some point Ewoud (I think it was Ewoud) suggested "rex/ssh" and "rex/mqtt" (so "feat/subfeat"), but I don't really like it

ehelms · 2026-03-06T20:23:55Z

Do we expect users to see it that way? I was hoping we could help connect our deployment tooling and options with how the user is presented these features through the application itself and our documentation. I won't block progress on this with that question, but I think we should answer it eventually.

evgeni · 2026-03-09T07:06:49Z

Do we expect users to see it that way? I was hoping we could help connect our deployment tooling and options with how the user is presented these features through the application itself and our documentation. I won't block progress on this with that question, but I think we should answer it eventually.

I'm the wrong one to tell the user view. But it gets even more complicated than this.
On the one hand, you have the transport: SSH or MQTT
On the other, the execution engine: Script or Ansible
To make it even more confusing: Ansible doesn't work over MQTT.

Otherwise the "refresh" handler runs while the Proxy is not yet registered, which doesn't work. To avoid double-restarts by the "restart" handler, only restart proxy if it's not freshly started

ShimShtein · 2026-03-10T09:21:59Z

docs/feature-metadata.md

 Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).

-The metadata is a Hash with the feature name as the key and the feature definition as the value.
+The metadata is a Hash with the feature (using dashes, not underscores if needed) name as the key and the feature definition as the value.


A general thought: maybe we should define the schema using a standard tool like openAPI or JSONSchema.

Yepp, that's a good idea long term

ShimShtein · 2026-03-10T09:37:31Z

src/filter_plugins/foremanctl.py



+def foreman_proxy_plugins(value):
+    dependencies = [FEATURE_MAP.get(feature, {}).get('dependencies', []) for feature in filter_content(value) if feature not in BASE_FEATURES]


Is this function recursive? As in does it pull dependencies of dependencies?

it's not, good point.

ShimShtein · 2026-03-10T10:28:55Z

src/roles/foreman_proxy/tasks/feature.yaml

+    name: foreman-proxy-{{ feature_name }}-yml
+    data: "{{ lookup('ansible.builtin.template', 'settings.d/' + feature_name + '.yml.j2') }}"
+  notify:
+    - Restart Foreman Proxy


While the notification will be executed only once, but just creating the secret should not trigger restart. Only the mounting of a new secret should cause a restart.
Unless this also overwrites an existing secret, even if it is already mounted.

It overwrites

ShimShtein · 2026-03-10T10:36:38Z

src/roles/foreman_proxy/tasks/feature.yaml

+  containers.podman.podman_secret:
+    state: present
+    name: foreman-proxy-{{ feature_name }}-yml
+    data: "{{ lookup('ansible.builtin.template', 'settings.d/' + feature_name + '.yml.j2') }}"


This means the settings.d template file should always exist. What about features that do not have a proxy plugin, like IoP?
We still want the feature.yaml file to execute the Include additional tasks part, but we don't want it to fail on missing settings.d template.

No, in that case we have no proxy plugin, just a standalone role

ShimShtein · 2026-03-10T10:39:40Z

src/roles/foreman_proxy/defaults/main.yaml

+foreman_proxy_base_features:
+  - logs
+foreman_proxy_plugins: []
+foreman_proxy_features: "{{ foreman_proxy_base_features + foreman_proxy_plugins }}"


If foreman_proxy_plugins contains only the list of ruby smart-proxy implementation of plugins, we will miss features that do not have a plugin associated with them, like IoP.

So? This role is only responsible for smart proxy plugins.

ShimShtein · 2026-03-10T10:44:43Z

src/roles/foreman_proxy/tasks/main.yaml

    validate_certs: false
+
+- name: Flush handlers to restart services
+  ansible.builtin.meta: flush_handlers


Why do you need an explicit run of handlers at this point? The handlers should be executed after all the roles have been executed: https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_handlers.html#controlling-when-handlers-run

Because that's too late

ShimShtein · 2026-03-10T10:45:46Z

src/roles/foreman_proxy/templates/settings.d/logs.yml.j2

@@ -1,2 +1,2 @@
 ---
-:enabled: https
+:enabled: {{ feature_enabled }}


It will be true/false, shouldn't it be https/false?

True is an alias for "all enabled protocols"

evgeni commented Nov 19, 2025

View reviewed changes

src/roles/foreman_proxy/defaults/main.yaml Outdated Show resolved Hide resolved

evgeni force-pushed the proxy-features branch from 7100b9a to c81b6c2 Compare November 19, 2025 13:05

evgeni commented Nov 19, 2025

View reviewed changes

src/roles/foreman_proxy/handlers/main.yml Show resolved Hide resolved

evgeni commented Nov 19, 2025

View reviewed changes

src/roles/foreman_proxy/tasks/feature.yaml Outdated Show resolved Hide resolved

evgeni force-pushed the proxy-features branch from 4b3451c to 64f8540 Compare November 20, 2025 08:06

Gauravtalreja1 reviewed Nov 20, 2025

View reviewed changes

src/roles/foreman_proxy/tasks/feature/remote_execution_ssh.yaml Outdated Show resolved Hide resolved

evgeni force-pushed the proxy-features branch from 922ca79 to d103cc6 Compare November 21, 2025 11:24

evgeni force-pushed the proxy-features branch 4 times, most recently from 6d74c47 to 3ee17bd Compare December 3, 2025 10:37

evgeni force-pushed the proxy-features branch from 3ee17bd to 194332a Compare December 3, 2025 12:01

evgeni force-pushed the proxy-features branch from 194332a to caf8416 Compare January 12, 2026 13:23

evgeni force-pushed the proxy-features branch from eaf7e75 to 53daace Compare January 12, 2026 15:42

pablomh mentioned this pull request Jan 13, 2026

add smart_proxy_refresh module theforeman/foreman-ansible-modules#1930

Merged

evgeni commented Feb 12, 2026

View reviewed changes

src/filter_plugins/foremanctl.py Outdated Show resolved Hide resolved

evgeni force-pushed the proxy-features branch 3 times, most recently from c6bc7ff to f309355 Compare February 12, 2026 15:46

evgeni mentioned this pull request Feb 16, 2026

Deploy hammer and smart proxy for development #373

Merged

evgeni force-pushed the proxy-features branch from f309355 to 4e4c695 Compare February 16, 2026 11:14

evgeni mentioned this pull request Feb 16, 2026

add feature metadata documentation #372

Merged

arvind4501 reviewed Feb 16, 2026

View reviewed changes

tests/foreman_proxy_test.py Show resolved Hide resolved

evgeni force-pushed the proxy-features branch 2 times, most recently from 04ac733 to f388e94 Compare March 6, 2026 10:13

evgeni marked this pull request as ready for review March 6, 2026 10:18

evgeni force-pushed the proxy-features branch 5 times, most recently from ae4c055 to 7533b1d Compare March 6, 2026 14:16

evgeni force-pushed the proxy-features branch from 7533b1d to eaffe96 Compare March 6, 2026 14:22

evgeni commented Mar 6, 2026

View reviewed changes

src/features.yaml Show resolved Hide resolved

ehelms reviewed Mar 6, 2026

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

ehelms reviewed Mar 6, 2026

View reviewed changes

src/roles/foreman_proxy/defaults/main.yaml Outdated Show resolved Hide resolved

evgeni force-pushed the proxy-features branch from eaffe96 to 2a4f6e8 Compare March 9, 2026 07:12

use dashes in feature names

968b39b

evgeni force-pushed the proxy-features branch from 2a4f6e8 to 24c2e82 Compare March 9, 2026 13:08

evgeni added 5 commits March 9, 2026 14:13

map proxy features

ef5db65

dynamically configure proxy features based on foremanctl features

1ba3e45

refresh proxy features

5093c0c

Flush handlers after registration

140f0f5

Otherwise the "refresh" handler runs while the Proxy is not yet registered, which doesn't work. To avoid double-restarts by the "restart" handler, only restart proxy if it's not freshly started

add rex feature

756510e

evgeni force-pushed the proxy-features branch from 24c2e82 to 756510e Compare March 9, 2026 13:14

ShimShtein reviewed Mar 10, 2026

View reviewed changes

evgeni added 2 commits March 10, 2026 14:05

consistently filter features

a66e8d7

pull all deps, not only the first level

3f7f64b



		def foreman_proxy_plugins(value):
		dependencies = [FEATURE_MAP.get(feature, {}).get('dependencies', []) for feature in filter_content(value) if feature not in BASE_FEATURES]

Conversation

evgeni commented Nov 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Gauravtalreja1 Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamruzicka commented Dec 3, 2025

Uh oh!

adamlazik1 commented Jan 5, 2026

Uh oh!

evgeni commented Jan 12, 2026

Uh oh!

adamruzicka commented Jan 12, 2026

Uh oh!

Uh oh!

evgeni commented Feb 16, 2026

Uh oh!

Uh oh!

arvind4501 commented Feb 16, 2026

Uh oh!

evgeni commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

evgeni commented Mar 6, 2026

Uh oh!

Uh oh!

Uh oh!

ehelms commented Mar 6, 2026

Uh oh!

Uh oh!

evgeni commented Mar 6, 2026

Uh oh!

ehelms commented Mar 6, 2026

Uh oh!

evgeni commented Mar 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Gauravtalreja1 Nov 20, 2025 •

edited

Loading

evgeni commented Mar 6, 2026 •

edited

Loading