[py][bidi]: add enable_webextensions option for chromium-based browsers

[navin772]

User description

🔗 Related Issues

More info - #15749 (comment)

💥 What does this PR do?

Chrome requires 2 flags for BiDi webextensions to work:

1. --remote-debugging-pipe
2. --enable-unsafe-extension-debugging

This PR adds a chrome option to enable these flags easily - enable_webextensions
User can either use the above chrome option or add the flags manually.

The webExtension.install method will return an exception if the user forgets to add these flags.

Usage:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.enable_webextensions = True
driver = webdriver.Chrome(options=options)

OR directly pass the flags

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--remote-debugging-pipe")
options.add_argument("--enable-unsafe-extension-debugging")

driver = webdriver.Chrome(options=options)

🔧 Implementation Notes

💡 Additional Considerations

🔄 Types of changes

• New feature (non-breaking change which adds functionality and tests!)

---

PR Type

Enhancement, Tests

---

Description

• Adds enable_webextensions option to Chrome for BiDi webextensions

  • Automatically manages required Chrome flags for webextension support
  • Allows enabling/disabling via property or manual flags

• Improves error handling in webExtension.install for missing flags

  • Raises descriptive exception if required flags are missing

• Updates and documents BiDi webextension test for Chrome

  • Explains usage of new option and required flags

• Updates test configuration to enable webextensions for Chrome BiDi

---

Changes walkthrough 📝

	Relevant files
Enhancement	options.py Add `enable_webextensions` property to Chrome options        py/selenium/webdriver/chrome/options.py Adds enable_webextensions property with getter/setter Automatically adds/removes required Chrome flags for webextension support Updates docstrings and internal logic for flag management +37/-0
Bug fix	webextension.py Improve error handling for Chrome webextension install      py/selenium/webdriver/common/bidi/webextension.py Adds error handling for missing Chrome webextension flags Raises descriptive exception if required flags are not set +12/-2
Tests	bidi_webextension_tests.py Update and document BiDi webextension test for Chrome        py/test/selenium/webdriver/common/bidi_webextension_tests.py Updates test to document Chrome webextension enabling Adds logic to handle Chrome-specific extension id behavior Provides usage examples for new option and flags +26/-3
Configuration changes	conftest.py Enable Chrome webextensions in BiDi test configuration      py/conftest.py Enables webextensions for Chrome when BiDi is enabled in test config Checks for presence of new option before setting +4/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-code-review]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ❌ 1234 - Not compliant Non-compliant requirements: • Fix issue where JavaScript in link's href is not triggered on click() in Firefox 42.0 Requires further human verification: • Need to verify if the PR changes affect Firefox click() behavior for JavaScript in href attributes 5678 - Not compliant Non-compliant requirements: • Fix "Error: ConnectFailure (Connection refused)" when instantiating ChromeDriver multiple times Requires further human verification: • Need to verify if the new Chrome BiDi webextension implementation affects ChromeDriver connectivity issues

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Flag Management The implementation removes flags when disabling webextensions, but doesn't check if they were manually added by the user. This could lead to unexpected behavior if a user manually added these flags and then toggled the property. else: # Remove webextension flags if disabling flags_to_remove = ["--enable-unsafe-extension-debugging", "--remote-debugging-pipe"] for flag in flags_to_remove: if flag in self._arguments: self._arguments.remove(flag) Exception Handling The error handling only checks for "Method not available" in the exception message, which might be fragile if error messages change in the future. Consider a more robust way to detect missing Chrome flags. if "Method not available" in str(e): raise WebDriverException( f"{str(e)}. If you are using Chrome, add '--enable-unsafe-extension-debugging' " "and '--remote-debugging-pipe' arguments or set options.enable_webextensions = True" ) from e

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Learned best practice	Add parameter validation check Add validation at the beginning of the method to check if at least one of the parameters is provided. This prevents potential errors when all parameters are None, which would lead to an undefined extension_data variable. py/selenium/webdriver/common/bidi/webextension.py [50-58] def install(self, path=None, archive_path=None, base64_value=None) -> Dict: """Installs a web extension from the remote end. :Args: - path: Path to the extension directory - archive_path: Path to the extension archive - base64_value: Base64 encoded extension archive :Returns: A dictionary containing the extension ID and other details """ + if path is None and archive_path is None and base64_value is None: + raise ValueError("At least one of path, archive_path, or base64_value must be provided") + if path is not None: extension_data = {"type": "path", "path": path} elif archive_path is not None: extension_data = {"type": "archivePath", "path": archive_path} elif base64_value is not None: extension_data = {"type": "base64", "value": base64_value} params = {"extensionData": extension_data} [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: Relevant best practice - Add null checks for parameters and properties before using them to prevent exceptions. Validate input parameters at the beginning of methods.	Low
General	Preserve exception traceback The exception handling creates a new WebDriverException that loses the original exception's traceback. Use raise ... from e syntax for all exception paths to preserve the original traceback information, which helps with debugging. py/selenium/webdriver/common/bidi/webextension.py [59-68] try: result = self.conn.execute(command_builder("webExtension.install", params)) return result except WebDriverException as e: if "Method not available" in str(e): raise WebDriverException( f"{str(e)}. If you are using Chrome, add '--enable-unsafe-extension-debugging' " "and '--remote-debugging-pipe' arguments or set options.enable_webextensions = True" ) from e - raise + raise e from e Apply / Chat Suggestion importance[1-10]: 3 __ Why: The suggestion to use raise e from e in the generic exception path is unnecessary, as a bare raise already preserves the traceback. The current code already uses raise ... from e for the custom message, so this change offers only marginal clarity and does not impact functionality or correctness.	Low
Update

[navin772]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

[navin772]

@oliverdunk can you have a look at this PR?

[cgoldberg]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

I guess the issue is that you don't want to add enable_webextensions to the driver fixture because it will affect most tests?

You could create the driver in the test and not use the fixture. We do that in a few tests. You could add a new fixture similar to the firefox_options fixture in conftest.py. Then use this fixture in your test, and set enable_webextensions from inside the test.

It's not a great solution, but I think it will help.

(let me know if that's not a good explanation)

[navin772]

@cgoldberg I have added a new pytest marker webextension and a check in conftest.py to set enable_webextensions to True only when this pytest marker is used (thus other tests won't be affected).

[shbenzer]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

I guess the issue is that you don't want to add enable_webextensions to the driver fixture because it will affect most tests?

You could create the driver in the test and not use the fixture. We do that in a few tests. You could add a new fixture similar to the firefox_options fixture in conftest.py. Then use this fixture in your test, and set enable_webextensions from inside the test.

It's not a great solution, but I think it will help.

(let me know if that's not a good explanation)

I agree with this - I don't think the new marker or conftest changes are necessary. We can create a driver specifically for the webextension tests, and then put a xfail_firefox marker (e.g. with reason "not implemented") on those tests for now. You can wrap the webextension tests in a test class to make this easier.

[navin772]

We can create a driver specifically for the webextension tests

Will the new driver be created inside the webextension test file or conftest.py? I am trying the former one with a class based approach (different class for Firefox and chrome/edge) and getting errors like the extension won't load (the install method won't fail but I can't see the extension being loaded in the browser)

Edit: Extension is loaded but pages.load("blank.html") is opening the page in the window that doesn't have the extension installed.

[cgoldberg]

Will the new driver be created inside the webextension test file or conftest.py

The solution I'm suggesting would create it within the test.

It's going to be difficult to have a common test that supports all browsers. I would write a test for Firefox (that's skipped by other browsers) and one for Chrome/Edge (that's skipped by other browsers). For an example, look at ./py/test/selenium/webdriver/firefox/firefox_sizing_tests.py.

[navin772]

@cgoldberg yes, I am trying it as you described above. I noticed one issue right now which results in my tests getting failures, while launching the driver, it opens 2 browser windows, extension installation is successful and when I do pages.load("blank.html"), it opens the blank.html page in the window that doesn't have the extension installed (the other tab has it installed).
If I do driver.get("https://www.webpagetest.org/blank.html") instead of pages.load("blank.html") it works fine.

I am trying to fix this.

[navin772]

I think pages.load uses the normal driver and not the chromium_driver I added. For now, only this is working:

chromium_driver.get("https://www.webpagetest.org/blank.html")

@cgoldberg @shbenzer can you please have a look and see if something can be done so that pages.load work as expected.

[shbenzer]

I think pages.load uses the normal driver and not the chromium_driver I added. For now, only this is working:

chromium_driver.get("https://www.webpagetest.org/blank.html")

the pages fixture is defined in conftest.py as the following:

@pytest.fixture
def pages(driver, webserver):
    class Pages:
        def url(self, name, localhost=False):
            return webserver.where_is(name, localhost)

        def load(self, name):
            driver.get(self.url(name))

    return Pages()

You're correct. The problem is this uses the initial driver - you'll need to create your own Pages class if you wish to use this functionality

[navin772]

you'll need to create your own Pages class

@shbenzer I guess that will be a lot of changes to do just for a single test file, also we have an issue in running this multi driver approach in CI-RBE which is giving a user data dir error (maybe due to these 2 drivers), although browser tests on github runner are working.

[cgoldberg]

Just a note on this:
Starting with Chrome 136, if you set --remote-debugging-pipe, you must also set a --user-data-dir:

https://developer.chrome.com/blog/remote-debugging-port

So enable_extensions will only work if that is also set.

Also note:
--remote-debugging-pipe makes the CDP connection between chromedriver and the browser use a pipe, not a port. So starting Chrome with this flag will make it so CDP things like driver.start_devtools() will not work and fail with a cryptic error.

[shbenzer]

Also note: --remote-debugging-pipe makes the CDP connection between chromedriver and the browser use a pipe, not a port. So starting Chrome with this flag will make it so CDP things like driver.start_devtools() will not work and fail with a cryptic error.

That’s a great note - we should definitely advise users about this in the docstring and the docs. I’ll make a pr real quick

[emanlove]

Just wanted to note that from my reading of the --remote-debugging-pipe description it is not exclusively a webextension command line switch. So if one adds it separately and chooses to disable web extensions via this method, as in,

    options = webdriver.ChromeOptions()
    options.add_argument("--remote-debugging-pipe")
    options.enable_webextensions = False

it will negate that command line switch and might be tough to debug. Probably the method documentation should note that disabling removes both switches.

[navin772]

This was discussed but no conclusion was made - #15794 (comment), what do you think would be the best way to resolve this?

Adding docs is good.