autoscout24.ch advanced search

This project is a basic search engine built with Python and Selenium to automate the search for car listings on AutoScout24. It is designed to provide functionality beyond the default features offered by the AutoScout24 platform.

About

How It Works

The search engine allows users to input parameters such as make, model, and year, which are used to pre-filter the results through query parameters. Once the initial result set is retrieved, the tool goes a step further by opening each car listing individually to look for additional, specific features or optional equipment — something that AutoScout24 does not natively support.

For example, while AutoScout24 allows filtering for general categories like "speakers," this tool can help find listings with specific options, such as a Bang & Olufsen sound system.

Why Selenium?

AutoScout24 employs effective scraping protection, making traditional methods of data extraction insufficient. This project uses Selenium, a browser automation framework, to navigate and interact with the website like a real user. Selenium ensures compatibility with AutoScout24's security measures, providing reliable access to the desired data. Note, that the browser is not run headless, as this can be detected easily.

Recommendations

To maximize performance and avoid unnecessary overhead:

• Use the built-in AutoScout24 filters to narrow down the search to around 100 or fewer listings before running the tool.
• Avoid overly broad queries to ensure the script runs efficiently and within reasonable time limits.

Features

• Searches car listings based on specified search criteria.
• Accepts cookies automatically to navigate the website.
• Handles pagination to retrieve multiple pages of listings.
• Takes screenshots of the listings and saves them locally.
• Displays results using NiceGUI for a simple web interface.

Requirements

• Python 3.x
• Selenium
• NiceGUI
• ChromeDriver (compatible with your version of Chrome)

(tested on WSL2 using Ubuntu)

Installation

1. Clone the repository:

   git clone https://github.com/mbolli/autoscout-advanced-search.git
   cd repository

2. Install the required Python packages:

   pip install selenium nicegui

3. Download and install ChromeDriver from here and ensure it is in your system's PATH or update the path in the code.

Usage

1. Open main.py and modify the searches variable as needed to customize your search criteria.

2. Run the script:

   python main.py

3. The search engine will navigate to the specified URL, perform the searches, and save screenshots of the listings in the ./media/ directory, if the special search matches.

4. After searching, a web interface will display the results.

Code Overview

Main Functions

• search_with_selenium(url, search):

  • Main function that performs the searching. It accepts a URL and a search dictionary containing the search parameters.

• process_car_listings(driver, ul, search_term, result):

  • Processes the car listings found on the current page, clicking on each listing to gather more details and taking screenshots. Screenshots are stored in the media folder.

• render_results(results):

  • Displays the results using NiceGUI, showing links to the car listings and their corresponding screenshots.

Search Parameters

The searches list in the main() function defines the search criteria. Each dictionary can include all query parameters as used by the AutoScout24 website:

• makeModelVersions[0][makeKey]: The make of the car (e.g. bmw).
• makeModelVersions[0][modelKey]: The model of the car (e.g. m340).
• makeModelVersions[1][makeKey]: Another make of the car (e.g. mercedes-benz)
• makeModelVersions[1][modelKey]: Another model of the car (e.g. c-43-amg)
• firstRegistrationYearTo: The latest year of registration (e.g. 2019)
• ...
• special: A list of special features to search for (e.g. ['individual', 'designo'])

Example Search

searches = [
    {
        'bodyTypes[0]': 'estate',
        'makeModelVersions[0][makeKey]': 'mercedes-benz',
        'makeModelVersions[0][modelKey]': 'e-63-amg',
        'firstRegistrationYearTo': '2017',
        'special': ['Bang & Olufsen', 'Burmester', 'B&O'],
    }, 
    {...}
]

License

See the LICENSE file for details.