Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 44 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,77 +3,74 @@
![Test](https://github.com/gardenlinux/parse_features_lib/actions/workflows/pytests.yml/badge.svg)
![security check](https://github.com/gardenlinux/parse_features_lib/actions/workflows/bandit.yml/badge.svg)

# Parse features lib
## Garden Linux Python Library

This library includes tooling to build and distribute [Garden Linux](https://github.com/gardenlinux/gardenlinux).
Python tooling to work with [Garden Linux](https://github.com/gardenlinux/gardenlinux) features, flavors, OCI artifacts, repositories, and releases.
It is primarily targeted at Garden Linux developers and CI pipelines rather than end users.

Features:
The library follows the intent of [Semantic Versioning](https://semver.org) for its public APIs.

- compare APT repositories
- parse features
- parse flavors
- push OCI artifacts to a registry
### Features

## Quickstart
- **Feature management**: parse, filter, and work with Garden Linux feature sets
- **Flavor processing**: parse `flavors.yaml` and generate flavor combinations
- **Repository utilities**: compare APT repositories and query package versions
- **OCI operations**: push OCI artifacts and manifests to container registries
- **S3 integration**: upload/download artifacts from S3 buckets
- **GitHub integration**: create and manage GitHub releases

### Example: get a list of features for a given cname
## Documentation

**Inclusion via poetry**:
Full documentation is available at the **Garden Linux Python Library Documentation** site:
[https://gardenlinux.github.io/python-gardenlinux-lib/](https://gardenlinux.github.io/python-gardenlinux-lib/)

`gardenlinux = { git = "https://github.com/gardenlinux/python_gardenlinux_lib", rev="0.6.0" }`
The docs include:

```python
from gardenlinux.features import Parser
- **Command-Line Interface**: `gl-features-*`, `gl-flavors-*`, `gl-oci`, `gl-s3`, `gl-gh-release`
- **API Reference**: modules, classes, and functions (e.g. `Parser`, `CName`, `Container`, `Repository`)
- **Release documentation**: versioning and release process

cname = "aws-gardener_prod"
feature_list = Parser().filter_as_list(cname)
print(f"features of {cname}:")
for feature in feature_list:
print(feature)
```

## Developer Documentation
## Installation

The library is documented with docstrings, which are used to generate the developer documentation available [here](https://gardenlinux.github.io/python-gardenlinux-lib/).
### Using `poetry` (from Git)

## Push OCI artifacts to a registry
Add the library as a dependency in your `pyproject.toml`:

this tool helps you to push oci artifacts.
```toml
[tool.poetry.dependencies]
gardenlinux = { git = "https://github.com/gardenlinux/python-gardenlinux-lib", rev = "0.10.5" }
```

### Installation
Then install:

```bash
git clone https://github.com/gardenlinux/python-gardenlinux-lib.git
mkdir venv
python -m venv venv
source venv/bin/activate.sh
poetry install
gl-oci --help
```

### Usage

The process to push a Gardenlinux build-output folder to an OCI registry is split into two steps: In the first step all files are pushed to the registry and a manifest that includes all those pushed files (layers) is created and pushed as well. An index entry that links to this manifest is created offline and written to a local file but not pushed to any index. This push to an index can be done in the second step where the local file containing the index entry is read and pushed to an index. The seperation into two steps was done because pushing of manifests takes long and writes to dedicated resources (possible to run in parallel). Updating the index on the other hand is quick but writes to a share resource (not possible to run in parallel). By splitting the process up into two steps it is possible to run the slow part in parallel and the quick part sequentially.

#### 1. Push layers + manifest

To push layers you have to supply the directory with the build outputs `--dir`. Also you have to supply cname (`--cname`), architecture `--arch` and version `--version` of the build. This information will be included in the manifest. You have to supply an endpoint where the artifacts shall be pushed to `--container`, for example `ghcr.io/gardenlinux/gardenlinux`. You can disable enforced HTTPS connections to your registry with `--insecure True`. You can supply `--cosign_file <filename>` if you want to have the hash saved in `<filename>`. This can be handy to read the hash later to sign the manifest with cosign. With `--manifest_file <filename>` you tell the program in which file to store the manifests index entry. This is the file that can be used in the next step to update the index. You can use the environment variable GL_CLI_REGISTRY_TOKEN to authenticate against the registry. Below is an example of a full program call of `push-manifest`
### Local development setup

```bash
GL_CLI_REGISTRY_TOKEN=asdf123 gl-oci push-manifest --dir build-metal-gardener_prod --container ghcr.io/gardenlinux/gl-oci --arch amd64 --version 1592.1 --cname metal-gardener_prod --cosign_file digest --manifest_file oci_manifest_entry_metal.json
git clone https://github.com/gardenlinux/python-gardenlinux-lib.git
cd python-gardenlinux-lib
python -m venv venv
source venv/bin/activate
poetry install
```

#### 2. Update index with manifest entry
## Quickstart

Parameters that are the same as for `push-manifest`:
### Example: list features for a given `cname`

- env-var `GL_CLI_REGISTRY_TOKEN`
- `--version`
- `--container`
- `--manifest-file` this time this parameter adjusts the manifest entry file to be read from instead of being written to
```python
from gardenlinux.features import Parser

A full example looks like this:
cname = "aws-gardener_prod"
feature_list = Parser().filter_as_list(cname)

```bash
GL_CLI_REGISTRY_TOKEN=asdf123 gl-oci update-index --container ghcr.io/gardenlinux/gl-oci --version 1592.1 --manifest_file oci_manifest_entry_metal.json
print(f"features of {cname}:")
for feature in feature_list:
print(feature)
```

For more examples and for all CLI tools, see the **Command-Line Interface** and **API Reference** sections in the docs:
[https://gardenlinux.github.io/python-gardenlinux-lib/](https://gardenlinux.github.io/python-gardenlinux-lib/)