Generating and evaluating relevant documentation

GERD is developed as an experimental library to investigate how large language models (LLMs) can be used to generate and analyze (sets of) documents.

This project was initially forked from Llama-2-Open-Source-LLM-CPU-Inference by Kenneth Leung.

Quickstart

If you just want to it try out, you can clone the project and install dependencies with pip:

git clone https://github.com/caretech-owl/gerd.git
cd gerd
pip install -e ".[full]"
python examples/hello.py

Source: examples/hello.py

```python --8<-- "examples/hello.py" ```

If you want to try this out in your browser, head over to binder 👉 . Note that running LLMs on the CPU (and especially on limited virtual machines like binder) takes some time.

Question and Answer Example

Follow quickstart but execute gradio with the qa_frontend instead of the example file. When the server is done loading, open http://127.0.0.1:7860 in your browser.

gradio gerd/frontends/qa_frontend.py
# Some Llama.cpp outut
# ...
# * Running on local URL:  http://127.0.0.1:7860

Click the 'Click to Upload' button and search for a GRASCCO document named Caja.txt which is located in the tests/data/grascoo folder and upload it into the vector store. Next, you can query information from the document. For instance Wie heißt der Patient? (What is the patient called?).

Prompt Chaining

Prompt chaining is a prompt engineering approach to increase the 'reflection' of a large language model onto its given answer. Check examples/chaining.py for an illustration.

python examples/chaining.py
# ...
====== Resolved prompt =====

system: You are a helpful assistant. Please answer the following question in a truthful and brief manner.
user: What type of mammal lays the biggest eggs?

# ...
Result: Based on the given information, the largest egg-laying mammal is the blue whale, which can lay up to 100 million eggs per year. However, the other assertions provided do not align with this information.

Source: examples/chaining.py

```python --8<-- "examples/chaining.py" ```

Config: config/gen_chaining.yml

```yaml --8<-- "config/gen_chaining.yml" ```

As you see, the answer does not make much sense with the default model which is rather small. Give it a try with meta-llama/Llama-3.2-3B. To use this model, you need to login with the huggingface cli and accept the Meta Community License Agreement.

Full Documentation

A more detailled documentation can be found here 👉 .