Custom Response Output for /invoke endpoint

[rajnikhil-netapp]

Below code is being used when /invoke is called with the input or request schema.

def create_chain(verbose=True, search_type="similarity", top_k=int(os.environ.get("TOP_K", 4)), default_user=''):
    logger.info("Creating chain with verbose=%s, search_type=%s, top_k=%s, default_user=%s", verbose, search_type, top_k, default_user)
    """
    This function creates a chain of language models and vector databases for use in
    natural language processing tasks. It takes in optional parameters such as verbose,
    search_type, top_k and default_user to customize the chain.
    Args:
        verbose (bool): If True, prints out the chain.
        search_type (str): The type of search to be performed.
        top_k (int): The number of results to return.
        default_user (str): The user to be used for the chain.
    Returns:
        chain (Chain): The chain of language models and vector databases.
    """
    vector_db = ElasticClient(user=default_user).as_retriever(
                            search_type=search_type,
                            search_kwargs={
                                "k": top_k
                                }
                        )
    verbose_kwargs = dict(verbose=verbose)
    llm = get_langchain_llm(user=default_user)
    condense_prompt = SearchPrompt().get_condense_question_prompt()
    question_prompt = SearchPrompt().get_question_prompt()

    chain = get_chain(llm, vector_db, condense_prompt, question_prompt, verbose_kwargs)
    logger.info("Chain created successfully")
    return chain

def get_chain(llm, vector_db, condense_prompt, question_prompt, verbose_kwargs):
    """
    This function generates a ConversationalRetrievalChainWithSources object from the
    given parameters. It takes in a Language Model, Vector Database, Condense Prompt,
    Question Prompt, and Verbose Keyword Arguments and returns a chain with the input type
    of ChatHistory.
    Args:
        llm (LanguageModel): The Language Model to use.
        vector_db (VectorDatabase): The Vector Database to use.
        condense_prompt (str): The prompt to use when condensing.
        question_prompt (str): The prompt to use when combining docs.
        verbose_kwargs (dict): The verbose keyword arguments to use.
    Returns:
        ConversationalRetrievalChainWithSources: A ConversationalRetrievalChainWithSources object.
    """
    chain = ConversationalRetrievalChainWithSources.from_llm(
        llm=llm,
        retriever=vector_db,
        condense_question_prompt=condense_prompt,
        combine_docs_chain_kwargs={
            "prompt": question_prompt
        },
        **verbose_kwargs
    )

    return chain.with_types(input_type=ChatHistory)

app = FastAPI(
    title="API Server",
    version="1.0",
    description="API Interface",
)

chain = create_chain(search_type=os.environ.get('VECTOR_SEARCH_TYPE', 'similarity'), default_user=os.environ.get('DEFAULT_SYSTEM_USER', ''))
add_routes(app, chain)

I am getting the response in the below format

{ "output": { "answer": "", "source_documents": [] }, "callback_events": [], "metadata": { "run_id": "0c45f613-7820-406b-9b8c-67ee9f6884ce" } }

How can I capture the run_id from the response.
By the way I want this to be done from the backend itself.
I understood that we can write a client and format the output.

The reason I want to do this in backend is because I want to utilize the input and output of the chain
read few params from the input and read few params from the output and construct a json object and want to save it to elasticsearch.

is this even possible?

[rajnikhil-netapp]

Another thing I am also planning to send session_id to /invoke method, How can I read it and use it further?