Tweeter Insight Engine

This project is a Tweeter Insight Engine that provides insights based on user input. The project is structured using Kubernetes and consists of several components including a webapp, an analyzer, an ingestor, and a Redis database.

Project Structure

The project is structured into two main folders:

1. k8s: This folder contains Kubernetes configurations for the different components of the project. These include:

   • Webapp (Deployment and Service[Load Balancer])
   • Analyzer (Deployment and Service)
   • Ingestor (Deployment and Service)
   • Redis (Deployment and Service)
   • Persistent Storage Claim

2. images: This folder contains the images for the ingestor, webapp, and analyzer.

Workflow

The workflow of the project is as follows:

1. The webapp, which serves as the frontend, receives input from the user (keywords and source[API, CSV]) and sends it to the analyzer for analysis.
2. The analyzer sends the keywords and source to the ingestor, which checks if the data exists in the Redis database. If not, it fetches the data from the source (API or CSV) and loads it into Redis.
3. Once the data is loaded into Redis, the ingestor sends a 200 OK response back to the analyzer. The analyzer then fetches the data from Redis and performs analysis on it.
4. The results of the analysis, which could be text data or image names, are returned as a list of JSON objects back to the webapp. If the analysis involves images, the image names are saved in the persistent storage claim that is shared between the webapp and the analyzer.
5. Finally, the webapp displays these insights in HTML format to the user.

Getting Started

To get started with this project, you need to have Kubernetes installed and configured on your machine. Once you have that, you can clone this repository and run deploy.sh, which runs all the k8s configurations.