Fam.ai (Novathon Hackathon) Project

hackathon/Fam.ai (Novathon)/README.md

@@ -0,0 +1,136 @@
+
+# Fam.ai - Gen AI-Based Medical Information Management System
+
+### Project by **Team Lethimcode**
+
+Fam.ai is a next-generation **AI-powered medical information management system**, secured using **blockchain technology**. The project utilizes **LLMware** and **RAG (Retrieval-Augmented Generation)** to enable seamless communication between users and the platform, ensuring efficient data storage, retrieval, and management.
+
+---
+
+<div align="center">
+  <img src="https://cryptologos.cc/logos/ethereum-eth-logo.png" alt="Ethereum Logo" width="100" height="100"/>
+  <img src="https://cryptologos.cc/logos/polygon-matic-logo.png" alt="Polygon Logo" width="100" height="100"/>
+  <img src="https://cdn.nwe.io/files/x/a7/7e/6427076892cbd1e8f289553666aa.jpg" alt="llmware" width="200" height="100"/>
+</div>
+
+---
+
+## 🚀 **Key Features**
+
+1. **Save Medical Prescription Summaries**  
+   Store and retrieve summaries of medical prescriptions for quick access and better healthcare management.  
+
+2. **Cancer Detection from Brain MRI**  
+   Advanced AI capabilities help detect cancerous patterns in Brain MRI scans.  
+
+3. **Generate Comprehensive Health Reports**  
+   Fam.ai provides an overall health report based on user inputs and medical history.
+
+---
+
+## 🌟 **Use Cases**
+
+1. **Tool for Doctors**  
+   A powerful assistant for doctors to manage patient records and improve diagnostic accuracy.  
+
+2. **Insurance Company Scoring**  
+   Simplifies risk evaluation for insurance companies with precise medical insights.  
+
+3. **Family Healthcare Management**  
+   Ensures secure storage and easy sharing of medical records for the entire family.
+
+---
+
+## 💻 **Tech Stack**
+
+- **Backend:** Python  
+- **Frontend:** React, Framer Motion, GSAP  
+- **AI Integration:** LLMware, RAG  
+- **Blockchain:** Solidity  
+
+---
+
+## 🛠️ **How to Run the Project**
+
+### **Frontend**
+
+1. Navigate to the frontend directory:  
+   ```bash
+   cd frontend
+   ```
+2. Install dependencies:  
+   ```bash
+   npm i
+   ```
+3. Start the development server:  
+   ```bash
+   npm run dev
+   ```
+
+---
+
+### **Backend**
+
+1. Navigate to the backend directory:  
+   ```bash
+   cd backend
+   ```
+2. Install Python dependencies:  
+   ```bash
+   pip install -r requirements.txt
+   ```
+3. Initialize the blockchain environment:  
+   ```bash
+   mkdir blockchain
+   cd blockchain
+   brownie init
+   ```
+4. Add the following contracts to the `contracts` folder:
+   - **NFTmint.sol**  
+   - **FundMe.sol**  
+5. Configure the environment by adding your private key and Web3 Infura Project ID in a `.env` file.  
+
+6. Compile the contracts:  
+   ```bash
+   brownie compile
+   ```
+7. Add the custom Polygon network:  
+   ```bash
+   brownie networks add custom polygon host=https://rpc.cardona.zkevm-rpc.com chainid=2442
+   ```
+8. Return to the backend folder and start the backend server:  
+   ```bash
+   cd ..
+   python main.py
+   ```
+
+---
+
+### **LLMware Integration**
+
+#### **Run with High Response Time (Cloud)**  
+Use the following Kaggle notebook for optimal LLMware performance:  
+[LLMware Notebook on Kaggle](https://www.kaggle.com/code/idhanush/notebook46fcbc644e)
+
+#### **Run Locally**  
+1. Download the notebook to your computer.  
+2. Open and execute the notebook using Jupyter Notebook locally.
+
+---
+
+## ✨ **Contributing**
+
+We welcome contributions to enhance Fam.ai!  
+Feel free to fork the repository, open issues, or submit pull requests.
+
+---
+
+## 📄 **License**
+
+This project is licensed under the [MIT License](LICENSE).
+
+---
+
+## 🧑‍💻 **Team Members**
+
+We are **Team Lethimcode**, passionate about revolutionizing healthcare with the power of AI and blockchain. 🚀

hackathon/Fam.ai (Novathon)/ai/README.md

@@ -0,0 +1 @@
+# FAM_AI_Novathon2024 - ai

hackathon/Fam.ai (Novathon)/ai/brain_cancer_detector.py

@@ -0,0 +1,42 @@
+from tensorflow.keras.models import load_model
+import cv2 as cv
+import imutils
+
+model = load_model('brain_tumor_detector.h5')
+mriImage = cv.imread('notumor.jpg', 1)
+
+def predictTumor(image_path):
+    image = cv.imread(image_path, 1)
+    gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
+    gray = cv.GaussianBlur(gray, (5, 5), 0)
+
+    # Threshold the image, then perform a series of erosions +
+    # dilations to remove any small regions of noise
+    thresh = cv.threshold(gray, 45, 255, cv.THRESH_BINARY)[1]
+    thresh = cv.erode(thresh, None, iterations=2)
+    thresh = cv.dilate(thresh, None, iterations=2)
+
+    # Find contours in thresholded image, then grab the largest one
+    cnts = cv.findContours(thresh.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
+    cnts = imutils.grab_contours(cnts)
+    c = max(cnts, key=cv.contourArea)
+
+    # Find the extreme points
+    extLeft = tuple(c[c[:, :, 0].argmin()][0])
+    extRight = tuple(c[c[:, :, 0].argmax()][0])
+    extTop = tuple(c[c[:, :, 1].argmin()][0])
+    extBot = tuple(c[c[:, :, 1].argmax()][0])
+
+    # crop new image out of the original image using the four extreme points (left, right, top, bottom)
+    new_image = image[extTop[1]:extBot[1], extLeft[0]:extRight[0]]
+
+    image = cv.resize(new_image, dsize=(240, 240), interpolation=cv.INTER_CUBIC)
+    image = image / 255.
+
+    image = image.reshape((1, 240, 240, 3))
+
+    res = model.predict(image)
+
+    return res
+
+# print(predictTumor("notumor.jpg"))

hackathon/Fam.ai (Novathon)/ai/chatbot.py

@@ -0,0 +1,38 @@
+from typing import Union
+from llmware.models import ModelCatalog
+
+context = '''User: Hi, can you help me keep track of the projects I'm working on?
+Assistant: Of course! Let me know about your projects, and I’ll create a record for them.
+
+User: Great! I’m currently working on a decentralized voting system using blockchain and smart contracts.
+Assistant: Got it. Could you provide more details, like the tech stack or any challenges you’re facing?
+
+User: Sure! The project is called VoteChain. I’m using Ethereum for smart contracts with Solidity, Python for the backend using FastAPI, and a React.js frontend. The challenge is ensuring voter privacy while maintaining transparency.
+Assistant: Noted! I’ll add VoteChain to your project list, including the tech stack and the voter privacy challenge.
+
+User: Also, I’m collaborating with a friend named Divya on this. She’s handling the UI/UX design and is really creative.
+Assistant: Nice! I’ll note that Divya is collaborating with you on VoteChain and is responsible for UI/UX design. Anything else?
+
+User: No, that’s all for now. Thanks!
+Assistant: You’re welcome! Let me summarize for you:
+
+Project Name: VoteChain
+Description: A decentralized voting system using blockchain and smart contracts.
+Tech Stack: Ethereum (Solidity), Python (FastAPI), React.js.
+Key Challenge: Ensuring voter privacy while maintaining transparency.
+Collaboration: Divya (UI/UX design).
+Does this look good to you?
+
+User: Perfect!'''
+
+chat_bot = ModelCatalog().load_model("bartowski/Meta-Llama-3-8B-Instruct-GGUF")
+def get_response(prompt:str, context:Union[None, str]=""):
+    response = chat_bot.inference(prompt, add_context=context)
+    return response
+
+print(get_response(prompt="What do you know about the user ?", context=context))
+
+
+
+
+

hackathon/Fam.ai (Novathon)/ai/test.py

@@ -0,0 +1 @@
+print("hello")

hackathon/Fam.ai (Novathon)/ai/vision.py

@@ -0,0 +1,19 @@
+import google.generativeai as genai
+from PIL import Image
+import dotenv
+import os
+dotenv.load_dotenv()
+genai.configure(api_key=os.environ.get('GOOGLE_AI_API'))
+model = genai.GenerativeModel(model_name="gemini-1.5-flash")
+def ask_ai(img, prompt):
+    response = model.generate_content([
+        prompt,
+        img])
+    return response.text
+
+pil_image = Image.open('demo-rx.png')
+classifier_prompt = "Classify the image. 1 for MRI and 2 for not MRI"
+ocr_prompt = "Provide the summary from the medical prescription"
+res = ask_ai(pil_image, ocr_prompt)
+print(res)
+

hackathon/Fam.ai (Novathon)/backend/README.md

@@ -0,0 +1 @@
+# FAM_AI_Novathon2024 - backend

hackathon/Fam.ai (Novathon)/backend/ai_functions/detector.py

@@ -0,0 +1,42 @@
+from tensorflow.keras.models import load_model
+import cv2 as cv
+import imutils
+model = load_model('ai_functions/brain_tumor_detector.h5')
+
+
+
+def predictTumor(image_path):
+    image = cv.imread(image_path, 1)
+    gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
+    gray = cv.GaussianBlur(gray, (5, 5), 0)
+
+    # Threshold the image, then perform a series of erosions +
+    # dilations to remove any small regions of noise
+    thresh = cv.threshold(gray, 45, 255, cv.THRESH_BINARY)[1]
+    thresh = cv.erode(thresh, None, iterations=2)
+    thresh = cv.dilate(thresh, None, iterations=2)
+
+    # Find contours in thresholded image, then grab the largest one
+    cnts = cv.findContours(thresh.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
+    cnts = imutils.grab_contours(cnts)
+    c = max(cnts, key=cv.contourArea)
+
+    # Find the extreme points
+    extLeft = tuple(c[c[:, :, 0].argmin()][0])
+    extRight = tuple(c[c[:, :, 0].argmax()][0])
+    extTop = tuple(c[c[:, :, 1].argmin()][0])
+    extBot = tuple(c[c[:, :, 1].argmax()][0])
+
+    # crop new image out of the original image using the four extreme points (left, right, top, bottom)
+    new_image = image[extTop[1]:extBot[1], extLeft[0]:extRight[0]]
+
+    image = cv.resize(new_image, dsize=(240, 240), interpolation=cv.INTER_CUBIC)
+    image = image / 255.
+
+    image = image.reshape((1, 240, 240, 3))
+
+    res = model.predict(image)
+
+    if res < 0.5:
+        return False
+    return True

hackathon/Fam.ai (Novathon)/backend/ai_functions/llmware_utils.py

@@ -0,0 +1,18 @@
+import requests
+
+from global_vars import Var
+
+
+def get_llmware_response(prompt, context):
+    url = Var.kaggle_url+'chat'
+    print(context)
+    print(url)
+    res = requests.post(url,
+                        json={'prompt': str(prompt), 'context': str(context)})
+    print(res)
+    res = res.json()
+    if not res:
+        return False
+    print(res)
+    llm_response = res['llm_response']
+    return llm_response

hackathon/Fam.ai (Novathon)/backend/ai_functions/llmware_utils.py~

@@ -0,0 +1,19 @@
+import requests
+
+from global_vars import Var
+
+
+def get_llmware_response(prompt, context):
+    url = Var.kaggle_url+'chat'
+    print(context)
+    print(url)
+    res = requests.post(url,
+                        json={'prompt': str(prompt), 'context': str(context)})
+    print(res)
+    res = res.json()
+    if not res:
+        return False
+    print(res)
+    res = res.get('response')
+    llm_response = res['llm_response']
+    return llm_response

hackathon/Fam.ai (Novathon)/backend/ai_functions/vision_ai.py

@@ -0,0 +1,18 @@
+import PIL
+import google.generativeai as genai
+from PIL import Image
+import dotenv
+import os
+
+dotenv.load_dotenv()
+genai.configure(api_key=os.environ.get('GOOGLE_AI_API'))
+model = genai.GenerativeModel(model_name="gemini-1.5-flash")
+classifier_prompt = "Classify the image. 1 for MRI and 2 for not MRI"
+ocr_prompt = "Provide the summary from the medical prescription"
+
+
+def ask_ai(img: PIL.Image, prompt: str):
+    response = model.generate_content([
+        prompt,
+        img])
+    return response.text

hackathon/Fam.ai (Novathon)/backend/auth/auth.py

@@ -0,0 +1,8 @@
+from fastapi import Header, HTTPException
+
+
+def get_address(address: str = Header(...)):
+    print(address)
+    if not address:
+        raise HTTPException(status_code=400, detail="Address header missing")
+    return address