Meeting RAG

Technical Description

• Implemented two pipeline to process the files, first takes the docs, extract images and texts, store images on cloudinary and caption them using mistral pixtral 12b model, then chunk texts and store all in the pinecone vector database.
• In second pipelines, each pdf page sent for ocr with ocr.space api, and stored each page on cloudinary and embedding stored in the pinecone.
• Implemented realtime transcriptions with interim and final transcript using web socket connections to server and then websocket connection with deepgram api dual channel.
• Implement some agent that search question on user behalf, and user also can type the questions. Retrieval fetched vectors and data with cosine similarity, internally enhance user query, web search using serper api, and return the response to user, and user can see actual details in web search, llm reply and doc search format (actual page similar to user question).
• Also allowed email based authentications

Tech Stack

AI Models