Gluu.Repair is seeking to enhance the efficiency of its cloud infrastructure by optimizing the deployment of large language models (LLMs) on Amazon EC2 instances.

Currently, the models are running continuously, leading to unnecessary costs. The goal of this project is to explore and implement solutions that allow these models to be triggered only on demand, thereby reducing operational expenses. Students will investigate the potential of using Amazon Bedrock or other cloud providers to host the LLMs more efficiently. Additionally, the project involves gaining a deeper understanding of embedding techniques in QDrant, a vector database, to improve data handling and retrieval processes. This project provides an opportunity for learners to apply their knowledge of cloud computing, machine learning, and database management in a practical setting.

The project deliverables include a comprehensive report detailing the proposed solution for on-demand LLM deployment, including cost-benefit analysis and implementation steps. Additionally, students will provide a demonstration of the optimized infrastructure setup using a selected cloud provider. The final deliverable will also include documentation on the embedding process in QDrant, highlighting any improvements made. This will ensure that Gluu.Repair can efficiently manage its LLM operations while minimizing costs.