Private Local LLM Developer: SQL Data Room Setup & On-Premise LLM Deployment (NDA Required)

Project Overview

We are looking for an experienced AI/ML developer to help us build a private, locally hosted Large Language Model (LLM) for our project. The goal is a secure, on-premise AI solution that runs entirely on our own server, with no reliance on external cloud AI services, so our data stays fully within our control.

You will work closely with our internal IT team from the planning stage through deployment.

Scope of Work

Phase 1: Discovery & Brainstorming
Collaborate with our IT team to understand our current infrastructure, data sources, and business needs. Recommend the best architecture, model choice (e.g., Llama, Mistral, Qwen), and approach (RAG, fine-tuning, or a combination).

Phase 2: SQL Data Room Development
Design and build a structured SQL data room that pulls data from our existing systems, stores and organizes historical data for long-term use, and prepares clean, well-structured data for the LLM to query and learn from.

Phase 3: Local LLM Deployment
Install, configure, and optimize an open-source LLM on our server. Connect the LLM to the SQL data room so it can retrieve and use our data accurately. Set up secure access controls and logging.

Phase 4: Testing, Handover & Documentation
Test for accuracy, performance, and security. Provide clear documentation and knowledge transfer so our IT team can maintain and expand the system.

Required Skills & Experience

Proven experience deploying open-source LLMs on local or on-premise servers
Strong SQL skills and database design experience (SQL Server, PostgreSQL, or MySQL)
Experience with RAG pipelines, vector databases, and embedding models
Familiarity with tools such as Ollama, vLLM, LangChain, LlamaIndex, or similar
Understanding of data security and privacy best practices
Good communication skills and comfort working alongside an internal IT team

Confidentiality

This project involves sensitive business data. The selected freelancer will be required to sign a Confidentiality / Non-Disclosure Agreement (NDA) before work begins. All data, code, and project details will remain the exclusive property of our company.

To Apply, Please Include

A brief summary of similar private or local LLM projects you have completed
Your proposed approach and the tools you would recommend
Estimated timeline and cost per phase
Your availability for meetings with our IT team

We look forward to working with a skilled professional who values security, clear communication, and quality work.