Linux Security Engineer — Air-Gapped AI Document System (Core + Gateway) | On-Site Delhi

Job Title
Linux Security Engineer — Air-Gapped AI Document System (Core + Gateway) | On-Site Delhi

Project Overview
We are building a secure, air-gapped, two-machine AI document processing system for handling sensitive financial documents. The system consists of:
• Core machine — an offline, hardened Linux server that performs retrieval, analysis, and enforcement. It must have no internet access by design.
• Gateway machine — a controlled ingress/egress point that mediates all content transfer into and out of the core, over a dedicated link-local connection.
This is a security-first infrastructure project, not a web app. The successful candidate will implement, harden, and deploy the full local stack on physical hardware provided by us.
Note: Detailed architecture, access model, and security design will be shared only after NDA and selection. This posting describes the required capabilities.

Scope of Work
The freelancer will be responsible for setting up and delivering the following capabilities across both machines:
1. Air-Gapped Network Isolation
• Configure the core machine with no default route / no internet access.
• Enforce strict firewall rules so only a single designated service identity can reach the gateway over a link-local peer.
• All other identities and services must be provably blocked from network egress.
• Lockdown must persist across reboots via a registered service.
2. Hardened Linux Foundation
• Ubuntu/Debian server hardening with least-privilege service accounts (multiple, scoped identities — not a single shared user).
• Directory tree with strict ownership, group segregation, and traversal control.
• Sealed configuration and key storage accessible only to designated identities.
• No blanket permissions; every path must have an explicit owner and access rule.
3. Vector Store + Search Index (RAG Backend)
• Install and secure a Qdrant vector database bound to loopback only, with API-key authentication and denial verified.
• Set up a trust-class-segregated index (e.g., BM25/FTS) where content classes are isolated at the namespace level — searches in one class must never surface results from another.
• Implement withdrawal/invalidation so removed content is no longer retrievable.
4. Redis with Unix Socket + ACLs
• Run Redis over a Unix domain socket only (no TCP exposure).
• Configure per-identity ACLs and credential files so each service authenticates with its own scoped key.
• Verify cross-identity access is denied.
5. Secure Content Ingestion (SME / Domain Transfer)
• Build the gateway-to-core content transfer pipeline (envelope-based pull model).
• All inbound content must pass through a quarantine / inert-extraction stage — active content (macros, scripts, embedded executables) is detected and blocked, never parsed.
• Content is canonicalized (Unicode normalization, hidden-character detection) before entering the trusted store.
6. LLM Integration with Prompt-Injection Defence
• Wire an LLM serving layer (local, e.g., llama-server + GGUF) behind a deterministic retrieval proxy.
• Implement prompt-injection screening on both inbound content and assembled prompts (pattern bundle + structural isolation of untrusted context).
• The model must never be in the authorization path — all access decisions are deterministic code, not model output.
7. Deterministic Access & Policy Enforcement
• Implement a capability/authorization layer that resolves “who may do what” from a signed policy file — no LLM, no inference in the decision path.
• Identity must be kernel-authenticated (e.g., socket peer credentials), never taken from the request body.
• Fail-closed: any uncertainty denies access.
8. Tamper-Evident Audit & Receipts
• Append-only, hash-chained audit ledger for all privileged operations.
• Per-gate evidence receipts (JSON + SHA256) stored in a location readable by an independent assurance role.
• Ledger integrity must be verifiable end-to-end.
9. Outbound Content Firewall
• A deterministic outbound scanner that blocks secrets, high-entropy tokens, protected-content anchors, and missing lineage before anything leaves the core.
• Layered checks (pattern, entropy, canary, lineage), fail-closed.
10. API Layer + Key Management
• Expose required functionality via a controlled local API.
• API keys stored in a sealed keystore, distributed per-service, never hardcoded.
• Rotation and revocation must be possible.
11. Backups + Recovery Drill
• Scheduled snapshots of the sealed configuration/state.
• A documented and tested recovery procedure.

Required Skills & Qualifications
Must-have (demonstrable experience):
• Advanced Linux systems administration (Ubuntu/Debian), systemd, permissions/ACLs, hardening.
• Bash and Python scripting (production quality, not throwaway).
• Network security: iptables/nftables, air-gapping, link-local networking, firewall rule design.
• Redis administration (Unix sockets, ACLs, persistence).
• Qdrant or comparable vector database deployment and security.
• RAG architecture — chunking, namespaced retrieval, trust/class segregation.
• LLM serving locally (llama.cpp / llama-server / GGUF) and safe integration.
• Prompt-injection defence and content-sanitization techniques.
• Secret/key management and least-privilege identity design.
• Tamper-evident / hash-chained audit logging.
Strongly preferred:
• Prior work on air-gapped or high-security environments.
• Experience with fail-closed security design (deny by default).
• Familiarity with financial-document handling or compliance-sensitive systems.
• Comfortable working from a detailed spec and asking precise clarifying questions.
This role is NOT suitable for general web/app developers without deep Linux security and infrastructure experience.

Deliverables
1. Fully configured and hardened core + gateway machines, deployed on our hardware.
2. All services running under correct identities with verified access boundaries.
3. Working ingestion → quarantine → trusted-store → retrieval → LLM pipeline.
4. Verified air-gap (denial tests proving non-authorized identities cannot reach network).
5. Hash-chained audit ledger with passing integrity check.
6. Backup + tested recovery procedure.
7. Runbook / handover documentation covering setup, operations, and incident response.
8. Knowledge-transfer session with our team.

Engagement Terms
• Location: Work must be delivered on-site at Ashok Vihar, Delhi, India. Remote-only candidates will not be considered. Hardware is provided by us.
• Payment: Milestone-based, with final payment released only on successful deployment and acceptance testing. A proposed milestone split is welcome in your bid.
• NDA required before sharing detailed architecture, credentials, or design documents.
• Timeline: Please state your estimated duration in the proposal.

How to Apply
In your proposal, please include:
1. Relevant experience — specifically air-gapped/security-hardened Linux deployments, RAG/vector-DB setups, and LLM integration. Links or descriptions of comparable work.
2. Your approach to enforcing a true air-gap and least-privilege identity model (high level).
3. Confirmation of on-site availability in Delhi and your earliest start date.
4. Proposed milestone plan and total bid.
5. Any clarifying questions you have.
Proposals that read as generic/template bids will be ignored. We are looking for a specialist who understands security-first infrastructure.