Senior AI Engineer

Responsibilities

• Lead the end-to-end design, build, and operation of enterprise AI systems on AI Factory platforms.
• Engineer and tune LLM inference serving stacks, specifically focusing on vLLM for latency, throughput, and cost optimization.
• Architect and operate MLOps pipelines covering model lifecycle, deployment, and observability.
• Design and engineer RAG applications using vector databases, including chunking, retrieval tuning, and reranking.
• Build and maintain LLM evaluation harnesses, including golden sets and regression suites.
• Operate Kubernetes clusters and high-performance storage/networking for AI workloads.
• Engage directly with client stakeholders to communicate technical status and strategic recommendations.
• Mentor junior engineers and contribute reusable patterns and tooling back to the practice.

Requirements

• 7+ years of software, data, or infrastructure engineering experience.
• 3+ years of experience specifically working with modern AI and LLM systems.
• Production-quality Python skills, including testing and version control fluency.
• Deep production Linux experience, including system internals and performance tuning.
• Proficiency with Docker for image management and runtime tuning.
• Hands-on experience with vLLM and vector databases.
• Practical experience with MLOps tooling and patterns.
• Strong client-facing communication skills and ability to lead technical workstreams.

Preferred Qualifications

• Experience with GPU operations, including CUDA toolchains and NVIDIA AI Enterprise.
• Familiarity with Ray for distributed training and inference scaling.
• Knowledge of Kubernetes administration (Helm, RBAC, ingress).
• Experience with fine-tuning workflows such as LoRA, QLoRA, or PEFT.
• Infrastructure as Code experience using Terraform or Ansible.
• Relevant certifications such as CKA, CKAD, or NVIDIA-Certified Associate.

About the Company

BlueAlly is a leading provider of IT services and solutions, helping organizations conquer IT complexity across cloud, cybersecurity, infrastructure, data, and application modernization. Headquartered in Cary, North Carolina, BlueAlly delivers across the full technology lifecycle—from strategy and design through implementation and managed services.