W3Global ยท 1 week ago
Lead AI Engineer (Search Modernization)- Hybrid
Dallas, TX
Contract
Hybrid
Mid, Senior Level
$120K/yr - $170K/yr
5+ years expW3Global is seeking a Lead AI Engineer to modernize and enhance their existing ElasticSearch system. The role involves integrating advanced semantic search and LLM-powered ranking techniques to transform traditional search into a personalized and high-precision experience.
ConsultingRecruitingStaffing Agency
Responsibilities
Analyze limitations in current regex & keyword-only search implementation on ElasticSearch
Enhance search relevance using:
BM25 tuning
Synonyms, analyzers, custom tokenizers
Boosting strategies and scoring optimization
Introduce semantic / vector-based search using dense embeddings
LLM-Driven Search & RAG Integration
Implement LLM-powered search workflows including:
Query rewriting and expansion
Embedding generation (OpenAI, Cohere, Sentence Transformers, etc.)
Hybrid retrieval (BM25 + vector search)
Re-ranking using cross-encoders or LLM evaluators
Build RAG (Retrieval Augmented Generation) flows using ElasticSearch vectors, OpenSearch, or AWS-native tools
Build and optimize search APIs for latency, relevance, and throughput
Design scalable pipelines for:
Indexing structured and unstructured text
Maintaining embedding stores
Real-time incremental updates
Implement caching, failover, and search monitoring dashboards
Deploy and operate solutions on AWS, leveraging:
OpenSearch Service or EC2-managed ElasticSearch
Lambda, ECS/EKS, API Gateway, SQS/SNS
SageMaker for embedding generation or re-ranking models
Implement CI/CD for search models and pipelines
Develop search evaluation metrics (nDCG, MRR, precision@k, recall)
Conduct A/B experiments to measure improvements
Tune ranking functions and hybrid search scoring
Partner with product teams to refine search behaviors with real usage patterns
Qualification
Required
5-10 years of experience in AI/ML, NLP, or IR systems, with hands-on search engineering
Strong expertise in ElasticSearch/OpenSearch: analyzers, mappings, scoring, BM25, aggregations, vectors
Experience with semantic search: Embeddings (BERT, SBERT, Llama, GPT-based, Cohere), Vector databases or ES vector fields, Approximate nearest neighbor (ANN) techniques
Working knowledge of LLM-based retrieval and RAG architectures
Proficient in Python; familiarity with Java/Scala is a plus
Hands-on AWS experience (OpenSearch, SageMaker, Lambda, ECS/EKS, EC2, S3, IAM)
Experience building and deploying APIs using FastAPI/Flask and containerizing with Docker
Familiar with typical IR metrics and search evaluation frameworks
Preferred
Knowledge of cross-encoder and bi-encoder architectures for re-ranking
Experience with query understanding, spell correction, autocorrect, and autocomplete features
Exposure to LLMOps / MLOps in search use cases
Understanding of multi-modal search (text + images) is a plus
Experience with knowledge graphs or metadata-aware search
Benefits
Benefits
Company
W3Global
W3Global is a leading provider of end-to-end consulting services, empowering businesses to achieve their strategic goals and optimize their operations.
501-1000 employees
H1B Sponsorship
W3Global has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (14)
2024 (7)
2023 (12)
2022 (12)
2021 (13)
2020 (33)
Funding
Current Stage
Late StageLeadership Team
Sridhar Venkatesan
CEO
Company data provided by crunchbase