Software Technology Inc. · 3 days ago
Lead AI Engineer
Austin, Texas Metropolitan Area
Contract
Hybrid
Lead/Staff
14+ years expSoftware Technology Inc. is seeking a Lead AI Engineer to modernize and enhance their existing ElasticSearch system. The role involves integrating advanced semantic search and LLM-powered ranking techniques to transform traditional search into an intelligent and personalized experience.
ConsultingInformation Technology
Hiring Manager
Ranadheer Reddy
Responsibilities
Analyze limitations in current regex & keyword-only search implementation on Elastic Search
Enhance search relevance using:
BM25 tuning
Synonyms, analyzers, custom tokenizers
Boosting strategies and scoring optimization
Introduce semantic / vector-based search using dense embeddings
Implement LLM-powered search workflows including:
Query rewriting and expansion
Embedding generation (OpenAI, Cohere, Sentence Transformers, etc.)
Hybrid retrieval (BM25 + vector search)
Re-ranking using cross-encoders or LLM evaluators
Build RAG (Retrieval Augmented Generation) flows using ElasticSearch vectors, OpenSearch, or AWS-native tools
Build and optimize search APIs for latency, relevance, and throughput
Design scalable pipelines for:
Indexing structured and unstructured text
Maintaining embedding stores
Real-time incremental updates
Implement caching, failover, and search monitoring dashboards
Deploy and operate solutions on AWS , leveraging:
OpenSearch Service or EC2-managed ElasticSearch
Lambda, ECS/EKS, API Gateway, SQS/SNS
SageMaker for embedding generation or re-ranking models
Implement CI/CD for search models and pipelines
Develop search evaluation metrics (nDCG, MRR, precision@k, recall)
Conduct A/B experiments to measure improvements
Tune ranking functions and hybrid search scoring
Partner with product teams to refine search behaviors with real usage patterns
Qualification
Required
14+ years exp Mandatory
5–10 years of experience in AI/ML, NLP, or IR systems, with hands-on search engineering
Strong expertise in ElasticSearch/OpenSearch: analyzers, mappings, scoring, BM25, aggregations, vectors
Experience with semantic search: Embeddings (BERT, SBERT, Llama, GPT-based, Cohere), Vector databases or ES vector fields, Approximate nearest neighbor (ANN) techniques
Working knowledge of LLM-based retrieval and RAG architectures
Proficient in Python; familiarity with Java/Scala is a plus
Hands-on AWS experience (OpenSearch, SageMaker, Lambda, ECS/EKS, EC2, S3, IAM)
Experience building and deploying APIs using FastAPI/Flask and containerizing with Docker
Familiar with typical IR metrics and search evaluation frameworks
Preferred
Knowledge of cross-encoder and bi-encoder architectures for re-ranking
Experience with query understanding, spell correction, autocorrect, and autocomplete features
Exposure to LLMOps / MLOps in search use cases
Understanding of multi-modal search (text + images) is a plus
Experience with knowledge graphs or metadata-aware search
Company
Software Technology Inc.
Software Technology, Inc.
201-500 employees
H1B Sponsorship
Software Technology Inc. has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (7)
2024 (8)
2023 (8)
2022 (11)
2021 (11)
2020 (15)
Funding
Current Stage
Growth StageCompany data provided by crunchbase