Oracle · 2 days ago
QA Engineer (Performance)- Dallas, TX
4835 Lyndon B Johnson fwy, Suite 540 Dallas, TX 75244, Dallas, Texas, US
Full-time
Onsite
Senior Level, Lead/Staff
$38K/yr - $133K/yr
8+ years expOracle is seeking a Performance QA Engineer to specialize in benchmarking and optimizing their Agentic AI platform. The role involves stress-testing the AI pipeline to ensure optimal performance and cost-efficiency in user experience.
Data GovernanceData ManagementEnterprise SoftwareInformation TechnologySaaSSoftware
Responsibilities
Latency Benchmarking: Measure and optimize TTFT (Time to First Token) and Total Request Latency for complex agentic workflows that involve multiple reasoning steps
Agentic Loop Stress Testing: Simulate high-concurrency environments to see how the system handles hundreds of autonomous agents running simultaneously, particularly focusing on API rate limits and GPU/compute bottlenecks
RAG Performance Analysis: Test the speed and efficiency of the vector database retrieval process. Identify how increasing the 'context window' size impacts overall system performance
Token Throughput Monitoring: Analyze the 'tokens per second' (TPS) metrics and identify when model-switching (e.g., from a large model to a smaller one) is necessary to maintain performance
Cost vs. Performance Optimization: Create reports that balance performance gains against token costs, helping the team find the 'sweet spot' for production-grade agents
Orchestration Bottleneck Identification: Use profiling tools to find delays in the 'hand-off' between different agents or between the agent and external tools (APIs, databases)
Automated Performance Regressions: Integrate performance testing into the CI/CD pipeline to ensure that new prompt versions or architectural changes don't degrade the agent's speed
Qualification
Required
8+ years in Performance Engineering, with a specific focus on AI/ML applications or high-concurrency distributed systems
Expert-level experience with performance testing tools like Locust, JMeter, or k6, specifically customized for Python-based AI backends
Strong ability to write custom scripts to simulate complex, multi-step user/agent interactions
Understanding of LLM-specific performance factors, such as quantization, KV caching, and the impact of different model architectures on latency
Experience with tools like Prometheus, Grafana, LangSmith, or Weights & Biases to monitor system health and AI-specific metrics
Experience testing the query latency of Vector Databases under heavy load
Benefits
Medical, vision, and dental benefits
401k retirement plan
Variable pay/incentives
Paid time off
Paid holidays
Company
Oracle
Oracle is an integrated cloud application and platform services that sells a range of enterprise information technology solutions.
10001+ employees
H1B Sponsorship
Oracle has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1271)
2024 (846)
2023 (995)
2022 (1192)
2021 (985)
2020 (755)
Funding
Current Stage
Public CompanyTotal Funding
$25.75BKey Investors
Sequoia Capital
2025-09-24Post Ipo Debt· $18B
2025-02-03Post Ipo Debt· $7.75B
1986-03-12IPO
Leadership Team
Esteban Rubens
Healthcare Field CTO
G
Gerard Warrens
Field CTO, Business Strategy and Transformative Technologies
Recent News
2026-01-10
2026-01-09
Business Insider
2026-01-09
Company data provided by crunchbase