Senior QA Automation Engineer (Python | Selenium | Pytest)
3Pillar Global
Job Description
<div>3Pillar is an AI transformation partner on a mission to help enterprises build the AI-native products and intelligent agents that will define the next era of business. With teams across North America, Europe, Latin America, and Asia, we work with the most ambitious companies in financial services, healthcare, media, and technology — helping them move faster, modernize boldly, and compete on their own terms. Our HelixAI platform and Helix Pods delivery model put our engineers at the center of real agentic transformation — doing work that is open, portable, and built to last. We are building the future of enterprise AI. </div><div><h3>Senior QA Automation Engineer </h3><p>You'll own test architecture and semantic validation for a mission-critical AI-powered system where <strong>consistency guarantees are non-negotiable</strong>. This isn't traditional QA; you're designing test harnesses that prove AI outputs are correct, not just crash-free.</p><p>Your core responsibility: build a golden-set validation harness that measures whether the system's LLM stage produces the right decisions at the right confidence levels. You'll design automated tests across unit, integration, E2E, and access-control layers. You'll also establish CI/CD quality gates that block bad builds—schema validation failures, cross-tenant isolation defects, missing floor controls, semantic inconsistencies—before they reach production.</p><p>This is front-loaded work (weeks 1–5), then you support the team through delivery. You'll mentor a junior test engineer and set the standard for test coverage in a compressed timeline.</p></div><h3>Desired Capabilities</h3><ul><div><li><h3>Leadership skills</h3></li><li><h3>Mentoring and coaching abilities</h3></li><li><p>Strategic thinking and decision-making skills</p></li><li><h3>Project management skills</h3></li><li><p>Stakeholder management and collaboration skills</p></li><li><p>Ability to motivate and inspire team members</p></li><li><h3>Continuous improvement mindset</h3></li><li><p>Familiarity with DevOps processes and tools.</p></li><li><p>Proficiency in programming languages such as Java, Python, or Ruby.</p></li></div></ul><h3>Key Responsibilities</h3><ul><div><li><p>Design comprehensive test architecture – plan unit, integration, E2E, and semantic validation test suites that cover the full pipeline from profile intake through roadmap generation; define coverage targets and acceptance criteria</p></li><li><p>Build semantic validation harnesses for AI outputs – design golden-set test cases (5–8 CISO-approved reference profiles) that measure AI stage precision/recall, rigor-tier correctness, floor-control invariants, traceability completeness, and run-to-run stability; a semantic regression blocks merge</p></li><li><p>Implement access-control testing – design automated probes at API and database layers that verify cross-tenant isolation; simulate compromised sessions and verify RLS policies enforce tenant boundaries</p></li><li><p>