We are looking for an experienced and hands-on QA Lead with 7+ years of experience in software quality assurance. The ideal candidate must have strong expertise in manual testing, automation testing, Python, test automation frameworks, and AI-powered product testing.
The candidate should have experience working in a product-based technology company and be capable of owning the complete quality lifecycle—from requirement analysis and test planning to automation, release sign-off, AI evaluation, and production-quality monitoring.
Role: QA Lead
Requirement: 1
Location: Bangalore (Domlur | WFO 5 days)
Experience: 7+ years
Requirements
Responsibilities:
Own the overall quality strategy for the assigned products and engineering teams.
Lead manual and automation testing across web applications, mobile applications, APIs, backend services, AI features, and third-party integrations.
Design, develop, and maintain scalable automation frameworks using Python.
Create comprehensive test plans, test scenarios, test cases, and release-quality reports.
Perform functional, regression, integration, API, database, exploratory, and performance testing.
Define testing strategies for AI/ML and Generative AI features, including chatbots, recommendation systems, search, summarisation, classification, and content-generation workflows.
Validate AI-generated responses for accuracy, relevance, consistency, completeness, safety, and business-rule compliance.
Test AI systems for hallucinations, inappropriate responses, prompt injection, data leakage, bias, and edge cases.
Build automated evaluation frameworks and datasets for testing LLM and AI-powered features.
Test Retrieval-Augmented Generation (RAG) workflows, including document retrieval, context relevance, response grounding, and citation accuracy.
Validate AI model and third-party LLM API integrations for reliability, latency, error handling, rate limits, token usage, and cost.
Establish baseline quality metrics and regression suites for AI-generated outputs.
Review product requirements, prompts, workflows, and technical designs to identify gaps and risks early in the development lifecycle.
Define and track quality metrics such as defect leakage, automation coverage, regression effectiveness, release readiness, AI response accuracy, hallucination rate, and latency.
Work closely with Product Managers, Developers, DevOps, Data Scientists, and AI/ML Engineers.
Lead release validation, QA sign-off, production sanity testing, and post-release monitoring.
Analyse production defects, support root-cause analysis, and implement preventive measures.
Mentor QA engineers and promote a strong quality-first culture across Product and Engineering teams.
Must-Have Qualifications
6+ years of experience in software testing and quality assurance.
Strong hands-on expertise in both manual and automation testing.
Proficiency in Python for developing automation frameworks and test utilities.
Strong experience with tools and frameworks such as Pytest, Selenium, Playwright, Appium, or Robot Framework.
Experience in API testing using Postman, Python Requests, REST Assured, or similar tools.
Good knowledge of database testing and strong proficiency in SQL.
Strong understanding of testing methodologies, QA processes, SDLC, and STLC.
Experience with functional, integration, regression, system, exploratory, and end-to-end testing.
Experience integrating automated tests with CI/CD pipelines.
Hands-on experience with Git, Jenkins, GitHub Actions, Jira, or similar tools.
Experience working in a product-based company and testing customer-facing products at scale.
Understanding of AI/ML concepts and experience testing AI-powered or Generative AI features.
Understanding of LLM behaviour, including non-deterministic outputs, hallucinations, context limitations, and prompt sensitivity.
Ability to design test datasets, evaluation criteria, and quality metrics for AI-generated outputs.
Strong analytical, debugging, problem-solving, and risk-identification skills.
Good communication, stakeholder-management, and team-leadership capabilities.
Ability to take complete ownership of product quality and release sign-off.
Good to Have
Experience testing LLM-based applications, AI chatbots, RAG systems, recommendation engines, or semantic search.
Experience with AI evaluation and observability tools such as LangSmith, DeepEval, Ragas, Promptfoo, TruLens, or similar platforms.
Familiarity with models and APIs from OpenAI, Gemini, Claude, or open-source LLM platforms.
Knowledge of prompt engineering and automated prompt-regression testing.
Experience evaluating AI systems for responsible AI, privacy, security, fairness, and bias.
Experience with performance-testing tools such as JMeter, Locust, or k6.
Experience testing microservices, distributed systems, and event-driven architectures.
Exposure to cloud platforms such as GCP, AWS, or Azure.
Knowledge of Docker, Kubernetes, Kafka, or similar technologies.
Experience with monitoring tools such as Grafana, Kibana, or Datadog.
Experience in recruitment technology, marketplaces, SaaS, or other high-scale products.
Education
Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.
Ideal Candidate
The ideal candidate is a hands-on QA leader who combines strong technical expertise with product and AI-quality thinking. They should be comfortable writing automation code, performing detailed manual testing, evaluating AI-generated responses, challenging requirements, identifying customer-impacting risks, and guiding teams towards reliable, safe, scalable, and high-quality product delivery.
Founded in 2019, Apna.co is India’s largest professional networking platform dedicated to helping India’s rising workforce unlock unique professional networking and skilling opportunities. The app is currently serving in 75 cities across India.
Having raised $190+ million from marquee investors like Insight Partners, Tiger Global, Lightspeed India, Sequoia Capital India, Rocketship.vc, and Greenoaks Capital, Apna is on a mission to enable livelihoods for billions in India. With over 35 million users, present in 75 cities and counting, and over 2 million employers that trust the platform, India has a new destination to discover relevant opportunities.
