Test Automation Senior Specialist (AI Testing) Job in Bangalore at Nasdaq

Nasdaq is hiring for the role of Test Automation Senior Specialist (AI Testing)!

Responsibilities of the Candidate:

• Develop and maintain comprehensive automated test suites for generative AI models, including language models, and multimodal systems.
• Create innovative testing strategies to validate the quality, coherence, and safety of AI-generated content.
• Design and implement automation frameworks to test generative AI products across multiple platforms and use cases.
• Collaborate closely with AI researchers, data scientists, and product developers to understand product requirements and unique testing challenges of generative systems.
•  Develop tools and scripts to generate diverse test data sets and simulate real-world generative AI scenarios.
• Implement automated tests for prompt engineering, fine-tuning processes, and model versioning.
• Continuously improve testing methodologies to keep pace with rapidly evolving generative AI technologies.
• Analyze test results and provide detailed reports on model performance, potential biases, and areas for improvement.

Requirements:

• Bachelor's or Master's degree in Computer Science, AI/ML, or related technical field.
• Minimum 5+ years of experience in software test automation, with at least 2 years focused on AI/ML systems, preferably generative AI.
• Strong programming skills in Python.
• Expertise in test automation tools and frameworks suitable for AI testing tools/frameworks like RAGAS, Deep eval, TrueLens, AWS Bedrock Evaluator, pytest, Robot Framework.
• Hands-on experience/knowledge of AI frameworks like Langchain or LlamaIndex.
• In-depth understanding of natural language processing, computer vision, and generative AI concepts.
• Experience with large language models (e.g., GPT family, BERT).
• Familiarity with prompt engineering and techniques for evaluating generative AI outputs.
• Knowledge of ethical AI principles and experience in testing for bias and fairness in AI systems.