AI Platform Engineer I Job in Pune at Rapid7

Rapid7 is hiring for the role of AI Platform Engineer I!

Responsibilities of the Candidate:

• Design, test, and refine prompts for general and use-case-specific applications.
• Evaluate and improve LLM output quality based on accuracy, latency, hallucination, and user satisfaction.
• Maintain a scalable prompt library with robust documentation and experimentation tracking.
• Build and maintain custom GPTs (via OpenAI) or Google Agents (Gemini, Vertex AI).
• Configure tools, memory, system prompts, and API-calling capabilities for agent workflows.
• Ensure agents follow internal data governance and compliance policies.
  Integrate LLM functionality into internal systems (e.g., dashboards, Slack, CRMs, internal portals).
• Work with APIs from OpenAI, Google Cloud, Azure, and others to create functional pipelines.
• Support rapid prototyping and deployment of AI features in close coordination with product and automation teams.
• Define key performance metrics (quality, reliability, usage patterns).
• Collaborate with data engineering to log, analyze, and visualize LLM performance.
• Implement feedback loops to continuously optimize prompt and agent behavior.
• Provide tooling and guidance to business users and product teams for prompt-based workflows.
• Contribute to internal AI playbooks and best practices around prompt usage and API integration.

Requirements:

• 1 -3 years of experience in AI/ML, NLP applications, or full-stack product development
• Hands-on experience with OpenAI APIs, Google Cloud Vertex/Gemini, or similar platforms
• Strong experience with prompt engineering and LLM evaluation methods
• Proficiency in Python and working with APIs, JSON, and vector databases
• Familiarity with LangChain, LlamaIndex, or custom agent frameworks
• Excellent collaboration and communication skills