Data Engineer II Job in Bengaluru at Flipkart

• Build and execute data modeling projects across multiple tech stacks i.e. big data, MPP, OLAP using agile development techniques. Build and integrate robust data processing pipelines for enterprise-level business analytics.
• Challenge the status quo and propose innovative ways to process, model, and consume data when it comes to tech stack choices or design principles. As needed, assist other staff with reporting, debugging data accuracy issues, and other related functions.

Requirements:

• 3-5 years experience with a Bachelor's Degree in Computer Science, Engineering, Technology or a related field required. 2 to 3 years of relevant software development experience with sound skills in database modeling (relational, multi-dimensional) & optimization and data architecture - databases e.g. Vertica
• Good understanding of streaming technologies like Kafka, and Spark Streaming. Proficiency in one of the programming languages preferably Java, Scala, or Python. Good knowledge of Agile, SDLC/CICD practices and tools with a good understanding of distributed systems
• Experience with Enterprise Business Intelligence Platform/Data platform sizing, tuning, optimization, and system landscape integration in large-scale, enterprise deployments. 
• Must have proven experience with Hadoop, MapReduce, Hive, Spark, and Scala programming.  Must have in-depth knowledge of performance tuning/optimizing data processing jobs, and debugging time-consuming jobs.
• Proven experience in the development of conceptual, logical, and physical data models for Hadoop, relational, EDW (enterprise data warehouse), and OLAP database solutions.
• Experience working extensively in multi-petabyte DW environments. Experience in engineering large-scale systems in a product environment
• You should have good hands-on experience in designing, implementing, and operating stable, scalable, solutions to flow data from production systems into analytical data platforms (big data tech stack + MPP) and into end-user-facing applications for both real-time and batch-use cases
• You should be able to work with business customers in a fast-paced environment understanding the business requirements and implementing analytical solutions.
• You should have good experience in the design, creation, management, and business use of large datasets. Do high-level design with guidance; Functional modeling, and break-down of a module. Thinking platforms & reuse
• Strong engineering mindset - build automated monitoring, alerting, self self-healing (restartability/graceful failures) features while building the consumption pipelines. Translate business requirements into technical specifications (fact/dimension/filters/derivations/aggregations