Data Engineer Job in Bangalore at Schneider Electric

Schneider Electric is hiring for the role of Data Engineer!

Responsibilities of the Candidate:

• Create and maintain optimal data pipeline architecture; assemble data sets that meet functional / non-functional requirements.
• Design the right schema to support the functional requirement and consumption pattern.
• Design and build production data pipelines from ingestion to consumption.
• Build the necessary datamarts, data warehouse required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
• Create necessary preprocessing and postprocessing for various forms of data for training/ retraining and inference ingestions as required
• Create data visualization and business intelligence tools for stakeholders and data scientists for necessary business/ solution insights
• Identify, design, and implement internal process improvements: automating manual data processes, optimizing data delivery, etc.

Requirements:

• Programming experience python programming, spark is must.
• Hands -on experience in SQL, writing analytical queries and windows functions.
• Hands – on experience in creating external tables, partitioning, parquet files.
• 2-3 years of solid experience in Big Data technologies a must.
• Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift)
• Knowledge of Python and Pyspark is an absolute must.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift, S3, Athena and familiarity with various log formats from AWS.
• Experience in AWS Glue ETL, AWS Crawler, AWS Lambda, Glue Data Catalog, AWS Glue Studio.
• Hands on experience with python programming, spark, shell scripting
• Knowledge of Database Concepts – Indexing, Partitioning is must.
• Knowledge of Data warehousing – Normalization, Denormalization, Star/Snow-flake schemas.
• Good Hands -on the table’s creations, DDL, DML and TCL
• Knowledge of Database Concepts – Indexing, Partitioning
• Knowledge of Data warehousing – Normalization, Denormalization, Star/Snow-flake schemas