Data Engineer Job in Hyderabad at Norstella

Norstella is hiring for the role of Data Engineer!

Responsibilities of the Candidate:

• Design and Build Data Pipelines: Create and maintain optimal data pipeline architecture for extracting, transforming, and loading (ETL) data from various sources.
• Data Management: Assemble large, complex data sets that meet functional and non-functional business requirements.
• Data Processing: Process incoming files by matching databases, creating resulting execution files, monitoring and reporting file contents, and posting files in required locations and alerting team members to facilitate campaign delivery. These file processing tasks are weekly or monthly. Weekly file processing tasks must be performed on Tuesday-Wednesday each week during strict time periods (between 6am-10am Eastern Standard Time) and require candidate to be online/available from 8am-12pm EST each Tues-Wed.
• Process Improvement: Identify, design, and implement internal process improvements, such as automating manual processes and optimizing data delivery.
• Collaboration: Work with stakeholders, including data scientists, analysts, and executives, to support their data infrastructure needs and assist with data-related technical issues.
• Data Quality: Ensure data quality and reliability by developing and testing architectures that enable data extraction and transformation for predictive or prescriptive modeling.

Requirements:

• Technical Proficiency: Experience with data pipeline and workflow management tools, such as Apache Airflow, AWS Glue, or similar.
• Programming Skills: Proficiency in programming languages like SQL, Python, Java, or Scala.
• Database Management: Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Big Data Tools: Familiarity with big data tools like Hadoop, Spark, or Kafka.
• Analytical Skills: Strong analytical skills with the ability to combine data from different sources and interpret trends and patterns.
• Degree: A bachelor’s degree in computer science, Engineering or a related field.
• Experience: Previous experience as a data engineer ideally in the digital advertising industry.
• This role is crucial for transforming raw data into client reporting and actionable insights.