Responsibilities

· Hands-on expertise with ETL pipelines using internal framework written in Apache Spark & Java

· Acquire data from primary or secondary data sources and maintain databases/data systems

· Identify, analyze, and interpret trends or patterns in complex data sets

· Manage data ingestion in real time Kafka/REST APIs for streaming data from Graph using Spark

· Develop and implement databases, data collection systems, data analytics and other strategies that optimize data efficiency and quality

· Filter and clean data by reviewing reports and performance indicators to locate and correct problems

Required Technical Skills:

· Strong hands-on development experience with both Spark, Kafka

· Strong Java experience particularly within the data space

· Experience in Big Data related technologies like AWS, Hadoop, HIVE, Presto

· Experience implementing complex ETL pipelines with Spark toolset.

· Experience in Graph

· Experience working in an Agile Team environment.

Preferred Skills

· Kafka, NoSQL, Scala, MongoDB

· Experience with large data volumes, processing, transforming and transporting large-scale data