AWS Big Data Engineer/Developer - LT Contract
SUMMARY
We are seeking a Data Engineer to develop new data platform and associated tools in a cloud infrastructure. This is an SME level who can design, code and provide Big Data Warehouse solutions across multiple environments. This platform and tools will be used across the organization and are critical to the deployment and release of new products, features and devices.
RESPONSIBILITIES
• Development of end-to-end Big Data analytics platform, web applications and services
• Design, code and provide data flow and ingestion processes
• Solve for complex data integrations across multiple systems
• Design and execute strategies for real-time data analysis and decisioning
• Deal with large quantity of metrics, and batch and real time data that needs to be processed
• Monitoring and Administration of cloud-based infrastructure and services
REQUIREMENTS
• 5+ years’ experience in application development using Java/Python based technologies
• Mastery level of Apache Spark with Pyspark and Java
• Experience with the deployment and performance tuning of Spark applications
• Hands-on experience deploying and managing Spark application in AWS EMR
• Experience with real time processing of Terabytes of data
• Expertise in SQL based databases like MySQL, SQL, and Oracle etc.
• Advanced knowledge with Big Data Tools like Hadoop, Elastic Search.
• Experience with Open stack and/or Amazon Cloud services
• Extensive Linux (Ubuntu preferred) experience
