Middle / Senior Big Data Engineer (AWS)
Godel is on the constant lookout for talented Big Data Engineer to join one of our teams engaged in the development of business applications for major British companies. We always have new and exciting projects with opportunities to improve your skills in a wide spectrum of technologies related to web development. If you are enthusiastic and ambitious developer, willing to take active participation in shaping of a product, this job is for you! These are just some of the technologies that you might work with at Godel Data Division:- Data Integration: Apache Airflow (MWAA)
- Data Processing: Apache Kafka (MSK), Apache Spark (EMR), Amazon Kinesis, AWS Batch
- Database & Data Warehouse: Amazon DynamoDB, Amazon Redshift, Amazon Athena
- Storage: Amazon S3, Amazon Glue
- Design and development of data platform (data lake) based on cloud services from AWS
- Developing batch & stream data processing solutions, ETL processes, automated workflows
- Testing data pipelines and data quality using modern approaches
- Implementing industry’s best practices for collecting, transforming and accessing data
- Communication with the customer and working directly with the Customer team
- Working within an agile team
- Talent Management
- Talent Acquisition
- Education
- Innovations
- Onboarding
- Consultancy
- Understanding of Cloud platforms such as AWS and desire to work with the following services: Amazon S3, EC2, ECR, AWS Batch, Amazon DynamoDB, Amazon Athena, Amazon Redshift
- Experience of working with data pipelines: ETL/ELT, Data Quality and Data Management. Preferable technologies like: Apache Airflow, Apache Kafka, Apache Spark
- Knowledge in at least one of the programming languages Python, Java
- Intermediate level of English
- Exposure to operation of business-critical data products
- CI/CD: Git, Terraform, Jenkins, Test driven approach (TDD/BDD)
- Metrics and Notifications: Prometheus, Grafana
- Enthusiasm for agile and lean development