Data Engineer SME

Seeking a Subject Matter Expert to help accelerate our deployment of Raw and Derived data products to AWS data lakes. Candidates should be experts in the field and be able to illustrate experience with building and publishing the data structures we are starting to produce to help our techs overcome obstacles and avoid pitfalls. They should also be capable of helping us accelerate our rate of production through optimization and automation using Terraform Enterprise scripts and optimized AWS, Apache, and other tool configurations and architectural design. And be experienced and flexible with changing demands working in an Agile development environment. Specifically we are looking for individuals who can show they have at least 5+ year of experience in Data Engineering role and be in a position to be a source of knowledge for our existing data engineers.

Experience with below tech is must have:

• AWS Tech Stack :

o AWS Lakeformation

o AWS EMR

o Apache Hudi

o Flink

o AWS Glue (Glue Catalog, Glue ETL and Crawler)

o AWS Athena

o Redshift,

o Sagemaker/ML

o AWS Lambda

o DynamoDB

o RDS (Relational Database Services).

• AWS S3: Strong foundational concepts like object data store vs block data store, encryption/decryption, storage tiers etc.

• Implementation and tuning experience in Streaming use cased and Big Data Ecosystem (such as EMR, Hadoop, Spark, Hudi, Kafka/Kinesis etc.)

• Should be able to build scalable data infrastructure and understand distributed systems concepts from a data storage and compute perspective.

• Have a good understanding of Data Lake concepts and Data Warehouse concepts. Should be familiar with concepts on Modern Data Architecture.

• Ability to define Standards and guidelines with understanding of various Compliance and Auditing needs.

• Be proficient in Python or Java to handle large volume data processing.

• Terraform Enterprise.

• Kafka

• Qlik Replicate

Additional helpful experience would include:

• AWS Services CloudTrail, SNS, SQS, CloudWatch, Step Functions

• Experience with Secrets Management Platform like Vault and AWS Secrets manager

• Experience with DevOps pipeline (CI/CD) - Bitbucket; Concourse

• Experience with RDBMS platforms and Strong proficiency with SQL

• Knowledge of IAM roles and Policies

• Experience with Event Driven Architecture

• Experience with native AWS technologies for data and analytics such as Kinesis, OpenSearch

• Databases - Document DB, Mongo DB

• Hadoop platform (Hive; HBase; Druid)

• Java, Scala, Node JS

• Workflow Automation

• Experience transitioning on premise big data platforms into cloud-based platforms such as AWS

• Background in Kubernetes, Distributed Systems, Microservice architecture and containers

• Experience with Rest APIs and API gateway

• Deep understanding of networking DNS, TCP/IP and VPN

• Kafka Schema Registry

• Confluent Avro

• Linux and Bash Scripting

Core Responsibilities

• Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery

• Lead the Design, Build, Test and Deployment of components in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead)

• Understand requirements / use case to outline technical scope and lead delivery of technical solution

• Works closely with the Product Owner to align on delivery goals and timing

• Collaborates with Data and Solution architects on key technical decisions

• Lead design and implementation of data quality check methods

• Ensures Test Driven development

• Strong skills with business stakeholder interactions

• Ensure data security and permissions solutions, including data encryption, user access controls and logging

Lorien, Powered By Impellam, is a digital, transformation, and technology recruitment specialist. We work with clients of all sectors, sizes, and tech needs – from digital giants to tech-enabled businesses – to attract and retain top talent around the

world. Through our network of specialists, we combine deep tech sector expertise with the scope, depth, and breadth of large-scale talent solutions, giving our clients and candidates the confidence and the agility to compete in the digital age.

• ED&I: Lorien, powered by Impellam Group, is an equal opportunity employer committed to diversity and inclusion.

• EEO: Impellam Group is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, gender, national origin, age, disability, veteran status, or any other factor determined to be unlawful

under applicable law.

• Accommodation: Impellam Group is an Equal Opportunity Employer. Impellam Group recruits, employs, trains, compensates, and promotes regardless of race, religion, color, national origin, gender identity, sexual orientation, physical ability, age,

veteran status, and other protected status as required by applicable law. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application, interview process, preemployment activity, and

the performance of crucial job functions. Please contact HR-InfoImpellamNA@impellam.com to request accommodation.