Senior Data Engineer

 

Description:

Main Skills: Python, API Developement, Microservices, Kubernetes, real-time messaging with Kafka

 

Required Qualifications

  • Strong in Python hands-on coding experience
  • Experience designing and building Microservices
  • Experience with REST API design and development using Python
  • Experience with deployment/scaling of apps on containerized environment (i.e. Kubernetes, AKS)
  • 3+ years experience with real-time and streaming technology (i.e. Azure Event Hubs, Azure Functions Kafka, Spark Streaming)
  • 3+ years experience with cloud-based platforms (i.e. Azure, GPC, AWS)
  • Experience building automated big data pipelines
  • Experience performing data analysis and data exploration
  • Experience working in an agile delivery environment
  • Experience with Snowflake, RDBMS and NoSQL Databases and hands-on query tuning/optimization
  • Experience working in multi-developer environment, using version control (i.e. Git)
  • Experience with orchestrating pipelines using tools (i.e. Airflow, Azure Data Factory)
  • Experience with data modeling and system architecture design
  • Experience partnering cross-functionally with other technical teams (i.e. data ingestion, data science, operational systems) to align priorities and achieve deliverable outcomes
  • Experience with setting coding standards, performing code reviews, and mentoring junior developers
  • Exposure/understanding DevOps best practice and CICD (i.e. Jenkins)
  • Ability to understand complex systems and solve challenging analytical problems
  • Strong critical thinking, communication, and problem solving skills and a quick learner

 

Preferred Qualifications

  • Experience in Azure/GCP.
  • Solid understanding of Kafka architecture and building consumers for high volume data streams.
  • Experience with bash shell scripts, UNIX utilities & UNIX Commands
  • Previous healthcare experience and domain knowledge

 

Education

  • Bachelor’s Degree in Computer Science, Engineering, Statistics, Physics, Math, or related field or equivalent experience
  • Preferred: Master’s Degree with coursework focused on advanced algorithms, mathematics in computing, data structures, etc.

Organization Xsell Resources
Industry IT / Telecom / Software Jobs
Occupational Category Senior Data Engineer
Job Location New York,USA
Shift Type Morning
Job Type Full Time
Gender No Preference
Career Level Intermediate
Experience 2 Years
Posted at 2024-02-01 8:16 pm
Expires on 2024-12-05