What you’ll do

• Lead critical efforts designing and building our next generation Streaming Data Platform, leveraging PubSub, Finagle, Scala, Dataflow, and Google Cloud Platform.
• Build scalable Dataflow data pipelines leveraging Airflow scheduler/executor framework
• Design our data models for optimal storage and retrieval and to meet machine learning modeling needs.
• Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company
• Take an active role in the technical growth of other team members and the processes that support overall team growth
• Lead or participate in code and technical design reviews

What we are looking for:

• 5+ years of significant professional experience in server-side development written in Scala, Java, Go, or Python
• 5+ years of professional experience developing large-scale, high-volume services using Finagle, Akka, or gRPC to handle petabytes of data
• 5+ years of experience managing data pipelines and infra using Dataflow, Spark, Hadoop, Flink, and Airflow.
• Professional experience across the entire software development process, from development environment, to CI & CD including CircleCi, Jenkins, or TravisCi, to production deployment, product and application performance measurement
• Confident articulating complex technical content to peers and partners across the organization via Technical Design Documents.
• Strong ability to communicate
• Proven ability in mentorship of junior engineers.

What we would like to see:

• DevOps experience with Google Cloud Platform, AWS, or Azure
• Data quality experience especially using & improving internal tools to automatically detect issues
• Experience with Data Quality tools including Deequ or Apache Griffin

Experience building Scala based services in a production environment