The Company is in search of a Lead Data Engineer with a focus on Databricks, Python, PySpark, and AWS. The successful candidate will be responsible for designing, building, and supporting scalable, production-grade data pipelines and cloud data solutions. This hands-on role demands strong technical ownership, problem-solving skills, and the application of engineering best practices across the data ecosystem. Key responsibilities include the development and maintenance of ETL/ELT, data ingestion, and transformation pipelines, as well as the optimization of data processing solutions. The role also involves working with large-scale structured and unstructured datasets, implementing data quality controls, and troubleshooting complex production issues. Applicants must have a minimum of 10 years' of experience in software engineering, data engineering, or related technology roles, with at least 8 years' of hands-on data engineering and DevOps experience. A Bachelor's degree in a relevant technical discipline is required, and a Master's degree is desirable. The ideal candidate will have a proven track record in enterprise-scale production data pipelines, strong proficiency in SQL, and experience with Apache Spark and distributed data processing. Desired skills include experience with Snowflake data engineering and integration, as well as knowledge of data governance, lineage, and quality. The role requires a professional with a strong background in financial services or other regulated environments, and the ability to work with cross-functional teams to deliver enterprise data solutions.

Hiring Company

State Street