Inclusively is partnering with a multinational professional services network to hire a Project Delivery Specialist - Lead Databricks Engineer - Data Lakes - Remote. **Please note: this role is NOT an internal position with Inclusively but with the partner company.**

ABOUT INCLUSIVELY

Inclusively is a digital tech platform that empowers job seekers with disabilities, caregivers, and veterans by using Success Enablers–accommodations and personalized workplace modifications that help all job seekers reach their full potential and excel. This includes all disabilities under the ADA, including mental health conditions (e.g. anxiety, depression, PTSD), chronic illnesses (e.g. diabetes, Long COVID), and neurodivergence (e.g. autism, ADHD).

Create your profile, select Success Enablers, and connect to jobs from our partnered employers who are committed to creating diverse and inclusive teams. When registering, you must acknowledge that this platform is for people with disabilities, caregivers, and veterans. However, Inclusively does not require candidates to disclose their specific disability to join the platform.

Work you’ll do/Responsibilities

• Lead end-to-end delivery of data lake/lakehouse workloads on Databricks: ingestion, processing, curation, and consumption layers.
• Architect and implement pipelines using Spark (PySpark/Scala), Delta Lake, and orchestration tooling (e.g., Databricks Workflows, ADF/Airflow—as applicable).
• Define data modeling patterns (bronze/silver/gold, dimensional/denormalized, CDC handling) and optimize for analytics use cases.
• Establish engineering standards: repo structure, coding conventions, branching strategy, documentation, and reusable frameworks.
• Implement data quality and validation controls (reconciliation, anomaly detection, schema enforcement, expectations).
• Drive performance tuning: partitioning, file sizing, Z-ORDER, caching, cluster sizing, job parallelism, and query optimization.
• Partner with governance/security to implement access controls (e.g., Unity Catalog, RBAC/ABAC), PII handling, encryption, and audit logging. Build/maintain CI/CD (continuous integration/continuous delivery) for notebooks/jobs (Databricks Repos, Git integration; IaC where applicable).
• Monitor and improve operational reliability: observability dashboards, alerting, runbooks, incident response support.
• Manage dependencies across teams (source system owners, platform/infrastructure, BI, data science) and provide delivery estimates and plans.
• Mentor engineers, conduct code reviews, and lead technical design sessions.
• Communicate regularly with Engagement Managers (Directors), project team members, and representatives from various functional and / or technical teams, including escalating any matters that require additional attention and consideration from engagement management
• Independently and collaboratively lead client engagement workstreams focused on improvement, optimization, and transformation of processes including implementing leading practice workflows, addressing deficits in quality, and driving operational outcomes

Qualifications

Required:

• 7+ years experience with Apache Spark, Databricks SQL, Python, ETL, Lakeflow, Cloud Orchestration
• 7+ years in data engineering with strong hands-on Databricks experience in production environments.
• Advanced skills in Spark and PySpark/Scala, with strong SQL capabilities.
• Deep experience with Delta Lake and lakehouse/data lake design patterns.
• Experience building batch and/or streaming pipelines; familiarity with CDC concepts.
• Proven technical leadership (leading squads, setting standards, design ownership).
• Experience with cloud storage and security fundamentals (object storage, networking basics, IAM).Bachelor's degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline; or equivalent experience
• Limited immigration sponsorship may be available
• Ability to travel 10%, on average, based on the work you do and the clients and industries/sectors you serve