Principal Architect - Databricks

What success looks like in this role: Databricks Platform Architecture Architect enterprise Databricks Lakehouse platforms on Azure, AWS, or GCP — including Unity Catalog, Delta Live Tables (DLT), and Databricks SQL. Design multi-workspace, multi-region Databricks deployments with workspace federation, network isolation (Private Link / V Net injection), and identity federation via Azure AD / Okta / SCIM. Define medallion architecture (Bronze / Silver / Gold) standards, enforcing schema evolution, data contracts, and SLA-tiered pipeline SLOs. Lead migration of legacy data warehouses (Teradata, Netezza, Snowflake, Synapse) to the Databricks Lakehouse — including SQL translation, workload profiling, and TCO modeling. Own cluster architecture decisions: autoscaling policies, instance fleet configurations, job vs. interactive cluster strategies, Photon engine enablement, and cost-per-query optimization. Design and implement Databricks Workflows and Delta Live Tables for mission-critical streaming and batch pipelines, including CDC (Change Data Capture) patterns using Autoloader and Structured Streaming. Data Governance & Security Implement Unity Catalog as the enterprise meta store: fine-grained access control (row/column-level security), lineage tracking, and cross-workspace catalog federation. Define data classification frameworks, PII masking strategies, and dynamic views for regulatory compliance (GDPR, HIPAA, SOX) within the Databricks platform. Architect Delta Sharing for secure, governed cross-organizational data sharing without data movement. Lead data mesh and data product design patterns on Databricks — aligning ownership, SLAs, and discoverability across domains. AI & ML Platform (Databricks ML) Design end-to-end ML platforms using Databricks ML Runtime, ML flow (Tracking, Registry, Projects), and Feature Store for model lifecycle management. Architect LLM / Generative AI workloads on Databricks: RAG pipelines, fine-tuning with Mosaic AI, vector search (Databricks Vector Search), and Model Serving endpoints. Define MLOps frameworks covering CI/CD for models, drift detection, A/B testing, and shadow deployments using ML flow and Databricks Workflows. Integrate Databricks AI/BI (Genie, Dashboards) for self-service analytics and natural language data querying. Stakeholder & Delivery Leadership Lead architecture reviews, technical workshops, and proof-of-concept engagements with C-suite and VP-level client stakeholders. Author solution architecture documents, reference architectures, and RFP/RFI responses for Databricks-led pursuits. Define center-of-excellence (CoE) standards, reusable accelerators, and Databricks best-practice playbooks for the Unisys Data & AI practice. Mentor and upskill a team of data architects and engineers; drive Databricks certification paths across the practice. Partner with Databricks field engineering on joint go-to-market opportunities and co-delivery engagements. #LI-SS1