Job Description

Our client is looking for a Databricks Architect for Long term project in Minneapolis, MN (Hybrid) below is the detailed requirements.

Job Title: Databricks Architect

Location: Minneapolis, MN (Hybrid 3 days onsite 2 days remote per week)

Duration: Long term contract

Experience Required:

• Candidate to build a Delta Lake. An understanding on Medallion architecture is important.
• Designed Batch and Real time data ingestion end to end.
• Data Lineage using Data Catalog maintained that and used the tool to discover data sources and targets;
• Designed Data Quality Strategies - identify CDE critical data elements, assigning data stewards from business side and coded the DQ metrics generation in curated zone/silver layer.
• Designed CDC/incremental loads, full load strategies using hash keys, key columns and merge designs.
• Designed Data Security PII/NPI, RBAC role based access control. Azure Data factory pipeline creation Azure DevOps.
• Designed using Azure Data factory pipeline and Databricks writing out to azure blob and then finally to snowflake as consumption were from Power BI /snowflake.
• Initially worked with client team and did extensive data analysis to create the model, data delivery mechanism, cloud storage, retention requirement, security, RBAC [role based access control].etc
• Streaming datasets/file- were captured using AUTOLOADER delta live tables into raw and designed to be updated into silver, gold.
• Provide guidance on Azure cost management strategies, including resource tagging, budgeting, and monitoring.
• Conduct Azure security assessments, vulnerability scanning, and threat modeling to enhance cloud security.
• Discussing potential projects and their parameters with clients, executives, and software developers.
• Planning out the blueprints for software projects, including defining the scope, allocating resources, setting deadlines, laying out communication strategies, and indicating tests and maintenance,
• Assembling and leading the project team.
• Participating in and supervising each stage of the project.
• Ensuring each project stays on schedule and adheres to the deadlines.
• Determining and overseeing consistent testing, evaluation, and troubleshooting of all products in all stages of completion.
• Tracking milestones, deliverables, and change requests.
• Serving as a liaison to communicate information regarding changes, milestones reached, and other pertinent information.
• Preparing project proposals and discussing potential projects with clients and stakeholders.
• Facilitating project initiation by defining project scope and requirements, and preparing the necessary documents and requirements Developing project plans and timelines to ensure the timely submission of project deliverables.
• Managing project budgets and resources to ensure the timely completion of milestones.
• Identifying and managing project risks.
• Facilitating team meetings and collaboration.
• Liaising for changes and negotiations with relevant stakeholders.
• Ensuring software quality standards are met and requirements are submitted within budget and on time.
• Closing the project and ensuring proper documentation"

Looking forward for your reply.

Job Tags

Similar Jobs