Senior D & A Engineer – Azure Databricks

Experience

6+ years.

Mandatory skills

Databricks, PySpark, AWS, RDBMS, NoSQL, Data Lake

Qualification

Bachelor’s Degree

Project Role / Job Title

Senior D & A Engineer – Azure Databricks

Job Specification

1. Proficient in Databricks and PySpark.
2. Hands on experience in Data Lake.
3. Hands-on experience in data mart and data warehousing including variant schemas (Star, Snowflake).
4. 5+ years of experience with advanced queries, stored procedures, views, triggers, etc.
5. 5+ years of experience of performance tuning queries.
6. 5+ years of experience of both DDL and DML.
7. 5+ years of experience designing enterprise database systems using any RDBMS tools preferred.
8. Experience in Lakehouse architecture is plus.
9. Experience with Cloud technologies – AWS, Snowflake is preferred.
10. Experience with Streaming is a plus.
11. Experience with Kafka is a plus
12. Deep understanding of one or more source/version control systems. Develops branching and merging strategies.
13. Working understanding of Web API, REST, JSON.
14. Working understanding of unit testing creation

Job Description

1. Designs, develops, and delivers solutions that meet business line and enterprise requirements.
2. Participates in rapid prototyping and POC development efforts.
3. Advances overall enterprise technical architecture and implementation best practices.
4. Assists in efforts to develop and refine functional and non-functional requirements.
5. Participates in iteration and release planning.
6. Informs efforts to develop and refine functional and non-functional requirements.
7. Demonstrates knowledge of, adherence to, monitoring and responsibility for compliance with state and federal regulations and laws as they pertain to this position.
8. Strong ability to produce high-quality, properly functioning deliverables the first time.
9. Delivers work product according to established deadlines.
10. Estimates tasks with a level of granularity and accuracy commensurate with the information provided.
11. Works collaboratively in a small team.
12. Excels in a rapid iteration environment with short turnaround times.
13. Deals positively with high levels of uncertainty, ambiguity, and shifting priorities.
14. .Accepts a wide variety of tasks and pitches in wherever needed.
15. Constructively presents, discusses and debates alternatives.
16. Takes shared ownership of the product.
17. Adhere to ISMS policies and procedure.