Artera Director of Data Engineering in Atlanta, Georgia
Director of Data Engineering
Title:Director of Data Engineering
More about this job >
Director of Data Engineering This role, reporting to Chief Data Officer will drive the development of data pipelines, warehouses and marts for key subject areas. This role will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. In depth understanding of Cloud environments: Storage, IaaS, PaaS, SaaS, Compute. The Data Engineer will support our analytics engineers, architects, and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our innovative data initiatives Duties and Responsibilities In addition to creating and maintaining an optimal pipeline architecture, typical duties and responsibilities for a Data Engineer position may include:
Evangelize and establish best practices, frameworks, and processes across data engineering and analytics.
Oversee technical strategy and architecture across data integration data lake, data warehousing and data access layers.
Lead metadata management implementation efforts aligning with Data Governance
Uphold, introduce, and refine data quality standards, engineering best practices, and KPIs
Drive the design and implementation of data privacy and security related processes across data lake, warehouses, and Business Intelligence layers.
Develop strong relationships with analytics leaders across the system to improve how data is leveraged.
Lead and manage the engagement with technical partners including offshore, consulting and technology partners.
Evaluate business needs and objectives
Conduct complex data analysis and report on results
Prepare data for prescriptive and predictive modeling
Combine raw information from different sources. Able to Synthesize data and its impact on business
Identify opportunities for data acquisition
Optimize the azure services for different workloads (batch, streaming, ad-hoc queries, ETL, ML etc.) and cost.
Manage own learning and contribute to technical skill building of the organization.
Inspire and cultivate the engineering mindset and systems thinking within the team.
Ensure conceptual and architectural integrity of the platform and solution.
Challenge existing structures and thought processes
Roll up sleeves and be directly involved in solving challenging technical problems
Constantly strive to improve engineering workflows and efficiency, and heavily utilize automation whenever applicable
Work closely with DevOps to optimize cloud infrastructure performance and scalability
10+ years of experience building, leading and inspiring effective data engineering teams building scalable data and analytics solutions.
5+ years of hands-on experience in modern data architecture and data modeling.
Proven experience in developing data solutions in Cloud (Azure preferred)
Proven experience in automating data life cycle management, metadata management and master data management (MDM).
Experience implementing perimeter security, data access policies and privacy controls.
Good understanding of distributed computing and big data architectures.
Personal qualities such as creativity, tenacity, curiosity, and passion for deep technical excellence.
Good interpersonal skills; comfort and competence in dealing with different teams within the organization and subsidiaries.
Requires an ability to interface with multiple constituent groups and build sustainable relationships.
Strong and effective communication skills (verbal and written).
Strong analytical, problem-solving skills.
Experience of working in a matrix organization.
Proactive problem solver.
Ability to prioritize and deliver.
Results-oriented, flexible, adaptable.
Work well independently and lead a remote team.
Versatile, creative temperament, ability to think out-of-the box while defining sound and practical solutions.
Ability to master new skills.
Familiar with Agile practices and methodologies
Professional data engineering experience focused on batch and real time data pipelines using Spark, PySpark, Python, SQL
Data warehouse (data modeling, programming), RDBMS exposure in Microsoft technologies like SSIS, SQL Server, SSRS
Understanding of data science and visualization technologies
Experience working on a cloud environment, preferably, Microsoft Azure Cloud Data Warehouse solutions
Previous experience as a data engineer or in a similar role
Technical expertise with data models, data mining, and segmentation techniques
Knowledge of programming languages (e.g. Java and Python)
Hands-on experience with SQL database design
Data engineering certification is a plus