Sr Azure Data Engineer Job at Zapcom, Dallas, TX

SzYvaUZxQUttU2lJblVUMkljUXpLenFLZHc9PQ==
  • Zapcom
  • Dallas, TX

Job Description

  • Design and implement data ingestion, transformation, and movement using Azure Data Factory (ADF), Azure Synapse, and Data Lake.
  • Collaborate with business stakeholders, data scientists, and engineers to build robust data solutions.
  • Develop and manage high-volume, real-time, and batch data ingestion pipelines using Azure Data Factory (ADF).
  • Implement event-driven architectures for real-time data movement and processing.
  • Develop large-scale data processing solutions using Azure Databricks and Apache Spark with PySpark, Scala, or Python.
  • Optimize data partitioning, caching, and indexing for efficient performance.
  • Manage complex transformations and aggregations for structured and unstructured datasets.
  • Design and implement high-performance data models in Azure Synapse Analytics using dedicated SQL Pools and Spark Pools.
  • Optimize query performance, workload management, and cost efficiency in Synapse Analytics.
  • Implement column store indexes, partitioning strategies, and data caching to enhance performance.
  • Design and manage secure, scalable data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2).
  • Implement RBAC (Role-Based Access Control), encryption, and data masking to ensure security.
  • Implement Azure Monitor, Log Analytics, and Application Insights for data pipeline monitoring and troubleshooting.
  • Optimize cost management, auto-scaling, and performance tuning across Azure services.

Requirements

  • 8+ years of experience in data engineering, ETL development, and cloud-based data integration,
  • Strong experience with Azure Data Factory (ADF): ETL/ELT pipeline orchestration and data movement.
  • Azure Databricks: Large-scale data transformation and big data processing with Apache Spark, PySpark, Scala, or Python.
  • Azure Synapse Analytics: Data warehousing, SQL Pools, Spark Pools, and performance tuning.
  • Azure Data Lake Storage Gen2 (ADLS Gen2): Secure, scalable data lake architecture.
  • SQL Server: Advanced T-SQL, stored procedures, indexing, and query optimization.
  • Power BI, Tableau, or other BI tools for data visualization.
  • Financial/Banking experience.

Job Tags

Full time,

Similar Jobs

Novo Holdings

Public Equity Senior Associate, Venture Investments Job at Novo Holdings

 ...responsible for originating, analyzing and managing our public equity investments. Our public investing activities implement a long-only...  ...transactions. As the public equities team works alongside the private equity team, the role will also provide exposure to the firm... 

Interlingva, Inc.

Portuguese Interpreter (Sacramento Area) Job at Interlingva, Inc.

 ...qualifications. QUALIFICATIONS Fluency in both Portuguese and English Experience performing interpreting work...  ...language Ability to interact with and relate to non-English speaking persons of any background and culture Ability to effectively... 

Direct Demo LLC

Flemington Costco Sales Rep - $23-$25/hr + COMMISSION! Job at Direct Demo LLC

 ...We provide a promotional kit and bi-weekly paycheck via direct deposit Bonus payout : We have various different products in Costco: Super Greens, Liquid Collagen, CoQ10, Turmeric, & Magnesium Gummies. This week we have 3 products on sale. Below, is an outline of... 

Fox Television Stations

Photographer Job at Fox Television Stations

Overview Of The Company Fox TV Stations FOX Television Stations owns and operates 29 full power broadcast television stations in the U.S. These include stations located in 14 of the top 15 largest designated market areas, or DMAs, and duopolies in 11 DMAs, including...

Ourisman Chrysler Jeep Dodge Of Alexandria

Sales Consultant Job at Ourisman Chrysler Jeep Dodge Of Alexandria

 ...oriented. Come join the #1 Team in the business. No experience necessary. Individuals with prior experience working in retail, restaurants, and hospitality are encouraged to apply. We provide paid training and ongoing mentorship and support. We make an investment in you...