Kashif Sohail — Data Engineer

Summary

§ 01

Profile

Data Engineer with a decade of experience spanning distributed data processing, cloud infrastructure, and production support for enterprise-scale systems. Currently at LTIMindtree, embedded with Microsoft's Azure messaging and data platform teams, building config-driven ingestion frameworks on Databricks Auto Loader and Spark, and designing Medallion Architecture pipelines on Delta Lake for reliable, traceable transformation at scale.

Work sits at the intersection of engineering and operations: implementing Unity Catalog governance with fine-grained row- and column-level security, tuning Spark jobs for performance, and resolving high-severity production incidents across Service Bus, Event Hub, Event Grid, and ADLS Gen2 under strict SLAs. Earlier experience with Hadoop, Hive, and AWS S3 built the distributed-systems foundation this now runs on.

Grounded in Python, PySpark, Scala, and SQL, certified in AWS Solutions Architecture and Azure cloud technologies, with an MS in Data Science from the University of Cumberlands.

Experience

§ 02

Azure Cloud & Infrastructure Engineer

LTIMindtree (Microsoft)

JAN 2023 — PRESENT

Improve system performance and reliability for Microsoft clients by resolving issues across Service Bus, Event Hub, Event Grid, and AI Search.
Built a config-driven ingestion framework using Azure Databricks Auto Loader and Spark for batch and real-time data from ADLS Gen2.
Designed Medallion Architecture (Bronze, Silver, Gold) on Delta Lake, improving data reliability and transformation efficiency.
Implemented Unity Catalog for centralized governance — fine-grained access control, schema enforcement, and lineage tracking.
Diagnosed latency, throttling, and auth issues via ARM logs, KQL in Azure Data Explorer, and ETW logs.
Managed high-pressure CritSit and Severity A cases, coordinating with product teams to maintain SLA compliance.

Azure Data Engineer

LTIMindtree (Microsoft)

JUN 2022 — DEC 2022

Developed custom ingestion pipelines using PySpark on Databricks, loading data from APIs, Azure SQL, and Blob Storage into ADLS Gen2.
Orchestrated end-to-end workflows using Azure Data Factory and Databricks Workflows for reliable, automated execution.
Optimized Spark jobs and clusters, achieving ~30% improvement in processing time.
Integrated CI/CD via Azure DevOps to automate deployment and version control of data workflows.
Investigated customer issues using Kusto Language Queries and Microsoft telemetry tools under agreed SLAs.

Data Engineer

PACT Business Solutions, India

MAY 2016 — NOV 2019

Loaded and validated raw data into Spark DataFrames; converted RDDs for downstream processing.
Implemented Spark SQL logic to join multiple data frames for application-specific aggregated results.
Worked extensively with Hadoop (HDFS) and distributed processing; designed partitioning/bucketing for faster retrieval.
Imported data from AWS S3 into Spark RDDs; developed UDFs in Hive across Avro, Parquet, and JSON formats.
Delivered agreed user stories on time each sprint within a Scrum team.

Kashif
Sohail.

Summary

Experience

Toolkit

Certifications

AWS Solutions Architect — Associate

Azure Developer — Compute Specialist

Education