Shiva Gupta

➡️ Phase 2: Data Engineer - Cloud Transformation and ELT Strategy

Focus: Cloud-Native ELT, Data Modeling, Governance, and Analytics (Snowflake, DBT, Azure Synapse).

• Experienced in the design of transformation pipelines using DBT directly within snowflake to enhance analytics and reporting capabilities, execute tests, and automate documentation.
• Designed and implemented a unified API management layer using Kong, streamlining connectivity between disparate cloud services and internal data sources while enforcing robust security protocols and monitoring.
• Executed data ingestion from diverse enterprise sources via REST/SOAP APIs, levearging Kong API gateway for secure authentication, rare limiting, and centralized traffic management to ensure high-availability extraction.
• Led a technical evaluation and deployment of Kong API Gateway to modernize the API infrastructure, resulting in improved latency and more granular control over service-to-service communication during cloud transformation.
• Conducted robust data quality checks and developed reusable dbt models and macros for increased development efficiency and enhanced governance across the transformation layers.
• Worked on ELT strategy and development across multiple cloud warehouses, including snowflake, DBT, and Azure Synapse Analytics..

➡️ Phase 1: ETL Lead - Core ETL Development

03/2018 - 02/2022
Focus: Foundational ETL, Project Management, and Legacy/On-Premises Tools.

• Managed project scope, schedule, and cost throughout the delivery life cycle.
• Developed complex ETL workflows using SSIS, Informatica, PowerCenter, and Talend.
• Collaborated with cross-functional source teams to identify and implement
  improvements in data systems.
• Designed and implemented slowly changing dimensions (SCDs) for tracking historical data changes within the relational data warehouse.
• Executed successful ETL and reporting rollouts across multiple geographies, streamlining deployment processes, ensuring compliance with business rules and data regulations.
• Performed database monitoring, backup, recovery, and performance tuning
  to ensure system integrity.
• Conducted data quality checks and developed reusable ETL components for efficiency.
• Crafted custom ETL (Extract, Transform, Load) processes tailored to specific project needs, enhancing data usability.

• Developed Informatica mappings that drive from Oracle DB to Postgres by keep tracking of Staging, DWH processes and auditing of the data.
• Involved in development of mappings and sessions using various transformations in Informatica 9.x/10.x
• Involved in creating the Mappings and Workflows for Various Source Systems.
• Preparation of ETL Specifications, Solution document, Unit Test Case, UTL, Migration Documents and Production Support Documents etc.
• Helped the Integration Testing Teams to understand the requirements. supervised the error and provided the correct code to them for retest in case any errors are reported.
• Defect tracking and prevention by performing causal analysis to log defects and fixed the Pre-prod defects.
• Responsible for business analyst tasks such as gathering requirements from customer, creating business requirement documents, sign-off and testing the funcionality.