Specialist Software Engineer - Data Engineer Databricks pySpark at Société GénéraleSociété Générale · KA, India

Experience: 4+ yrs

Posted: 2026-11-05

Skills: Python, SQL, AWS, CI/CDResponsibilities --------------------Specialist Software Engineer – Data Engineer (Databricks-pySpark) ResponsibilitiesUnderstand business and technical requirements and translate them into scalable, maintainable data engineering solutions. Own the design and development of robust, high‑performance data pipelines using Databricks(pySpark) . Build and optimize data flows within the Hadoop ecosystem (Hive, HDFS, Starburst, Oozie). Ensure performance tuning, resource optimization, and efficient cluster usage on Databricks platform. Contribute to data architecture decisions related to storage, processing, security, and exposure. Apply industrialization best practices including CI/CD, unit testing, integration testing, and pipeline validation. Model data based on business needs and maintain shared data components, dictionaries, and governance rules. Perform functional and technical analysis, document specifications, and validate implemented solutions. Participate in Agile ceremonies and collaborate with Product Owners, architecture, business, and Ops teams. Support production processes, troubleshoot issues, update runbooks, and maintain system stability. Ensure compliance with data quality, traceability, and security standards across the data lifecycle. Engage in continuous improvement activities, proposing upgrades, refactoring, and automation opportunities. Work closely with global teams and support cross‑location coordination when required. Demonstrate ownership during key delivery phases, including planned on‑call or deployment activities.Profile required --------------------Profile RequiredExperienced Data Engineer with strong expertise in Databricks , Hadoop ecosystem components, and large‑scale data processing environments.Minimum QualificationsBachelor of Engineering in Computer Science, Information Technology and equivalent streams. Strong hands‑on experience in pySaprk for distributed processing and perform …truncated… ants receive consideration for employment…Sign in to see the original postingBrowse more developer jobs on RoleBurst