Skip to content
All projects
Data Pipelines & MigrationAirtel X Labs

Siebel CDC Reporting Pipeline

A continuous change-data-capture pipeline from Oracle GoldenGate and Solace queues into Hive, feeding daily postpaid and Telemedia reports.

Context

Postpaid and Telemedia business reports depend on Siebel data that changes throughout the day.

Problem

Reports needed a reliable, continuous feed of changes rather than repeated full extracts from the source.

My role

I built the continuous CDC pipeline.

Approach & architecture

  • Captures change data for postpaid and Telemedia from Oracle GoldenGate and Solace queues.
  • Applies changes to Hive tables that feed daily business reports.

Results

  • Daily postpaid and Telemedia reports are fed by a continuous CDC pipeline into Hive.

Tech stack

  • CDC
  • GoldenGate
  • Solace
  • Hive
  • Ab Initio

What I'd do next

  • Add automated reconciliation between source counts and Hive to catch missed or duplicate events.