Senior Data Engineer - AI Personalization & Real-Time Customer Data

Join our team and what we'll accomplish together

The AI Products and Platform (AIPP) team builds the foundational capabilities that enable AI-driven personalization, decisioning, and real-time customer experiences at scale. Our work supports intelligent interactions across digital and assisted channels, helping ensure customers receive timely, relevant, and responsible experiences.

The team develops and operates platforms for customer segmentation, anomaly detection, and AI-based decisioning, where machine learning models are used to arbitrate recommendations and outcomes. These capabilities rely on high-quality, real-time data to function reliably in production.

While Data Engineers on the team do not build machine learning models, they play a critical role in enabling and operationalizing AI by designing event-driven data pipelines, data architecture, and foundational datasets that allow models to be trained, scored, monitored, and safely activated in real time. Their work ensures AI outputs are reliable, explainable, and actionable across downstream systems.

This is a hands-on engineering role focused on event-driven pipelines, customer segmentation, and data ingestion.

You will partner closely with Data Scientists, Product Managers, IVR/CCAI teams, and Software Engineers to productionize ML models, enable real-time triggers, and support internal tooling (such as future offer-loading or segmentation UIs).

What you will do:

• Build and operate real-time data pipelines that ingest customer events (payments, app behavior, IVR interactions) and power ML product decisions across channels.
• Develop event-driven architectures using GCP services (Pub/Sub, Dataflow, Cloud Run, BigQuery) to support low-latency triggers and segmentation.
• Partner with Data Scientists and ML engineers to productionize AIPP models (e.g., ML2), transforming model outputs into reliable, explainable data feeds.
• Translate business and marketing requirements into scalable data logic, including customer segments, eligibility rules, and offer mappings.
• Enable IVR and CCAI experiences by supporting data flows for intelligent routing, proactive messaging, and self-serve journeys.
• Support internal tools and future UI capabilities by providing clean datasets, schemas, and APIs for Software and Full Stack Engineers.
• Monitor, troubleshoot, and improve production pipelines, ensuring data quality, reliability, and observability during peak periods.
• Optimize for performance and cost, balancing streaming workloads, BigQuery usage, and API integrations.
• Enable responsible AI practices through validation, lineage, and monitoring so personalization decisions remain transparent and trustworthy.
• Contribute to long-term architecture and standards, helping AIPP evolve into a steady-state personalization platform.

What you bring

• 4+ years of experience in Data Engineering or backend data systems
• Strong experience with Python and SQL in production environments
• Hands-on experience with GCP, especially BigQuery and Pub/Sub
• Experience designing and operating streaming or event-driven pipelines
• Understanding of data modeling, data quality, and pipeline reliability
• Experience integrating data via APIs, including retries, rate limiting, and failure handling
• Bachelor’s degree in Computer Science, Engineering, Data Systems, or a related technical field OR equivalent hands-on industry experience building production-grade data pipelines

Nice to Have:

• Experience with Adobe Experience Platform or other CDPs
• Exposure to AI/ML feature pipelines or real-time model scoring
• Experience with Dataflow (Apache Beam), dbt, or Airflow/Composer
• Familiarity with personalization or decisioning platforms (PEGA-like concepts)
• Telecom or high-volume consumer data experience