Real time big data modelling and delivery

request
Request
A solution to update tens of millions of profiles daily, calculate a score vector for each and deliver real-time "Call to Action" prompts on web pages.
approach
Approach
Implemented Kafka + Hadoop + Spark for profiling, and a smart caching solution for fast delivery.
result
Result
Our system ingests data from multiple sources (Webtrekk, DMP, CRM), recalculates all scores overnight in under 3 hours, and the "Call to Action" engine retrieves information in under 5ms for each impression.
null