If you join us these will be your responsibilities:
Architect and implement our new data infrastructure based on the Lambda Architecture with Apache Spark (including Mesos, Kafka, Cassandra, AWS EC2, SQS and S3) with special focus on providing real-time data for mobile app analytics using Spark Streaming and probabilistic data structures for high scalability.
Design, architect and implement highly scalable distributed services using functional programming techniques in Scala/Python.
- Expert level proficiency in SQL. Ability to perform complex analysis with large volumes of storage technology and the big data ecosystem, and the lambda architecture
- Experience in warehousing and data modelling
- Proficiency in at least one of the scripting languages: Python
- Proficiency in functional programming concepts and languages is a plus (e.g. Scala)
- Real-time pipeline construction (like Kafka, Kinesis, Spark Streaming) is a plus
- Machine learning experience a plus, especially in Python scientific libraries and/or Spark’s MLlib
- Experience building consumer-facing products (e.g. recommendations) a plus