As an early member of our Data Scientist team, you will establish the data-driven product development culture for our customer AI Ads Quality projects. You should expect to define our north-star metrics, design our first A/B tests, and establish source-of-truth dashboards that the program can use to answer their own questions regarding performance and quality.
This role requires a visionary team player who thrives on crafting exceptional data-driven products, embracing the challenge of creating excellence where no precedent exists.
**Main Responsibilities**:
- Collaborate with our client to create, implement and maintain operational, QA and performance metrics; aligned with business specific needs related to AI.- Define and interpret A/B tests that help answer critical questions about the quality of Expert Raters and trends towards best practices.- Establish a data-driven product development culture by driving the definition, tracking, and operationalizing of queries-, projects-, and program-level metrics.- Develop and socialize dashboards, reports, and other ways of enabling the team and program to answer product data questions in a self-serve way.-
- Drive and direct analysis of current workflows and systems to identify opportunities for process improvement, automation, and efficiency enhancement.- This is a high-impact role in which you will have a direct influence on how product decisions are made.- Lead the design, analysis, and interpretation of experiments proactively perform data exploration on engagement behaviors to discover future opportunities.- Support re-pricing exercises and/or new business through business intelligence data readiness.- Able to create, train, teach AI-operational metrics: Kripendorf's alpha, chance corrected agreement, raw agreement, confidence intervals, stratified sampling, pseudo sampling, structured sampling, and others. - **Mandatory**.
**Qualifications**:
- Advanced knowledge of statistical models and metrics; knows how to balance analytical rigor with pragmatism and actionability.- Ability to communicate technical and statistical concepts clearly and concisely among audiences at many different levels, and be solutions oriented.- Ability to lead effectively and collaborate with cross-functional teams, bridging the gap between business and technology.- Willingness to embrace uncertainty and contribute to defining industry standards in an evolving landscape.- Synthesize quantitative data from multiple sources (surveys, behavioral, 3rd party), including visualization of data, into clear and compelling narratives in decks and memos, tailored to the needs and communication preferences of the audience.- An analytical mindset; you love digging into the data (including survey data, public "big data" repositories, and internally logged data) for hidden connections and patterns and to proactively uncover insights based on critical business questions.- Demonstrated knowledge in NLP, large language models, or generative AI - **Desirable**.
**Studies & Experience**:
- English B2** - Required**.- Oral and written comprehension.-
- Appropriate use of language.-
- 3+ years experience in a quantitative role navigating highly ambiguous environments, ideally as a founding data scientist or product analyst at a hyper-growth product company or research org** - Required**.- Advanced degree in Statistics, Mathematics, Physics, Economics or related quantitative field - **Desirable**.- Demonstrated experience working, digesting and/or processing Big Data models - **Desirable**.- Experience in Research and Development projects.- Experience validating quantitative insights with qualitative methods (e.g. surveys, UXR) - **Desirable**.- Proposed, designed, and run rigorous experiments with clear insights and product recommendations utilizing SQL, Python, and/or R**:
- Required**.- Able to work on SQL, Hive, Spark, Presto, Python, or related to drive analysis.**Benefits and Perks**:
- Competitive Salary.- Mental health and wellness support.- National Holidays OFF.- Paid Parental Leave.- Annual merit increase based on performance.- Life insurance.- PTO.