National Debt Relief, LLC. · United States
Skills: Python, SQL
Overview --------
National Debt Relief is seeking a data-savvy Manager of AI Analytics to serve as an individual-contributor analytics expert supporting the measurement, testing, and optimization of agent-replacement AI across voice and chat channels. This role will execute the analytics framework that governs AI interaction performance by designing tests, tracking KPIs, and building the dashboards that give the business real-time visibility into how AI is performing against quality, containment, and client experience goals.
Reporting to the Director of Digital & AI Analytics, this individual will be a hands-on analytical partner to AI Analytics and Engineering teams, translating complex model and interaction data into clear, actionable recommendations that inform the pace and scope of AI scale-up.
Responsibilities ----------------
Execute the measurement framework for AI Analytics, tracking performance of agent-replacement AI interactions across voice and chat channels. Design and run test analysis for AI model and prompt experiments (e.g., model version A/B tests, containment vs. escalation testing), translating results into go/no-go recommendations. Build and maintain dashboards and scorecards tracking AI-handled interaction KPIs — containment rate, accuracy, quality/CSAT, and escalation rate. Partner with AI Analytics and Engineering teams to ensure AI interaction data (transcripts, model outputs, outcomes) flows cleanly into unified reporting. Analyze AI-handled interaction volume and performance trends to support decisions on scaling AI across channels, balancing performance thresholds and client experience. Collaborate with data engineering to validate and maintain data flows into Snowflake for AI analytics use cases. Communicate findings and recommendations clearly to both technical and non-technical stakeholders, including Analytics leadership. Identify data quality issues or gaps in AI interaction tracking and drive resolution with cross-functional partners. S…
Sign in to see the original posting
Browse more developer jobs on RoleBurst