blackcanyonconsulting · Bethesda, MD
Skills: Python, SQL
Overview We are seeking a Data Scientist to support applied machine learning, natural language processing, AI evaluation, data quality, and analytical product development for a federal permitting modernization initiative. The Data Scientist will work across structured and unstructured data, including public comments, permitting records, compliance information, documents, and geospatially linked datasets. A key focus of the role will be developing representative evaluation datasets and quantitative methods to measure the performance, reliability, and limitations of AI-enabled and agentic solutions. This position is remote. Duties & Responsibilities Develop analytical and machine-learning approaches supporting permitting and natural-resource use cases. Build and maintain labeled datasets, evaluation corpora, and benchmark datasets for AI-enabled services. Define quantitative evaluation methodologies for classification, extraction, retrieval, summarization, entity matching, and other AI/NLP capabilities. Evaluate LLM and agentic workflows , including output quality, retrieval performance, task completion, failure modes, and reliability. Analyze public-comment datasets and develop reusable features and evaluation datasets for comment-processing services. Apply NLP and machine-learning techniques to structured and unstructured data. Evaluate embedding models, semantic retrieval strategies, document-ranking techniques, and AI-generated outputs. Develop statistical approaches for detecting anomalies, reconciliation issues, missing data, and inconsistent records. Partner with Content Pipeline Engineers to measure OCR, extraction, and document-retrieval performance. Partner with Data Engineers to ensure analytical datasets and evaluation pipelines are reproducible and production-ready. Support entity matching and harmonization across data originating in multiple source systems. Design experiments and validation approaches with clearly defined success metrics. Contribute to c…
Sign in to see the original posting
Browse more developer jobs on RoleBurst