| Sr. Data Scientist at Remote, Remote, USA |
| Email: [email protected] |
|
From: rajkumar, NITYA Software Solutions Inc. [email protected] Reply to: [email protected] Hi, I hope you are doing well. I am Raj from NITYA Software Solutions Inc. I've included below the job description, please let me know if you are interested, and reply with one updated resume, which I've expected. Role: Sr. Data Scientist Develop test strategies for evaluating AI/ML models, ensuring outputs align with business requirements and user expectations. Design and execute evaluation pipelines using frameworks like DeepEval for generative AI model testing. Automate the evaluation of model accuracy, fluency, factuality, safety, and bias. Create adversarial test cases to validate AI behavior under edge scenarios, such as data poisoning, jailbreaks, and prompt injections. Assess and validate the retrieval augmented generation (RAG) system, including retrieval accuracy and latency. Build automated testing for conversational AI chatbots, covering dialog coherence, context retention, and response diversity. Use metrics like BLEU, ROUGE, perplexity, and embedding similarity to evaluate generative model output. Collaborate with data scientists, ML engineers, and developers to address bugs and performance issues. Implement tools for test data generation that simulate real-world user inputs and edge cases. Report model performance through dashboards and metrics-driven reporting frameworks. Thank you, Rajkumar EMAIL ID:[email protected] NITYA Software Solutions Inc. Keywords: artificial intelligence machine learning access management Idaho Sr. Data Scientist [email protected] |
| [email protected] View All |
| 08:24 PM 26-Dec-24 |