Chatbot Evaluation through Q&A Test Sets

Fermé
Contact principal
Ample Insight Inc.
Toronto, Ontario, Canada
Associate Partnerships Manager
(1)
3
Projet
Parcours académique
80 heures de travail au total
Apprenant.e
Canada
Niveau Intermédiaire

Portée du projet

Catégories
Analyse des données Apprentissage automatique Intelligence artificielle Data science
Compétences
data science trend analysis large language modeling machine learning chatbot generative artificial intelligence data visualization chatgpt
Détails

Join our team to help advance the evaluation of a cutting-edge data analysis chatbot. This is a hands-on opportunity to work in a fast-paced environment, contributing to a project that spans multiple industries and enhances our chatbot's ability to provide accurate, relevant insights across diverse data domains.


In this role, you’ll develop a collection of Question & Answer (Q&A) sets designed to test and benchmark the performance of our AI-driven chatbot. You’ll work directly with structured datasets across specific industries to create targeted Q&A sets, helping us improve and refine our chatbot’s data analysis capabilities.


Who Should Apply:

  • Experience using ChatGPT, Llama, and/or other LLMs.
  • Students in fields such as Data Science, Computer Science, Business Analytics, or similar.
  • Strong analytical and problem-solving skills, with an interest in data science and machine learning.
  • Experience with large datasets and familiarity with data visualization and trend analysis.
  • Working experience in industries relating to healthcare, environment, marketing, and others.


Livrables
  • Craft Comprehensive Q&A Test Sets: Create detailed question sets and reference answers, working with various Large Language Models (LLMs) and verifying responses to ensure accuracy.
  • Benchmark and Validate: Ensure consistency in answers through manual validation, helping us establish a ground truth for effective evaluation.



Mentorat
Expertise et connaissances du domaine

Fournir des connaissances spécialisées et approfondies et l'industrie générale des idées pour une compréhension globale.

Compétences Connaissances et expertise

Partager des connaissances sur des compétences techniques spécifiques, des techniques, méthodologies requises pour le projet.

Assistance pratique

Implication directe dans les tâches du projet, en offrant des conseils et démonstration de techniques.

Outils et/ou ressources

Donner accès aux outils, logiciels et ressources nécessaires pour la réalisation du projet.

Réunions régulières

Enregistrements programmés pour discuter des progrès, relever les défis et fournir des commentaires.

À propos de l'Compagnie

Compagnie
Toronto, Ontario, Canada
11 - 50 employé.es
Éducation, Entertainment, Environment, It & computing, Science

Ample Insight is a leading Canadian data consulting firm helping organizations across diverse industries build and scale AI and data solutions. We have a proven track record of solving some of the most complex technical and business problems for leading startups and Fortune 500 companies.