Github Models: Evaluation and Inference Provider

Project preview

Project Overview: Working on this one … but I’m really excited for it

Objectives

  1. TBD

Features

  1. Data Collection
    • Identify and collect statistically significant data sample
  2. Model Evaluation
    • Looking to identify model with answers that deviate the least from reality
    • balancing model performance with model cost
    • Two use case will be evaluated
  • Task Execution
    • Use selected model(s) to execute task

Technology Stack

  • Github Models: for AI Inference and Model evaluation
  • Github Actions: For data collection
  • GitHub Copilot: Who do you think made it possible to get this done in a few hours
  • Cloudflare D1: Database as a service
  • Cloudflare Worker/Pages: Hosting this web page

Outcome

  • TBD