HyperAIHyperAI

Command Palette

Search for a command to run...

6 months ago

MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation

Satya Krishna Gorti Ilan Gofman Zhaoyan Liu Jiapeng Wu Noël Vouitsis Guangwei Yu Jesse C. Cresswell Rasa Hosseinzadeh

MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL
  Translation

Abstract

Text-to-SQL generation enables non-experts to interact with databases vianatural language. Recent advances rely on large closed-source models like GPT-4that present challenges in accessibility, privacy, and latency. To addressthese issues, we focus on developing small, efficient, and open-sourcetext-to-SQL models. We demonstrate the benefits of sampling multiple candidateSQL generations and propose our method, MSc-SQL, to critique them usingassociated metadata. Our sample critiquing model evaluates multiple outputssimultaneously, achieving state-of-the-art performance compared to otheropen-source models while remaining competitive with larger models at a muchlower cost. Full code can be found at https://github.com/layer6ai-labs/msc-sql.

Code Repositories

Benchmarks

BenchmarkMethodologyMetrics
text-to-sql-on-bird-big-bench-for-large-scaleMSc-SQL
Execution Accuracy % (Dev): 65.6
text-to-sql-on-spiderMSc-SQL
Execution Accuracy (Test): 84.7

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp