#Sqlpro studio for windows full

This is quite impressive all based on semantic inference of schema naming. GROUP BY MONTH(OrderDate), DATENAME(MONTH, OrderDate) How deeply is the model able to reason over the schema: Summarize sales by month before tax and after tax (show name of month) SELECT Which shows the previous result hid two different users named “May”.

#Sqlpro studio for windows full

This principal extends to describing the shape of the resulting dataset…inviting a pseudo-query language of sorts: list categories for user named May include id and full user name SELECTĪ.A0 AS UserID, A.A1 + ' ' + A.A2 AS FullName, Is rejected, versus: list categories for user named May Note: objective need not be expressed as a question: year with highest sales amountĬlarifying implied context can also increase clarity: list categories for May SELECT TOP 1 YEAR(OrderDate) AS Year, SUM(TotalDue) AS TotalSales Provides the semantic cue that SUM is appropriate. Where: what year had the highest sales amount One aspect that comes to light rather quickly is how the model (naturally) responds to semantic nuance and clarity.Īn objective such as: what year had the most salesĮvaluates the COUNT of product sales: SELECT TOP 1 YEAR(OrderDate) AS SaleYear, COUNT(SalesOrderID) AS TotalSales UsageĪs with any LLM processing, it is possible to experience false-positive (bad query) and false-negative (no query) behaviors particularly in response to ambiguous or conflicting direction. Attempting to marginally tune the prompts with additional training shots doesn’t further this goal (at the cost of consuming tokens that are needed for schema expression).Īttempts to consolidate screening and query generation into a single prompt invariably resulted in failed screening for reference objectives such as: list all databasesĮxperimenting with more complex prompting resulted in significant latency, increases cost, with marginal functional impact on query generation.

GenerateQuery: Performs the query generation given the schema and objective.īoth prompts are intentionally low/no-shot approaches in the interest of exposing core model capability.

IsQuery: Screen whether objective is solvable with a SELECT statement against the schema.

We use the objective to retrieve the top-ranked schema for use in two sequential prompts: Schema expressions are stored in volatile semantic memory. (Example: AdventureWorksLT.yaml) Query Generation We ended up favoring YAML since it lends itself to the inclusion of description meta-data and appears to be well understood by the model. Selecting a compact and expressive format to express this schema within the prompt is critical since everything in the prompt counts towards the token-limit. The schema model includes platform direction in order for the model to produce the correct SQL variant (since SQL is notoriously vendor specific). Providing a full-fidelity schema model, besides being incredibly token rich, just doesn’t demonstrate functional improvement for the classes of objectives we explored. If needed, an optional description can be included for additional semantic clarity.Ĭolumn meta-data is highly abbreviated to include just name, reference (Foreign Key) and description. To start, we use an abbreviated object model to express schema meta-data for the model to reason over:Ī well-designed schema is generally imbued with semantic intention via the table and column names. Since GPT-4 demonstrates significantly higher capability for SQL generation (GPT-3.5-turbo generated valid SQL ~70% less than GPT-4 in our limited testing), this is the model we’ve targeted.

Must discriminate across multiple schemas (in order to support multiple data-sources or to decompose a large schema).Prompts cannot be hardcoded to a specific database schema or platform.Injecting data into the prompt-frame is also a non-starter (due to the token limit).Synchronizing an existing database to vector-storage is a non-starter as there is no desire to introduce consistency considerations or any type of data-movement.Some basic principles that informed our approach: Let’s dive right in, check out the video below to view the sandbox in action. We will share our approach, learnings and some best practices. While alternative approaches such as local models or semantic search services exist, the focus here is to zoom into the natural abilities (and limitations) of GPT-4 to produce relevant SQL queries. This has been an area of interest for years ( WikiSQL, Spider, etc.). We are excited to share this sandbox that enables you explore the capabilities of LLM to generate SQL queries (or SELECT statements): NL2SQL. We’ve heard from many in the community who want to use Semantic Kernel to query their relational database using natural language expressions.