surface↓summarization↓consensus
Summarizationaggregate · returns varchar

CONSENSUS

Finds common ground among texts via centrality + LLM summary

Per-group — reads the whole group in one call.

summarizationllmscales-largejson

Syntax

CONSENSUS({{ texts }})
CONSENSUS({{ texts }}, '{{ prompt }}')

Arguments

nametypedescription
textsJSON—
prompt(optional)VARCHAR—

About

Find consensus or common ground among a collection of texts. Returns a short summary of what most items agree on or have in common. Backend: hybrid — same pattern as THEMES. For collections up to 30 texts, the LLM reads all of them directly. For larger collections, the cascade embeds every text with bge-m3, picks the 30 texts closest to the collection centroid (the "most representative of the collective view"), and passes only those to the LLM. This gives scale-invariant consensus extraction — CONSENSUS over 100K rows costs the same LLM budget as CONSENSUS over 100 rows. Why central texts instead of random sampling or a full cluster analysis? Consensus is fundamentally "what does the center of the distribution look like" — texts near the centroid by definition carry the shared themes, while outliers carry divergent views. By selecting the most-central texts we bias the LLM toward the consensus view and away from outliers, which is exactly what the operator asks for. For LLM-only consensus (every text in the prompt, no centrality filtering), use CONSENSUS_LLM — see consensus_llm.cascade.yaml.

Examples

Finds consensus in positive service reviews

WITH
  test_data AS (
    SELECT
      *
    FROM
      (
        VALUES
          ('The service was excellent'),
          ('Staff was friendly and helpful'),
          ('Great customer service experience'),
          ('Very attentive and professional team')
      ) AS t (review)
  )
SELECT
  CONSENSUS (review)
FROM
  test_data

Nearby rabbit holes

same domain
scalar

CONDENSE

Condense individual text into brief summary (scalar, per-row)

summarizationllmtext
aggregate

CONSENSUS_LLM

LLM-only consensus (escape hatch for CONSENSUS)

summarizationllmllm-escape-hatchjson
aggregate

MERGE_TEXTS

Combine multiple text values into one coherent output

summarizationllmtext
aggregate

SUMMARIZE

Summarize a group of texts into one concise overview

summarizationllmtext
scalar

SUMMARIZE_URLS

Extracts URLs from text, fetches with browser, returns summary

summarizationllmurl
aggregate

THEMES

Extract N main topics from texts (embed centrality + LLM naming)

summarizationllmscales-largejson
Climb back to The Looking Glass