Skip to content

AI agent that querys multiple popular LLMs and evaluates agreement for the best possible response with zero LLM bias, utilizing text normalization, Sentence-Transformers semantic embeddings, and hierarchical agglomerative clustering. Extractive consensus engine utilizes MiniLM-L6-v2 for summarizing agreeing content.

Notifications You must be signed in to change notification settings

JansenMok/ConsensusAI

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 

Repository files navigation

ConsensusAI

Cross-model consensus engine for large language models (LLMs).

Project Idea

ConsensusAI is a solo research project that aims to give the user the best possible answer by comparison of the most popular available AI LLM models available to the public. The consensus engine queries multiple commercially available LLMs, normalizes their responses, then analyzes semantic agreement or disagreement.

  • If the models broadly agree (tunable), ConsensusAI produces a concise summary of the combined responses with absolute minimal bias.
  • If the models disagree, ConsensusAI highlights where the responses diverge and whether some models agree while others do not.

Features

  • Batch Normalization: clean noisy outputs like boilerplate, markdown, emojis, AI disclaimers, and more.
  • Agreement Engine: compute pairwise cosine similarity, cluster responses, and detect consensus.
  • Extractive consensus: assemble a bias-minimized answer with overlapping content within responses.
  • Testing Suite: pytest cases for normalization and agreement detection.

TODO

  • develop output with minimal bias
  • write tests for output
  • develop API querying for all LLMs

About

AI agent that querys multiple popular LLMs and evaluates agreement for the best possible response with zero LLM bias, utilizing text normalization, Sentence-Transformers semantic embeddings, and hierarchical agglomerative clustering. Extractive consensus engine utilizes MiniLM-L6-v2 for summarizing agreeing content.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages