Pairwise Comparison Tools for Large Language Model-Based Writing
Evaluation
![[R logo]](../../../doc/html/Rlogo.svg)
![[Up]](../../../doc/html/left.jpg)
![[Top]](../../../doc/html/up.jpg)
Documentation for package ‘pairwiseLLM’ version 1.1.0
Help Pages
| alternate_pair_order | Deterministically alternate sample order in pairs |
| anthropic_compare_pair_live | Live Anthropic (Claude) comparison for a single pair of samples |
| anthropic_create_batch | Create an Anthropic Message Batch |
| anthropic_download_batch_results | Download Anthropic Message Batch results (.jsonl) |
| anthropic_get_batch | Retrieve an Anthropic Message Batch by ID |
| anthropic_poll_batch_until_complete | Poll an Anthropic Message Batch until completion |
| build_anthropic_batch_requests | Build Anthropic Message Batch requests from a tibble of pairs |
| build_bt_data | Build Bradley-Terry comparison data from pairwise results |
| build_elo_data | Build EloChoice comparison data from pairwise results |
| build_gemini_batch_requests | Build Gemini batch requests from a tibble of pairs |
| build_openai_batch_requests | Build OpenAI batch JSONL lines for paired comparisons |
| build_prompt | Build a concrete LLM prompt from a template |
| check_llm_api_keys | Check configured API keys for LLM backends |
| check_positional_bias | Check positional bias and bootstrap consistency reliability |
| compute_reverse_consistency | Compute consistency between forward and reverse pair comparisons |
| ensure_only_ollama_model_loaded | Ensure only one Ollama model is loaded in memory |
| example_openai_batch_output | Example OpenAI Batch output (JSONL lines) |
| example_writing_pairs | Example dataset of paired comparisons for writing samples |
| example_writing_samples | Example dataset of writing samples |
| fit_bt_model | Fit a Bradley–Terry model with sirt and fallback to BradleyTerry2 |
| fit_elo_model | Fit an EloChoice model to pairwise comparison data |
| gemini_compare_pair_live | Live Google Gemini comparison for a single pair of samples |
| gemini_create_batch | Create a Gemini Batch job from request objects |
| gemini_download_batch_results | Download Gemini Batch results to a JSONL file |
| gemini_get_batch | Retrieve a Gemini Batch job by name |
| gemini_poll_batch_until_complete | Poll a Gemini Batch job until completion |
| get_prompt_template | Retrieve a named prompt template |
| list_prompt_templates | List available prompt templates |
| llm_compare_pair | Backend-agnostic live comparison for a single pair of samples |
| llm_download_batch_results | Extract results from a pairwiseLLM batch object |
| llm_submit_pairs_batch | Submit pairs to an LLM backend via batch API |
| make_pairs | Create all unordered pairs of writing samples |
| ollama_compare_pair_live | Live Ollama comparison for a single pair of samples |
| openai_compare_pair_live | Live OpenAI comparison for a single pair of samples |
| openai_create_batch | Create an OpenAI batch from an uploaded file |
| openai_download_batch_output | Download the output file for a completed batch |
| openai_get_batch | Retrieve an OpenAI batch |
| openai_poll_batch_until_complete | Poll an OpenAI batch until it completes or fails |
| openai_upload_batch_file | Upload a JSONL batch file to OpenAI |
| parse_anthropic_batch_output | Parse Anthropic Message Batch output into a tibble |
| parse_gemini_batch_output | Parse Gemini batch JSONL output into a tibble of pairwise results |
| parse_openai_batch_output | Parse an OpenAI Batch output JSONL file |
| randomize_pair_order | Randomly assign samples to positions SAMPLE_1 and SAMPLE_2 |
| read_samples_df | Read writing samples from a data frame |
| read_samples_dir | Read writing samples from a directory of .txt files |
| register_prompt_template | Register a named prompt template |
| remove_prompt_template | Remove a registered prompt template |
| run_anthropic_batch_pipeline | Run an Anthropic batch pipeline for pairwise comparisons |
| run_gemini_batch_pipeline | Run a Gemini batch pipeline for pairwise comparisons |
| run_openai_batch_pipeline | Run a full OpenAI batch pipeline for pairwise comparisons |
| sample_pairs | Randomly sample pairs of writing samples |
| sample_reverse_pairs | Sample reversed versions of a subset of pairs |
| set_prompt_template | Get or set a prompt template for pairwise comparisons |
| submit_anthropic_pairs_live | Live Anthropic (Claude) comparisons for a tibble of pairs |
| submit_gemini_pairs_live | Live Google Gemini comparisons for a tibble of pairs |
| submit_llm_pairs | Backend-agnostic live comparisons for a tibble of pairs |
| submit_ollama_pairs_live | Live Ollama comparisons for a tibble of pairs |
| submit_openai_pairs_live | Live OpenAI comparisons for a tibble of pairs |
| submit_together_pairs_live | Live Together.ai comparisons for a tibble of pairs |
| summarize_bt_fit | Summarize a Bradley–Terry model fit |
| together_compare_pair_live | Live Together.ai comparison for a single pair of samples |
| trait_description | Get a trait name and description for prompts |
| write_openai_batch_file | Write an OpenAI batch table to a JSONL file |