Open Reasoning Tasks
1 Tasks List
This is a collection of open reasoning tasks for language models.
| Task | Description | Tags |
|---|---|---|
| Analogical Problem Solving | This task involves using analogies to solve problems by applying solutions from one domain to another similar situation. It evaluates the model’s ability to recognize… | Problem Solving, Analogical Reasoning, Creative Thinking, Interdisciplinary Application, Systems Thinking, Synthetic |
| Analogy Creation | This task involves creating analogies that effectively compare two different concepts, situations, or objects to highlight similarities in their relationships or structures.… | Creative Thinking, Abstract Reasoning, Conceptual Mapping, Communication, Explanatory Skills, Synthetic |
| Analyzing Cultural Differences | This task involves comparing and contrasting different cultural practices, beliefs, or norms to identify similarities, differences, and potential reasons for these… | Cultural Analysis, Comparative Studies, Social Norms, Cross-Cultural Communication, Global Awareness, Synthetic |
| Analyzing Decision-Making Processes | This task involves examining and evaluating the steps, factors, and reasoning behind decision-making processes. It assesses the model’s ability to understand complex… | Critical Thinking, Decision Analysis, Problem Solving, Strategic Planning, Risk Assessment, Synthetic |
| Analyzing Decision Trees | This task involves interpreting and evaluating decision trees to understand the logic behind decision-making processes. It assesses the model’s ability to follow branching… | Decision Making, Logical Reasoning, Tree Structures, Conditional Logic, Problem Diagnosis, Synthetic |
| Analyzing Historical Counterfactuals | This task involves considering alternative outcomes of historical events by changing key factors or decisions. It evaluates the model’s ability to understand complex… | Historical Analysis, Counterfactual Thinking, Cause and Effect, Critical Thinking, Scenario Planning, Synthetic |
| Analyzing Rhetorical Strategies | This task involves identifying and evaluating the rhetorical strategies used in a given text or speech. It assesses the model’s ability to recognize persuasive techniques… | Rhetoric, Persuasion, Communication Analysis, Critical Thinking, Language, Synthetic |
| Assessing Risk and Uncertainty | This task involves evaluating scenarios with incomplete information to determine potential risks and uncertainties. It assesses the model’s ability to identify possible… | Risk Assessment, Decision Making, Probability, Scenario Analysis, Critical Thinking, Synthetic |
| Bias Detection | This task involves identifying and explaining biases present in given scenarios, statements, or data. It evaluates the model’s ability to recognize various types of biases… | Critical Thinking, Bias Identification, Data Interpretation, Logical Reasoning, Research Methodology |
| Bias Mitigation | Given a statement and the bias that led to it, this task involves constructing a statement as close to the ground truth as possible. It evaluates the model’s ability to… | Critical Thinking, Bias Correction, Logical Reasoning, Data Interpretation, Objective Analysis |
| Calculating Probabilities | This task involves calculating probabilities for various scenarios, including simple and compound events. It evaluates the model’s ability to apply probability theory, use… | Mathematics, Probability Theory, Statistical Reasoning, Problem Solving, Quantitative Analysis |
| Categorizing Information into Hierarchies | This task involves organizing information into hierarchical structures based on relationships, properties, or other logical criteria. It evaluates the model’s ability to… | Classification, Hierarchical Thinking, Conceptual Organization, Pattern Recognition, Logical Structuring, Synthetic |
| Causal Chain Analysis | This task involves identifying and analyzing a sequence of events or factors that lead to a specific outcome. It evaluates the model’s ability to understand cause-and-effect… | Cause and Effect, Systems Thinking, Environmental Science, Economics, Complex Systems Analysis, Synthetic |
| Completing Analogies | This task involves completing analogies in the form “A is to B as C is to ?” to evaluate the model’s ability to recognize relationships between pairs of words and apply them… | Language, Reasoning, Analogies, Vocabulary, Relationships |
| Constructing Valid Arguments | This task involves creating logically sound arguments to support a given conclusion or claim. It evaluates the model’s ability to use premises, apply logical reasoning, and… | Logic, Critical Thinking, Argumentation, Reasoning, Premise-Conclusion Relationships, Synthetic |
| Counterfactual Analysis | Counterfactual analysis involves examining hypothetical scenarios that are contrary to what actually happened. This task requires the model to consider alternative outcomes… | Critical Thinking, Historical Analysis, Cause and Effect, Hypothetical Scenarios, Complex Systems, Synthetic |
| Critical Factor Identification in Theory of Mind | Understand that similar agents may act differently, or different agents may act similarly, based on a third factor. This task evaluates the model’s ability to identify these… | Theory of Mind, Behavioral Analysis, Critical Thinking, Comparative Psychology, Social Cognition |
| Critiquing Argument Structures | This task involves analyzing and evaluating the structure, logic, and effectiveness of arguments. It assesses the model’s ability to identify strengths and weaknesses in… | Critical Thinking, Logical Analysis, Argumentation, Fallacy Identification, Reasoning Skills, Synthetic |
| Curry’s Paradox (semantic) | This task involves examining a specific form of semantic paradox known as Curry’s Paradox. It tests the model’s ability to reason about self-reference, logical implication… | Logic, Paradoxes, Self-Reference, Semantic Analysis, Critical Thinking, Philosophical Reasoning |
| Deciphering Ambiguous Instructions | This task involves interpreting and clarifying instructions that are unclear, incomplete, or potentially contradictory. It evaluates the model’s ability to identify… | Language Interpretation, Critical Thinking, Clarification, Problem Solving, Instruction Analysis, Synthetic |
| Deconstructing Complex Systems | This task involves breaking down complex systems, processes, or phenomena into their constituent parts and explaining how these parts interact to create the overall system.… | Systems Analysis, Cause and Effect, Component Interaction, Process Understanding, Analytical Thinking, Synthetic |
| Deconstructing Metaphors | This task involves analyzing and explaining the meaning behind metaphors. It evaluates the model’s ability to understand figurative language, interpret symbolic… | Figurative Language, Literary Analysis, Symbolic Interpretation, Critical Thinking, Communication Skills, Synthetic |
| Deducing Motives from Actions | This task involves analyzing the actions of individuals or groups and inferring their underlying motivations or intentions. It evaluates the model’s ability to understand… | Psychological Analysis, Behavioral Interpretation, Critical Thinking, Motivation Theory, Context Consideration, Synthetic |
| Deducing Rules from Examples | This task involves analyzing a set of examples to infer the underlying rule or pattern that governs them. It evaluates the model’s ability to recognize patterns, generalize… | Pattern Recognition, Logical Reasoning, Inductive Reasoning, Linguistic Analysis, Mathematical Thinking, Synthetic |
| Deductive Logic Puzzles | This task involves solving deductive logic puzzles to evaluate the model’s ability to use given information, make logical inferences, and arrive at a correct conclusion. | Logic, Deductive Reasoning, Problem Solving, Critical Thinking, Inference |
| Describing Spatial Relationships | This task involves accurately describing the relative positions and orientations of objects in space. It evaluates the model’s ability to understand and communicate spatial… | Spatial Reasoning, Descriptive Skills, Object Orientation, Visual-Spatial Awareness, Communication |
| Detecting Sarcasm and Irony | This task involves identifying and explaining instances of sarcasm or irony in given statements or scenarios. It evaluates the model’s ability to recognize subtle linguistic… | Language Comprehension, Context Analysis, Figurative Language, Social Intelligence, Linguistic Nuance, Synthetic |
| Determining Alternative Outcomes | This task involves analyzing historical events or decisions and reasoning about possible alternative outcomes if key factors had been different. It evaluates the model’s… | Historical Analysis, Counterfactual Thinking, Cause and Effect, Critical Thinking, Scenario Planning |
| Distinguishing Correlation from Causation | This task involves analyzing given scenarios or statistical relationships to identify cases where correlation does not imply causation, especially in unintuitive or… | Critical Thinking, Statistics, Data Analysis, Logical Fallacies, Causal Reasoning |
| Distinguishing Fact from Opinion | This task involves differentiating between factual statements and opinions in given texts or scenarios. It evaluates the model’s ability to recognize objective, verifiable… | Critical Thinking, Information Literacy, Objectivity, Media Analysis, Reasoning, Synthetic |
No matching items