Since the launch of ChatGPT in November 2022, terms like “inference,” “reasoning,” and “training data” have permeated everyday conversation, showcasing the profound impact of AI on our lives. These concepts, once confined to computer science labs and tech conferences, are now discussed casually across bars and subways.
Much has been written—and will continue to be written—about improving AI agents and copilots as decision-makers. However, it's essential to remember that, at least in the near term, AI is meant to augment human decision-making, not replace it. For instance, consider a product marketing manager querying her AI tool to find “Which customer segments have the lowest Net Promoter Score (NPS)?” After receiving the answer, she might ask follow-up questions, such as, “What if we segment by geography?” and use that data to refine her promotional strategy.
Looking ahead, we may reach a point where a CEO could instruct an AI, “Design a promotions strategy based on our data, industry best practices, and insights from our last launch,” leading to a plan that rivals the performance of a skilled human product marketing manager. In an even further future, AI may autonomously identify the need for a promotions strategy and initiate the process itself—functioning as an autonomous Chief Marketing Officer (CMO).
For now, until we achieve artificial general intelligence (AGI), humans will remain a crucial part of significant decision-making processes. While many speculate on AI's potential to transform our professional lives, it’s important to consider what it won’t change anytime soon: the value of good human decision-making. Imagine your business intelligence team, supported by AI agents, collaborating on a new promotional strategy analysis. How can you leverage that data effectively? Here are some proven strategies I advocate:
Before Reviewing the Data:
1. Set Clear Criteria: Establish go/no-go criteria before seeing the data. People often shift the goalposts, such as saying, “We’re so close; I believe another year of investment will yield results.” This tendency can lead to prolonged pursuit of unviable projects. To counter this, outline specific criteria beforehand (e.g., “We proceed if over 80% of survey respondents are willing to pay $100 for this product”) to maintain objectivity when analyzing the data.
During the Data Review:
2. Individual Documentation: Before discussing findings, have all decision-makers independently document their thoughts. This avoids groupthink, where dominant opinions can suppress valid concerns. By sharing written insights afterward, you facilitate a comprehensive discussion that values diverse expertise. For further insight, refer to Asch’s conformity studies.
When Making the Decision:
3. Discuss Mediating Judgments: Understand that each significant decision comprises smaller decisions. Cognitive scientist Daniel Kahneman emphasizes that these smaller aspects, such as cost comparisons and expected accuracy, influence the larger decision. Make these explicit during discussions to enhance decision quality.
4. Document Rationale: Record the reasoning behind decisions—e.g., “We anticipate a 20% cost reduction and stable customer satisfaction within nine months”—to allow for honest reassessment in future reviews. This creates a data-driven feedback loop that helps clarify what strategies worked or didn’t, distinguishing between skill and luck.
5. Establish Kill Criteria: Similar to your initial decision criteria, identify metrics that would indicate a project isn’t performing well enough to continue. For instance, “If over 50% of users request to speak to a human after interacting with our chatbot for over a minute, the project should be reconsidered.” This preemptive analysis helps maintain intellectual honesty and ensures unbiased evaluations of project viability.
While this process may seem labor-intensive, these practices quickly become second nature for your team. The added time invested yields significant returns by ensuring that all insights are expressed and risks are managed effectively, allowing for learning from both successes and failures.
As long as humans are involved in data-driven decisions alongside AI agents, mastering the interplay of human insight and AI-generated analysis will remain vital, particularly in navigating cognitive biases.
Ranking
