Multiple Attribute Decision Matrices, Right-Handed

decision matrices (MADM) are a useful way to compare multiple alternatives and choose the option that best fits your needs and budget. By examining a set of criteria for each option, you can be sure you have a clear understanding of the decision space.

However, they are often misinterpreted or misused. This article explains how to use multi-attribute decision matrices and avoid pitfalls often associated with their use. It also lays the groundwork for a different approach that borrows key concepts from MADM without falling into its obscurantist pitfalls.

Inspirational Example: Tent Selection

My family is in the market for a new tent. So, we did what we usually do: we googled “best car camping tent.” One of the first results was a GearLab article called “The Best Camping Tents | Tested and Ranked.”

In the article, GearLab rates 16 tents on a scale of 1 to 10 across five factors. They weigh those qualities, then rate the tents from 1-16 based on a weighted score. This is a specific example of a multi-factor decision matrix.

The purpose of MADM

MADM is often treated as a way for data to make decisions on behalf of stakeholders. In a GearLab article, they recommend one “best” tent based on their MADM findings. I want to emphasize that MADM does not do that do decision; it it informs it.

It can best be understood as a useful tool for organizing comparisons across alternatives, eliminating clearly inferior options, and uncovering top contenders. Used correctly, it helps decision makers see the range of available options rather than pointing them to a single “right” choice.

If used poorly, it can steer a decision downward and leave the decision maker with a bad taste in their mouth about “data-driven” decision-making.

In short, the purpose of MADM is to give decision makers a better understanding of their options, eliminate negative options, and present value propositions, not to make a decision automatically.

How to Use MADM

Here is my basic MADM guide:

1. Identify the decision maker, the decision area, and the attributes.
2. Define weights for each attribute.
3. Collect data and calculate weighted scores.
4. Plot the products against the price and find the effective frontier.
5. Present findings and recommendations to the decision maker.

Briefly, I will explain each one in more detail.

First, find out who the decision maker is. Are you making this analysis for someone else's decision, or for your own? For this example, let's assume it's because of your decision.

Defining the decision point is usually straightforward. You need to know the type of object (such as a tent) being considered and identify the top n options. Make sure you properly sample all the options, not just the ones that come to mind.

Then, assign a few attributes. Come up with a list of things that would make the product more useful or valuable.

After explaining the features, I recommend talking to the decision maker. Once you start talking to the decision maker, make sure you are using their priorities, not yours.

Rate the qualities in order of importance, and consider the trade-offs. Tradeoff questions like “Would I trade an inch of headroom from 71 inches to 70 inches for a less windproof tent?” Then, assign attribute weights according to these responses and tabulate them for later use. These will never be perfect, even when the analysis is for your own use.

Now you have something like this.

Collecting data can vary in difficulty. In this case, it's relatively straightforward. Search for each tent, go to “tech specs” for more information, and reviews for more. Record that data in your decision matrix. If it's not specific, you may need to assign a value independently to each attribute, but be sure to define your criteria, or at least your general reasoning, when you do this.

For the GearLab tents, they rated each attribute on a scale of 1 to 10, as shown below.

Now, your decision matrix looks like this. Note that to keep the chart readable, I omitted the “quality” attribute.

All that remains is to calculate the weighted points. To do this, take the sum of the product of the weights and the values ​​of each item. You now have a completed decision matrix. I have also included a reference value.

Next, plot the weighted scores of each item by its price, align them to the structure, and plot the efficiency boundary:

In this case, we can see eight tents in the effective border. Being on the efficient frontier means that we cannot get a better weighted score at the same or lower price. This is the main idea that MADM provides: to identify which options are strongly governed and involve a reasonable trade-off between quality and cost.

If this plot seems familiar, it may be because you've seen a similar plot in a successful financial risk return. One axis is what you want to be low (value/risk), and the other is what you want to be high (score/return).

So which would you recommend? If my budget is $600 and I want the highest quality tent I can afford, I would choose the North Face Wawona 6.

Look here: I drew a line on the budget, then selected the first tent to the left of that line on the effective border. I would do the same thing if I had a “quality budget” and draw a line, then pick a starting point on the efficiency frontier above the line.

All that's left now is to present your findings to the decision maker. When doing this, I recommend directing them to the building and identifying and defining the effective boundary. Something as simple as “at that point, you can't get a better deal for the same price” will suffice. Show the option with the highest rating. If you know their budget in advance, make appropriate recommendations.

Note that if we use the score weighted by price, we lose a lot of information and cannot decide which tent to choose. It is acceptable to include this information, but it is not necessary, as it sometimes tells a misleading story. For example, if a tent only costs $5 at a garage sale and is as big as a leading competitor, but it leaks when it rains, it's not a real competitor. However, the rating may show it as the “best value” option. For the same reason, price should be kept separate from MADM attributes and used only as a constraint or tradeoff.

The conclusion

Now that you understand how MADM works, its shortcomings are easy to see. It has a tendency to ignore certain details in decision-making by lumping everything into one point and taking a linear approach to all aspects (ie, a rise from 70 inches to 71 inches is considered equally important as a rise from 40 inches to 41 inches, which may not be the case).

It is important to understand the workings of MADM to appreciate the progress achieved by using the following method. In the second part of this two-part series, I will propose an alternative approach to MADM that retains its power while producing recommendations that are more in line with the priorities of decision makers.

Author's Note

If you enjoyed this, I write about analytical thinking, decision science, optimization, and data science. And I share new work and related thoughts on LinkedIn.