New methods of solving ’hard’ computer problems

There are some computer problems so hard that computer scientists consider them out of reach. They label them “intractable” and move on.

But researchers at Cornell University, Ithaca, N.Y., have developed tools to solve such problems, at least in certain practical situations. Mostly their approach is to have the computer do what a human being might do: stop, go back and start over and try something different.

“Even though these problems are intractable in the worst cases, we can find structure in real-world problems that we can exploit so that the problems do not fall into the worst-case scenario,” explains Carla Gomes, Cornell associate professor of computing and information science and applied economics and management.

Gomes and coworker Bart Selman, Cornell associate professor of computer science, have studied these hard problems for several years. Gomes will describe the work and outline some new techniques for dealing with hard problems in a talk at today (Feb. 21) at 2 p.m. at the annual meeting of the American Association for the Advancement of Science at the Marriott Wardman Park hotel, Washington, DC. Her talk is part of a symposium on “The Pervasiveness of Extreme Events in Science, Economics and Engineering.” Selman is the moderator of the session. Gomes is director of the Cornell Intelligent Information Systems Institute.

The hard problems are “combinatorial,” in that the computer is given a large set of variables and is programmed to come up with the most effective combination of values to assign to the variables to meet certain constraints. Examples include scheduling airline flights, routing electric power through a grid, predicting economic trends, proving that a computer program performs as intended in every case, and of course the classic: playing chess.

Computers attack these problems by trying every possible combination and selecting the one with the best outcome — or, sometimes, the first decent answer that shows up. An airline, for example, might want the best way to route its 24 aircraft over the next two weeks with six flights a day into 15 airports, using 50 pilots and 65 co-pilots, choosing a result that uses the least fuel. The computer starts choosing different combinations of value settings, creating an ever-expanding tree of possibilities, repeating the process until all possibilities have been compared, or an adequate solution is found. Computers can do this a lot faster than human beings.

The catch is that sometimes the computer chooses a tree that takes too long to complete, even by computer standards. For some problems, a graph of the time it takes to evaluate each of the many possible trees is a skewed version of the traditional “bell curve,” with a tail that extends very far to the right. In other words, there are a large number of possible combinations that take a long time to try. If the computer gets lucky and finds a good solution before it gets around to trying one of those difficult ones, the answer comes up in a few minutes or seconds. But if it happens to hit a combination in the “heavy tail” of the graph, the analysis could take anywhere from days to millennia.

This explains why parallel cluster computers often can solve combinatorial problems more easily, Gomes notes. Each processor is trying a different path, and while some may be stuck on the heavy tails, sometimes one processor gets lucky. Computer scientists refer to this as a “super-linear speedup.”

Gomes, Selman and others have come up with a number of strategies to deal with heavy-tailed phenomena in computational problems. One of the most effective approaches is to find a “backdoor set” — a small number of key variables whose values can be fixed in advance. In an airline scheduling problem with 10,000 variables, Gomes says, sometimes fixing just 12 of them ahead of time makes the problem easy. Most airline scheduling is still done by hand, she notes, and the people who do it use a similar approach, nailing down certain key flights and then filling in the rest of the schedule.

“Humans are very good at seeing the big picture and seeing what’s critical,” she explains, Chess-playing computers, she points out, analyze every possible move and its consequences many moves ahead, while chess master Gary Kasparov only analyzes a few possible lines of play that his experience suggests.

The mathematical models of heavy-tailed phenomena, Gomes says, have applications outside computer science. In natural language, for example, word frequency is heavy-tailed: the center of the bell curve represents common words, but there are many, many words used infrequently. Statistics of hits on the World Wide Web exhibit heavy-tails because there are a few sites that are very popular, and many that are rarely visited. Massive power blackouts are one demonstration that the probability of extreme events is more real than previously thought.