Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study.

The trade-offs between different desirable plan properties -- e.g.PDDL temporal plan preferences -- are often difficult tounderstand. Recent work addresses this by iterative planning withexplanations elucidating the dependencies between such planproperties. Users can ask questions of the form ``Why does the plannot satisfy property p?'', which are answered by ``Because then wewould have to forego q''. It has been shown that such dependenciescan be computed reasonably efficiently. But is this form ofexplanation actually useful for users? We run a large crowd-workeruser study (N=100 in each of 3 domains) evaluating that question. Toenable such a study in the first place, we contribute a Web-basedplatform for iterative planning with explanations, running in standardbrowsers. Comparing users with vs. without access to the explanations,we find that the explanations enable users to identify bettertrade-offs between the plan properties, indicating an improvedunderstanding of the planning task.