STRESS-TESTING ALGORITHMS: GENERATING NEW TEST INSTANCES TO ELICIT INSIGHTS

Informing Multiobjective Optimization Benchmark Construction Through Instance Space Analysis

Estefania Yap, Mario Andres Munoz, Kate Smith-Miles

2022-12-01

The role of carefully constructed benchmark suites in algorithm design and testing is critical. Within the continuous multiobjecti..

Optimal construction of montages from mathematical functions on a spectrum of order-disorder preference

Kate Smith-Miles, Mario Andres Munoz

2022-11-09

We previously generated diverse mathematical functions that are difficult for optimization algorithms. Represented as 2D contour p..

Revisiting Facial Age Estimation With New Insights From Instance Space Analysis

Kate Smith-Miles, Xin Geng

2022-05-01

When demonstrating the effectiveness of a new algorithm, researchers are traditionally encouraged to compare their algorithm's per..

Generating Diverse Clustering Datasets with Targeted Characteristics

Luiz Henrique dos Santos Fernandes, Kate Smith-Miles, Ana Carolina Lorena

2022-01-01

When evaluating clustering algorithms, it is important to assess their performance in retrieving clusters of datasets with known s..

Relating instance hardness to classification performance in a dataset: a visual approach.

Pedro Yuri Arbs Paiva, Camila Castro Moreno, Kate Smith-Miles, Maria Gabriela Valeriano, Ana Carolina Lorena

2022-01-01

Machine Learning studies often involve a series of computational experiments in which the predictive performance of multiple model..

Algorithm selection and instance space analysis for curriculum-based course timetabling

Arnaud De Coster, Nysret Musliu, Andrea Schaerf, Johannes Schoisswohl, Kate Smith-Miles

2021-09-10

We propose an algorithm selection approach and an instance space analysis for the well-known curriculum-based course timetabling p..

On the diversity and robustness of parameterised multi-objective test suites

Estefania Yap, Mario Andres Munoz, Kate Smith-Miles

2021-06-28

The development of optimisation algorithms heavily relies on comparing performance across benchmark problem suites. In continuous ..

Analyzing randomness effects on the reliability of exploratory landscape analysis

Mario Andres Munoz, Michael Kirley, Kate Smith-Miles

2021-02-12

The inherent difficulty of solving a continuous, static, bound-constrained and single-objective black-box optimization problem dep..

Revisiting where are the hard knapsack problems? via Instance Space Analysis

Kate Smith-Miles, Jeffrey Christiansen, Mario Andres Munoz

2021-01-05

In 2005, David Pisinger asked the question “where are the hard knapsack problems?”. Noting that the classical benchmark test insta..

A transformation technique for the clustered generalized traveling salesman problem with applications to logistics

Pouya Baniasadi, Mehdi Foumani, Kate Smith-Miles, Vladimir Ejov

2020-09-01

The clustered generalized traveling salesman problem (CGTSP) is an extension of the classical traveling salesman problem (TSP), wh..

Generating New Space-Filling Test Instances for Continuous Black-Box Optimization

Mario A Munoz, Kate Smith-Miles

2020-09-01

This article presents a method to generate diverse and challenging new test instances for continuous black-box optimization. Each ..

Generation techniques for linear programming instances with controllable properties

Simon Bowly, Kate Smith-Miles, Davaatseren Baatar, Hans Mittelmann

2020-09-01

This paper addresses the problem of generating synthetic test cases for experimentation in linear programming. We propose a method..

Instance Space Analysis of Combinatorial Multi-objective Optimization Problems

E Yap, MA Munoz, K Smith-Miles, A Liefooghe

2020-07-01

In recent years, there has been a continuous stream of development in evolutionary multi-objective optimization (EMO) algorithms. ..

Instance space analysis for a personnel scheduling problem

Lucas Kletzander, Nysret Musliu, Kate Smith-Miles

2020-04-24

This paper considers the Rotating Workforce Scheduling Problem, and shows how the strengths and weaknesses of various solution met..

On normalization and algorithm selection for unsupervised outlier detection

Sevvandi Kandanaarachchi, Mario A Munoz, Rob J Hyndman, Kate Smith-Miles

2019-11-21

This paper demonstrates that the performance of various outlier detection methods is sensitive to both the characteristics of the ..

Instance space analysis for unsupervised outlier detection

S Kandanaarachchi, MA Muñoz, K Smith-Miles

2019-01-01

Mapping the Effectiveness of Automated Test Suite Generation Techniques

Carlos Oliveira, Aldeida Aleti, Lars Grunske, Kate Smith-Miles

2018-09-01

Automated test suite generation (ATSG) is an important topic in software engineering, with a wide range of techniques and tools be..

Instance spaces for machine learning classification

Mario A Munoz, Laura Villanova, Davaatseren Baatar, Kate Smith-Miles

2018-01-01

This paper tackles the issue of objective performance evaluation of machine learning classifiers, and the impact of the choice of ..

Performance Analysis of Continuous Black-Box Optimization Algorithms via Footprints in Instance Space

Mario A Munoz, Kate A Smith-Miles

2017-12-01

This article presents a method for the objective assessment of an algorithm's strengths and weaknesses. Instead of examining the p..

Visualising forecasting algorithm performance using time series instance spaces

Yanfei Kang, Rob J Hyndman, Kate Smith-Miles

2017-04-01

It is common practice to evaluate the strength of forecasting methods using collections of well-studied time series datasets, such..

Generating custom classification datasets by targeting the instance space

Mario A Munoz, Kate Smith-Miles

2017-01-01

While machine learning has evolved at a fast pace in the last decades, the testing procedure of new methods may be not keeping pac..

Non-parametric model of the space of continuous black-box optimization problems

Mario A Munoz, Kate Smith-Miles

2017-01-01