Biography

Dr Steven Harford is a Computer Scientist with over 10 years of industrial and academic experience. He holds a B.Sc. degree (first class honours) and a Ph.D. degree, both of which he received from Dublin City University in 1997 and 2006 respectively. His doctoral research, which was carried out under the supervision of Dr Mark Humphrys, applied Artificial Neural Network technology to the problem of retrieving melodies by content. He was formerly employed as a Software Engineer at IONA Technologies, where he worked on their flagship product Orbix. Currently, Dr Harford is a Director of Rubicon Investment Consulting, which he co-founded in 2005.

Ph.D. Abstract

Human listeners are capable of spontaneously organizing and remembering a continuous stream of musical notes. A listener automatically segments a melody into phrases, from which an entire melody may be learnt and later recognized. This ability makes human listeners ideal for the task of retrieving melodies by content. This research introduces two neural networks, known as SONNET-MAP and ReTREEve, which attempt to model this behaviour. SONNET-MAP functions as a melody segmenter, whereas ReTREEve is specialized towards content-based retrieval (CBR).

Typically, CBR systems represent melodies as strings of symbols drawn from a finite alphabet, thereby reducing the retrieval process to the task of approximate string matching. SONNET-MAP and ReTREEve, which are derived from Nigrin's SONNET architecture, offer a novel approach to these traditional systems, and indeed CBR in general. Based on melodic grouping cues, SONNET-MAP segments a melody into phrases. Parallel SONNET modules form independent, sub-symbolic representations of the pitch and rhythm dimensions of each phrase. These representations are then bound using associative maps, forming a two-dimensional representation of each phrase. This organizational scheme enables SONNET-MAP to segment melodies into phrases using both the pitch and rhythm features of each melody. The boundary points formed by these melodic phrase segments are then utilized to populate the ReTREEve network.

ReTREEve is organized in the same parallel fashion as SONNET-MAP. However, in addition, melodic phrases are aggregated by an additional layer; thus forming a two-dimensional, hierarchical memory structure of each entire melody. Melody retrieval is accomplished by matching input queries, whether perfect (for example, a fragment from the original melody) or imperfect (for example, a fragment derived from humming), against learned phrases and phrase sequence templates. Using a sample of fifty melodies composed by The Beatles, results show that the use of both pitch and rhythm during the retrieval process significantly improves retrieval results over networks that only use either pitch or rhythm. Additionally, queries that are aligned along phrase boundaries are retrieved using significantly fewer notes than those that are not, thus indicating the importance of a human-based approach to melody segmentation. Moreover, depending on query degradation, different melodic features prove more adept at retrieval than others.

The experiments presented in this thesis represent the largest empirical test of SONNET-based networks ever performed. As far as we are aware, the combined SONNET-MAP and ReTREEve networks constitute the first self-organizing CBR system capable of automatic segmentation and retrieval of melodies using various features of pitch and rhythm.

Downloads

The following documents, which I authored, provide more details of my research:

Content-Based Retrieval of Melodies using Artificial Neural Networks. PhD thesis, Dublin City University, 2006.

Automatic Segmentation, Learning and Retrieval of Melodies using a Self-Organizing Neural Network. In Proceedings of the International Conference on Music Information Retrieval, 2003.

Contact Details

I can be contacted at the following email address: steven.harford@rubiconic.ie