Research

Hate Speech Detection on Twitter: Feature Engineering v.s. Feature Selection

The increasing presence of hate speech on social media has drawn significant investment from governments, companies, and empirical research. Existing methods typically use a supervised text classification approach that depends on carefully engineered features. However, it is unclear if these features contribute equally to the performance of such methods. We conduct a feature selection analysis in such a task using Twitter as a case study, and show findings that challenge conventional perception of the importance of manual feature engineering: automatic feature selection can drastically reduce the carefully engineered features by over 90% and selects predominantly generic features often used by many other language related tasks; nevertheless, the resulting models perform better using automatically selected features than carefully crafted task-specific features.

Robinson, D., Zhang, Z. and Tepper, J. (2018) Hate
speech detection on Twitter : feature engineering v.s. feature selection. In The Semantic Web: ESWC 2018 Satellite Events. ESWC: European
Semantic Web Conference, 03-07 Jun 2018, Crete, Greece. Springer , pp. 46-49. ISBN
9783319981918
https://doi.org/10.1007/978-3-319-98192-5_9

Extracting finite structure from infinite language

This paper presents a novel connectionist memory-rule based model capable of learning the finite-state properties of an input language from a set of positive examples. The model is based upon an unsupervised recurrent self-organizing map with laterally interconnected neurons. A derivation of functional-equivalence theory is used that allows the model to exploit similarities between the future context of previously memorized sequences and the future context of the current input sequence. This bottom-up learning algorithm binds functionally related neurons together to form states. Results show that the model is able to learn the Reber grammar perfectly from a randomly generated training set and to generalize to sequences beyond the length of those found in the training set.

McQueen, T., Hopgood, A.A., Allen, T.J. and Tepper, J.A. , 2005. Extracting finite structure from infinite language. Knowledge-Based Systems, 18 (4-5), pp. 135-141. ISSN 0950-7051

A corpus-based connectionist architecture for large-scale natural language parsing

We describe a deterministic shift-reduce parsing model that combines the advantages of connectionism with those of traditional symbolic models for parsing realistic sub-domains of natural language. It is a modular system that learns to annotate natural language texts with syntactic structure. The parser acquires its linguistic knowledge directly from pre-parsed sentence examples extracted from an annotated corpus. The connectionist modules enable the automatic learning of linguistic constraints and provide a distributed representation of linguistic information that exhibits tolerance to grammatical variation. The inputs and outputs of the connectionist modules represent symbolic information which can be easily manipulated and interpreted and provide the basis for organizing the parse. Performance is evaluated using labelled precision and recall. (For a test set of 4128 words, precision and recall of 75% and 69%, respectively, were achieved.) The work presented represents a significant step towards demonstrating that broad coverage parsing of natural language can be achieved with simple hybrid connectionist architectures which approximate shift-reduce parsing behaviours. Crucially, the model is adaptable to the grammatical framework of the training corpus used and so is not predisposed to a particular grammatical formalism.

Tepper, J.A., Powell, H.M., and Palmer-Brown,D (2002) A corpus-based connectionist architecture for large-scale natural language parsing, Connect. Sci. 14 (2) (2002)
93–114 2002.

Connectionist natural language parsing

The key developments of two decades of connectionist parsing are reviewed. Connectionist parsers are assessed according to their ability to learn to represent syntactic structures from examples automatically, without being presented with symbolic grammar rules. This review also considers the extent to which connectionist parsers offer computational models of human sentence processing and provide plausible accounts of psycholinguistic data. In considering these issues, special attention is paid to the level of realism, the nature of the modularity, and the type of processing that is to be found in a wide range of parsers.

Palmer-Brown, D., Tepper, J. A., and Powell, H. M (2002)
Connectionist natural language parsing. Trends in Cognitive Sciences, 2002, 6 (10), 437-442 Jan 1, 2002

A Multi-recurrent Network for Crude Oil Price Prediction

Crude oil is fundamental for global growth and stability. The factors influencing crude oil prices and more generally, the oil market, are well known to be dynamic, volatile and evolving. Subsequently, crude oil prediction is a complex and notoriously difficult task. In this paper, we evaluate the Multi-recurrent Network (MRN), a simple yet powerful recurrent neural network, for oil price forecasting at various forecast horizons. Although similar models, such as Long Short-Term Memory (LSTM) networks, have shown some success in this domain, the MRN is a comparatively simplified neural network model which exhibits complex state-based memories that are both flexible and rigid. We evaluate the MRN against the standard Feedforward Multilayered Perceptron (FFMLP) and the Simple Recurrent Network (SRN) in addition to the current state-of-the-art LSTM for specifically modelling the shocks in oil prices caused by the financial crisis. The in-sample data consists of key indicator variables sampled across the pre-financial crisis period (July 1969 to September 2003) and the out-sample data used to evaluate the models, is before, during and beyond the crisis (October 2003 to March 2015). We show that such simple sluggish state-based models are superior to the FFMLP, SRN and LSTM models. Furthermore, the MRN appears to have discovered important latent features embedded within the input signal five years prior to the 2008 financial crisis. This suggests that the indicator variables could provide Central Banks and governments with early warning indicators of impending financial perturbations which we consider an invaluable finding and worthy of further exploration.

Orojo, O., Tepper, J. A., Mcginnity, T. M., and Mahmud, M., (2019) A Multi-recurrent Network for Crude Oil Price Prediction Networks. 2019 IEEE Symposium Series on Computational Intelligence (SSCI), 2019, pp. 2940-2945, doi: 10.1109/SSCI44817.2019.9002841.

Does money matter in inflation forecasting?

This paper provides the most fully comprehensive evidence to date on whether or not monetary aggregates are valuable for forecasting US inflation in the early to mid 2000s. We explore a wide range of different definitions of money, including different methods of aggregation and different collections of included monetary assets. In our forecasting experiment we use two non-linear techniques, namely, recurrent neural networks and kernel recursive least squares regression – techniques that are new to macroeconomics. Recurrent neural networks operate with potentially unbounded input memory, while the kernel regression technique is a finite memory predictor. The two methodologies compete to find the best fitting US inflation forecasting models and are then compared to forecasts from a naive random walk model. The best models were non-linear autoregressive models based on kernel methods. Our findings do not provide much support for the usefulness of monetary aggregates in forecasting inflation. Beyond its economic findings, our study is in the tradition of physicists’ long-standing interest in the interconnections among statistical mechanics, neural networks, and related nonparametric statistical methods, and suggests potential avenues of extension for such studies.

Binner J.M., Tino P, Tepper J.A., Anderson R.G, Jones B and Kendall G. (2010) Does money matter in inflation forecasting? Physica A, Statistical Mechanics and its Applications, 389, 4793-4808. ISSN 0378-4371.

Predictable non-linearities in U.S. inflationPredictable non-linearities in U.S. inflation

This paper compares the out-of-sample inflation forecasting performance of two non-linear models; a neural network and a Markov switching autoregressive (MS-AR) model. We find that predictable non-linearities in inflation are best accounted for by the MS-AR model.

Binner, J., Elgar, T., Nilsson, B. and Tepper, J.,(2006) Predictable non-linearities in U.S. inflation. Economics Letters, 93 (3), pp. 323-328. ISSN 0165-1765

Tools for non-linear time series forecasting in economics – an empirical comparison of regime switching vector autoregressive models and recurrent neural networks

The purpose of this study is to contrast the forecasting performance of two non-linear models, a regime-switching vector autoregressive model (RS-VAR) and a recurrent neural network (RNN), to that of a linear benchmark VAR model. Our specific forecasting experiment is U.K. inflation and we utilize monthly data from 1969 to 2003. The RS-VAR and the RNN perform approximately on par over both monthly and annual forecast horizons. Both non-linear models perform significantly better than the VAR model.

Binner, J., Elgar, T., Nilsson, B. and Tepper, J., (2004) Advances in Economics: Applications of Artificial Intelligence in Finance and Economics, 2004, (19), 71-91 • Jan 1, 2004

Dr Tepper and his internationally renown research collaborators have also presented their macro-economic research at the following conferences:

Binner, J., Tepper, J., and Kelly, L (2017) On the robustness of sluggish state-based neural networks for providing useful insight into the New Keynesian Phillips curve. To appear in Fourth International Conference of the Society for Economic Measurement, Samberg Center at the Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, USA, July 26-28, 2017

Binner, J., Tepper, J., and Kelly, L (2017) On the robustness of sluggish state-based neural networks for providing useful insight into the output gap. Bank of England Conference on “Financial Services Indices, Liquidity and Economic Activity”, Bank of England, London, 22-24 May 2017.

Binner, J., Tepper, J., and Kelly, L (2016) Multi-stage learning of US price levels using forecasts from sluggish state-based neural networks. Third International Conference of the Society for Economic Measurement Electra Palace Hotel, Thessaloniki, Greece, 6-9th July 2016.

Binner, J., Kelly, L., Tepper, J., and Chauvet. M (2015) Forecasting Macroeconomic Time Series: A Comparison of Regime Switching and Recurrent Neural Network Methods. Second International Conference of the Society for Economic Measurement, OECD Conference Centre, 2 rue André Pascal 75016 Paris, France, 22-24th July 2015.

Belongia, M., Binner, J., Tepper J and Kelly, L (2014) Were the Central Banks Correct in Abandoning Monetary Aggregates as Targets? Comparative Evidence from the USA and Switzerland. First International Conference of The Society for Economic Measurement, University of Chicago, Chicago, USA 18-20 August 2014.

A novel deep mining model for effective knowledge discovery from omics data

Knowledge discovery from omics data has become a common goal of current approaches to personalised cancer medicine and understanding cancer genotype and phenotype. However, high-throughput biomedical datasets are characterised by high dimensionality and relatively small sample sizes with small signal-to-noise ratios. Extracting and interpreting relevant knowledge from such complex datasets therefore remains a significant challenge for the fields of machine learning and data mining. In this paper, we exploit recent advances in deep learning to mitigate against these limitations on the basis of automatically capturing enough of the meaningful abstractions latent with the available biological samples. Our deep feature learning model is proposed based on a set of non-linear sparse Auto-Encoders that are deliberately constructed in an under-complete manner to detect a small proportion of molecules that can recover a large proportion of variations underlying the data. However, since multiple projections are applied to the input signals, it is hard to interpret which phenotypes were responsible for deriving such predictions. Therefore, we also introduce a novel weight interpretation technique that helps to deconstruct the internal state of such deep learning models to reveal key determinants underlying its latent representations. The outcomes of our experiment provide strong evidence that the proposed deep mining model is able to discover robust biomarkers that are positively and negatively associated with cancers of interest. Since our deep mining model is problem-independent and data-driven, it provides further potential for this research to extend beyond its cognate disciplines.

Alzubaidi, A., Tepper, J. and Lotfi, A., (2020) A novel deep mining model for effective knowledge discovery from omics data. Artificial Intelligence in Medicine, 104: 101821. ISSN 0933-3657

Also see:
Alzubaidi, A. , Tepper, J. and Lotfi, A. , (2020). Deep mining for determining cancer biomarkers. HealthManagement.org – The Journal, 20 (6), pp. 462-464. ISSN 1377-7629

A Computational Intelligence Tool to Support the Design of Outcome-Based Teaching

The use of models, frameworks, and toolkits in learning design serve to support teaching practitioners in producing well-structured learning designs for students. However, existing learning design tools inherently lack an objective metric system which is able to measure the degree to which an educational design is well-formed according to either the principles of constructive alignment or more generally design practices that students find satisfactory in practice. Such a metric system, that could integrate measures of educational theory and practice, would enable teaching practitioners to make more informed design decisions such as which profile of activities/assessments to use for a particular set of learning outcomes. This paper presents the first computational intelligence tool that measures educational design quality in a way that is underpinned by both the theoretical principles of constructive alignment and how it is used in practice. Furthermore, the alignment metrics computed are calibrated by student satisfaction scores to promote those structures that are preferred in practice rather than from a theoretical standpoint thus offering more pragmatic and realistic design solutions.

Bafail, A., Tepper, J., and Liggett, A (2017) A Computational Intelligence Tool to Support the Design of Outcome-Based Teaching. To appear in Proceedings of Canada International Conference on Education (CICE-2017), University of Toronto Mississauga, 26-29 June 2017.

Assessment for learning systems analysis and design using constructivist technique

This paper describes an innovative approach to assessment design that enables first year undergraduate students to learn Systems Analysis and Design concepts in a way that is relevant to them. The approach is inherently constructivist from two standpoints. Firstly, common group knowledge of board games is used as a means for learning subject-specific knowledge and secondly, concept mapping is used to enable groups to visualise and evolve their understanding over time. The preliminary results reveal a significant impact on student achievement and also strong student feedback in support of the approach.

Tepper, J (2014) Assessment for learning systems analysis and design using constructivist technique. Proceedings of Third HEA STEM Annual Learning and Teaching Conference, University of Edinburgh on 30 April – 1 May 2014.

Module assessment: assessment, content, standards alignment and grade integrityModule assessment: assessment, content, standards alignment and grade integrity

An exploratory exercise considered the design of exams in modules and alignment. Interviews with six module leaders in different disciplines revealed good practices around exam design. However, the discussions also elicited potential threats to grade integrity. These are categorised as:
• lack of transparency in the alignment of learning outcomes and standards;
• unassessed or sparse assessment of some learning outcomes in comparison to others;
• exam design, specially where optional questions exist, was found to pose some challenges to grade integrity as questions some times were not found to be of same level of difficulty;
• extraneous factors to the exam design, such as organisation of the delivery in tutorials and previous coverage of content, were equally seen to challenge grade integrity.
Overall, the small scale study has revealed that professional development around assessment design and alignment of standards needs to pay much closer attention to:
• the relation between learning outcomes and definition of criteria and level descriptors
• the relation between assessment design and delivery factors (e.g. previously seen or rehearsed questions as opposed to unseen questions)
• procedures and guidance to support checks, perhaps at design stages, of alignment and due coverage of learning outcomes in the assessment.

Tomas, C., Thomas, G., and Tepper, J (2014) Module assessment: assessment, content, standards alignment and grade integrityModule assessment: assessment, content, standards alignment and grade integrity
SIG1 conference • Aug 29, 2014SIG1 conference • Aug 29, 2014

Connecting scholarship to teaching: The merits of a scientific interdisciplinary approach

The paper examines the merits of a development methodology for promoting Scholarship of Teaching and Learning and engendering an academic learning community within an interdisciplinary setting. A case study which depicts low levels of engagement due to the common resistors associated with unfunded projects is reviewed. A second case study with a revised methodology illustrates how an effective support and evaluation framework for a sabbatical scheme has been adopted across a number of science-based disciplines. A force field analysis reveals the value of this approach for developing academic learning communities whose activities move beyond Ashwin and Trigwell’s level one SoTL.

Liggett, A. and Tepper, J. A. (2010) Connecting scholarship to teaching: The merits of a scientific interdisciplinary approach.
The London Scholarship of Teaching and Learning 8th International Conference Proceedings 2010 Disciplines, Pedagogies and Cultures for SoTL, Volume 5, University of West London, ISBN: 978-0-9569534-0-7

Also see:

Tepper, J., McNeil, J. and Liggett, A. (2006) Connecting scholarship to teaching: a method of
iterative refinement . In J. Fanghanel and D. Warren Scholarship of Teaching and
Learning (SoTL), 6th Annual International Conference, City University, London, UK, 18-19
May 2006.

Measuring constructive alignment: an alignment metric to guide good practice

We present a computational model that represents and computes the level to which an educational design is constructively aligned. The model is able to provide ‘alignment metrics’ for both holistic and individual aspects of a programme or module design.
A systemic and structural perspective of teaching and learning underpins the design of the computational model whereby Bloom’s taxonomy is used as a basis for categorising the core components of a teaching system and some basic principles of generative linguistics are borrowed for representing alignment structures and relationships. The degree of alignment is computed using Set theory and linear algebra. The model presented forms the main processing framework of a software tool currently being developed to facilitate teachers to systematically and consistently produce constructively aligned programmes of teaching and learning. It is envisaged that the model will have broad appeal as it allows the quality of educational designs to be measured and works on the principle of ‘practice techniques’ and ‘learning elicited’ as opposed to content.

Tepper, J. A (2006). Measuring constructive alignment: an alignment metric to guide good practice,” in 1st UK Workshop on Constructive Alignment, Higher Education Academy Information and Computer Sciences (ICS), Subject Centre and Nottingham Trent University, UK

Performance-guided Neural Network for Rapidly Self-Organising Active Network
Management

A neural network architecture is introduced for the real-time learning of input sequences using external performance feedback. The target problem domain
suggests the use of Adaptive Resonance Theory (ART) networks that are able to function in a robust and fast real-time adaptive active network environment, where user requests and new proxylets (services) are constantly being introduced over time. The architecture learns, self-organises and self-stabilises in response to the user requests and maps the requests according to the types of proxylets available. However, the ART1 architecture and the original algorithm are modified to incorporate an external feedback mechanism whereby the performance of the system is fed into the network periodically. This modification, namely the ‘snap-drift’
algorithm, uses fast convergent, minimalist learning (snap) when the overall network performance has been poor and slow learning (drift towards user request input patterns) when the performance has been good. A key concern of the research is to devise a mechanism that effectively searches for alternative solutions to the ones that have already been tried, guided simultaneously by the input data (bottom-up information) and the performance feedback (top-down information). Preliminary simulations evaluate the two-tiered architecture using a simple operating environment consisting of simulated training and test data.

Lee, S. W., Palmer-Brown, D., Tepper, J., and Roadknight, C. M., (2002), Performance-guided neural network for rapidly self-organising active network management. In Proceedings of the 2nd International Conference on Hybrid Intelligent Systems (HIS2002), Chile, 2002.

Performance-guided neural network for self-organising network management

A neural network architecture is introduced for real-time learning of input sequences using external performance feedback. Some aspects of Adaptive Resonance Theory (ART) networks are applied because they are able to function in a fast real-time adaptive active network environment where user requests and new proxylets (services) are constantly being introduced over time. The architecture learns, self-organis es and self-stabilises in response to user requests, mapping the requests according to the types of proxylets available. However, in
order make the neural networks respond to performance feedback, we introduce a modification to the original ART1 network in the form of the ‘snap-drift’ algorithm, that uses fast convergent, minimalist learning (snap) when the overall network performance is poor, and slow learning (drift towards user request input pattern) when the performance is good. Preliminary simulations evaluate the two-tiered architecture using a simple operating environment consisting of simulated training and test data.

Lee, S. W., Palmer-Brown, D., Tepper, J., and Roadknight, C. M., (2002) Performance-guided neural network for self-organising network management. In Proceedings of the London Communications Symposium, LCS 2002, 9 – 10 September 2002, London, pp 269 – 272.