Gavin Shaw

Reliable representations for association rules

Association rule mining has contributed to many advances in the area of knowledge discovery. However, the quality of the discovered association rules is a big concern and has drawn more and more attention recently. One problem with the quality of the discovered association rules is the huge size of the extracted rule set. Often for a dataset, a huge number of rules can be extracted, but many of them can be redundant to other rules and thus useless in practice. Mining non-redundant rules is a promising approach to solve this problem. In this paper, we first propose a definition for redundancy, then propose a concise representation, called a Reliable basis, for representing non-redundant association rules. The Reliable basis contains a set of non-redundant rules which are derived using frequent closed itemsets and their generators instead of using frequent itemsets that are usually used by traditional association rule mining approaches. An important contribution of this paper is that we propose to use the certainty factor as the criterion to measure the strength of the discovered association rules. Using this criterion, we can ensure the elimination of as many redundant rules as possible without reducing the inference capacity of the remaining extracted non-redundant rules. We prove that the redundancy elimination, based on the proposed Reliable basis, does not reduce the strength of belief in the extracted rules. We also prove that all association rules, their supports and confidences, can be retrieved from the Reliable basis without accessing the dataset. Therefore the Reliable basis is a lossless representation of association rules. Experimental results show that the proposed Reliable basis can significantly reduce the number of extracted rules. We also conduct experiments on the application of association rules to the area of product recommendation. The experimental results show that the non-redundant association rules extracted using the proposed method retain the same inference capacity as the entire rule set. This result indicates that using non-redundant rules only is sufficient to solve real problems needless using the entire rule set.

Concise representations for approximate association rules

The quality of association rule mining has drawn more and more attention recently. One problem with the quality of the discovered association rules is the huge size of the extracted rule set. Often for a dataset, a huge number of rules can be extracted, but many of them can be redundant to other rules and thus useless in practice. Mining non-redundant rules is a promising approach to solve this problem. In this paper, we firstly propose a definition for redundancy; then we propose a concise representation called reliable basis for representing non-redundant association rules for both exact rules and approximate rules. We prove that the redundancy elimination based on the reliable basis does not reduce the belief to the extracted rules. We also prove that all association rules can be deduced from the reliable basis. Therefore the reliable basis is a lossless representation of association rules. Experimental results show that the reliable basis significantly reduces the number of extracted rules.

Using association rules to solve the cold-start problem in recommender systems

Recommender systems are widely used online to help users find other products, items etc that they may be interested in based on what is known about that user in their profile. Often however user profiles may be short on information and thus it is difficult for a recommender system to make quality recommendations. This problem is known as the cold-start problem. Here we investigate using association rules as a source of information to expand a user profile and thus avoid this problem. Our experiments show that it is possible to use association rules to noticeably improve the performance of a recommender system under the cold-start situation. Furthermore, we also show that the improvement in performance obtained can be achieved while using non-redundant rule sets. This shows that non-redundant rules do not cause a loss of information and are just as informative as a set of association rules that contain redundancy.

Deriving non-redundant approximate association rules from hierarchical datasets

Association rule mining plays an important job in knowledge and information discovery. However, there are still shortcomings with the quality of the discovered rules and often the number of discovered rules is huge and contain redundancies, especially in the case of multi-level datasets. Previous work has shown that the mining of non-redundant rules is a promising approach to solving this problem, with work by [6,8,9,10] focusing on single level datasets. Recent work by Shaw et. al. [7] has extended the non-redundant approaches presented in [6,8,9] to include the elimination of redundant exact basis rules from multi-level datasets. Here we propose a continuation of the work in [7] that allows for the removal of hierarchically redundant approximate basis rules from multi-level datasets by using a datasets hierarchy or taxonomy.

Enhancing an Incremental Clustering Algorithm for Web Page Collections

With the size and state of the Internet today, a good quality approach to organizing this mass of information is of great importance. Clustering web pages into groups of similar documents is one approach, but relies heavily on good feature extraction and document representation as well as a good clustering approach and algorithm. Due to the changing nature of the Internet, resulting in a dynamic dataset, an incremental approach is preferred. In this work we propose an enhanced incremental clustering approach to develop a better clustering algorithm that can help to better organize the information available on the Internet in an incremental fashion. Experiments show that the enhanced algorithm outperforms the original histogram based algorithm by up to 7.5%.

Extracting Non-redundant Approximate Rules from Multi-level Datasets

Association rule mining plays an important job in knowledge and information discovery. Often the number of the discovered rules is huge and many of them are redundant, especially for multi-level datasets. Previous work has shown that the mining of non-redundant rules is a promising approach to solving this problem, with work in focusing on single level datasets. Recent work by Shaw et. al. has extended the non-redundant approaches presented in to include the elimination of redundant exact basis rules from multi-level datasets. In this paper, we propose an extension to the work in to allow for the removal of hierarchically redundant approximate basis rules from multi-level datasets through the use of the datasetpsilas hierarchy or taxonomy. Experimentation shows our approach can effectively generate both multi-level and cross level non-redundant rule sets which are lossless.

Utilizing Non-redundant Association Rules from Multi-level Datasets

Association rule mining and recommender systems are two popular methods for obtaining knowledge and information from datasets. However, both of these methods suffer from limitations. Traditionally association rule mining has focused on extracting as many rules as possible from flat datasets. More recently, issues over the number of rules and obtaining rules from datasets with multiple concept levels have come into focus. Recommender systems have been popular with users when it comes to helping find similar interests to those they already have. However, recommender systems suffer from two major problems, cold start and novelty. The aims of our research is to develop an approach for extracting non-redundant multi-level and cross-level association rules from datasets with multiple concept levels and utilise them in a recommender system with the aim of potentially solving the cold start and novelty problems.

Interestingness Measures for Multi-Level Association Rules

Association rule mining is one technique that is widely used when querying databases, especially those that are transactional, in order to obtain useful associations or correlations among sets of items. Much work has been done focusing on efficiency, effectiveness and redundancy. There has also been a focusing on the quality of rules from single level datasets with many interestingness measures proposed. However, with multi-level datasets now being common there is a lack of interestingness measures developed for multi-level and cross-level rules. Single level measures do not take into account the hierarchy found in a multi-level dataset. This leaves the Support-Confidence approach, which does not consider the hierarchy anyway and has other drawbacks, as one of the few measures available. In this chapter we propose two approaches which measure multi-level association rules to help evaluate their interestingness by considering the database’s underlying taxonomy. These measures of diversity and peculiarity can be used to help identify those rules from multi-level datasets that are potentially useful.

Sustainability in Infrastructure Asset Management

The expectation to integrate sustainability aspects (social, environmental, and economic success) into the design, delivery, and operation of infrastructure assets is growing rapidly and globally. There are now several tools and frameworks available to benchmark and measure sustainable performance of infrastructure projects and assets. This paper briefly describes the infrastructure sustainability (IS) rating tool developed by the Australian Green Infrastructure Council (AGIC) that was launched in February 2012. This tool evaluates sustainability initiatives and potential environmental, social, and economic impacts of infrastructure projects and assets. The rating tool provides the following benefits to industry: a common national language for sustainability; a vehicle for consistent application and evaluation of sustainability in tendering processes; assists in scoping whole-of-life sustainability risks, enabling smarter solutions that reduce risks and costs; fosters resource efficiency and waste reduction, reducing costs; fosters innovation and continuous improvement in sustainability outcomes; and builds an organization’s credentials and reputation in its approach to sustainability. The infrastructure types covered by this tool include transport, energy, water, and communication. The key themes of sustainability evaluation will be briefly presented in this paper, and they include management and governance; use of resources; emissions, pollution, and waste; ecology; people and place; and innovation.

Decision Support System for Infrastructure Sustainability in Operations

There is an increasing awareness of sustainability and climate change and its impact on infrastructure and engineering asset management in design, construction, and operations. Sustainability rating tools have been proposed and/or developed that provide ratings of infrastructure projects in differing phases of their life cycle on sustainability. This paper provides an overview of decision support systems using sustainability rating framework that can be used to prioritize or select tasks and activities within projects to enhance levels of sustainability outcomes. These systems can also be used to prioritize projects within an organization to optimize sustainability outcomes within an allocated budget.