Marius Mihailescu

Iterative Data Processing on Big Data

We know that a computer application/product is scalable if it works as expected, even when its size or volume (or the size or volume of its environment) regarding data and computations has changed to improve the user’s computation necessities. In most situations, rescaling means increasing the volume or size of the computation capabilities. This is an important characteristic of cloud computing, which for big data in particular, helps because large amounts of data need to be manipulated, processed, cleaned, and analyzed; in many situations, increased computation capabilities are needed. Also, it is very important that the system run normally, even when, for example, a cluster node is down.

Haskell in Big Data

We have already discussed big data in Chapter 8. In this chapter, we provide a deeper overview of big data and its challenges. This chapter covers how data is generated, and presents some of the tools and methods used in big data. It also presents an example of MapReduce in Haskell.

Haskell in the Cloud

This chapter talks about programming in Cloud Haskell, a domain-specific language to develop programs in a distributed computing environment in Haskell. The chapter focuses on presenting the processes, messages between processes, how to use channels and ports, and closures.

Designing a Shared Memory Approach for Hadoop Streaming Performance

This chapter discusses Hadoop and Hadoop Streaming. It presents an improved model for streaming and examples of Hadoop Streaming.

Large-Scale Design in Haskell

There are approaches to manage the complexity of computations. We talked about some of them in previous chapters; here we will explain why they are used in large-scale design. We will also discuss new approaches and provide some examples.

Transactional Memory Case Studies

Safety and ease in programming are two advantages of transactional memory. If the transactions are used correctly, then it is almost impossible for problems to occur in parallel code (for example, deadlocks). The programmer mostly needs to assign transactions (and maybe some transaction variables). It is not necessary to identify the locks or their correct order to prevent deadlocks or other problems. How do you use transactions correctly? All shared data is passed through transaction variables to threads. Transactional data is accessed only through transactions; and in transactions, there are no operations that can be rolled back.

Big Data and Large Clusters

MapReduce represents a simple programming model, used in applications that generate and process large sets of data. All what the programmer needs to do is to implement the map and the reduce functions, as follows: map function processes a (key, value) pair, resulting an intermediary list of (key, value) pairs, and the reduce function takes as parameter the list resulted from map and merges all intermediary values that correspond to the same intermediary key.

Debugging Techniques Used in Big Data

In this chapter, you learn what big data means and how Haskell can be integrated with big data. You also see some debugging techniques.

Concurrency Design Patterns

In this chapter, we have chosen to present the most common problems that could occur in big data applications. One of best solutions to these problems is to use design patterns. Research contributions in functional programming continue to be made in this area, including attempts to make functional versions of OOP design patterns. Haskell is a very good programming language for big data, but some of patterns have implementations only in object-oriented programing languages. This is not an impediment for using both Haskell and design patterns, however, because they could be easily made interoperable, as you will see in this chapter. A good design pattern reference is Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides (also known as the Gang of Four) (Addison-Wesley Professional, 1994).

Strategies Used in the Evaluation Process

In programming languages, evaluation strategies represent a collection of rules that are used when expressions are evaluated or computed. The way in which arguments are passed to functions represents a particular case for evaluation strategies.