Chunk List: Concurrent Data Structures
CCHUNK LIST i
Chunk List – Concurrent Data Structures
Daniel J. Szelogowski University of Wisconsin – Whitewater [email protected]
ABSTRACT
Chunking data is obviously no new concept; however, I had never found any data structures that used chunking as the basis of their implementation. I figured that by using chunking alongside concurrency, I could create an extremely fast run-time in regards to particular methods as searching and/or sorting. By using chunking and concurrency to my advantage, I came up with the chunk list — a dynamic list-based data structure that would separate large amounts of data into specifically sized chunks, each of which should be able to be searched at the exact same time by searching each chunk on a separate thread. As a result of implementing this concept into its own class, I was able to create something that almost consistently gives around 20x-300x faster results than a regular ArrayList. However, should speed be a particular issue even after implementation, users can modify the size of the chunks and benchmark the speed of using smaller or larger chunks, depending on the amount of data being stored. Notes: • A full implementation can be found at https://github.com/danielathome19/Chunk-List • All code examples given are in the C • Examples are given for each primary method the class should have implemented. HUNK LIST ii
TABLE OF CONTENTS
1. Discussion....................................................................................................................................1
2. Implementation Details .............................................................................................................4
3. Complexity Analysis.................................................................................................................13
4. Usage & Examples ...................................................................................................................14
References .....................................................................................................................................17 Appendix – Method Headers & Complexity Table ...................................................................18
HUNK LIST 1
1. DISCUSSION
A chunk list is an array-based list of elements in which data is stored in inner lists of a certain capacity, allowing for easily modifiable and faster runtimes based on the number of elements being stored. A simple way to conceptualize a chunk list would be an ArrayList (dynamic array) of ArrayLists. The main list would contain the “chunks”, or ArrayLists that are not allowed to be filled past a specific capacity. Any time a “chunk” has reached capacity, a new ArrayList is added and items are added to that chunk from thereon. By doing this process and splitting our list into chunks, we can use parallel processing to our advantage. Using concurrency, we can run each chunk on a separate thread when doing tasks such as searching or removing.
This can be expressed visually as a table: as an example, a chunk list containing the numbers 1 – 50 where the chunk size is set to 10 elements. (See Figure 1)
Fig. 1:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
Relatively speaking, “chunking” as a concept can be defined similarly to its psychological definition: Chunking is a term referring to the process of taking individual pieces of information (chunks) and grouping them into larger units [1]. Essentially, we are dividing up data into multiple partitions in order to manipulate each one concurrently. One chunk may contain a large or small portion of our dataset, depending on how we want the elements to be partitioned and the amount of data being stored collectively across the entire structure. HUNK LIST 2
Chunking data for purposes of efficiency is a highly common practice. To compare, network optimization utilizes the same idea in the form of packets: On the Internet, the network breaks an [data/messages] into parts of a certain size in bytes, known as packets. Packets are used to carry ‘chunks’ of information across the internet before piecing it all together into the final product. As a result, the network becomes more efficient: the network can balance the transfer across multiple pieces of equipment rapidly, down to a millisecond basis, and if there is an issue with a piece of equipment on the network during the transfer process, packets can be routed around it to ensure the entire piece of data is delivered [2]. For additional security, each of these packets can also be encrypted individually. Using the chunk list, we can perform the same operation: if desired, one could hash items with a different encryption key for each various chunk upon insertion, for example.
The basis of the data structure makes it useful for storing very large and very small amounts of elements. Unsorted lists benefit especially: • Fast searching • Fast removal • Fast insertion In any scenario, a chunk list can be used in place of an ArrayList especially, as well as something such as a Binary Search Tree, as searching may be faster based on processing power.
Implementation is easy and short, and sorting is quick even with large amounts of chunks. With the ease of adjustability of chunk size, the capacity can be modified to allow for higher speed and efficiency. On average, the HUNK LIST 3 most optimal chunk size was tested to be 5% of the total list size, falling just ahead of the square root of the total size [4].
The most likely real-life scenario in which a chunk link would be preferred are for video games and optimization – particularly in the sense that many video games today, especially sandbox-style games such as Minecraft use a process called ‘chunking’ for map data – combining and decompiling maps into ‘chunks’ in order to load only the parts of the map within a radius of the player for the purpose of increased performance by reducing the amount of entities loaded within a visible area [5]. HUNK LIST 4
2. IMPLEMENTATION
The basis of the chunk list is the inner list. This is best implemented using some sort of dynamic list, such as ArrayList (or List in C
Example: using System; using Sytem.Collections.Generic; using System.Threading.Tasks; class ChunkList> myList; private int chunkSize; private const int DEFAULT_SIZE = 1000; public ChunkList() : this(DEFAULT_SIZE) { } public ChunkList(int chunkSize) { this.chunkSize = chunkSize; myList = new List
>(); } }
Multithreading is an especially important part of chunk list implementation, as the basis of the list’s speed is primarily the result of concurrency. For most methods in a chunk HUNK LIST 5 list, a new thread can be created for each chunk to be iterated through. A good example of this lies within C
Index-Based Methods
Accessing or modifying an element at a specified index (such as get, set, or removeAt methods) is somewhat more complex than in a regular list.
To get the chunk where the position would be located, divide the index by the chunk size and cast it to an integer: 𝑐 ℎ 𝑢𝑛𝑘 = 𝑖𝑛t ( 𝑖𝑛𝑑𝑒𝑥 / 𝑐 ℎ 𝑢𝑛𝑘𝑆𝑖𝑧𝑒 ) This method will work regardless of the current capacity of the chunk, given that we are accessing the chunk relative to the span of the list. The same applies for the index within the chunk that we need to access. Should this seem to be an issue, one could simply step down by one index to avoid a null index, or simply throw an error that the index contains no data (yet). To get the position in the chunk where the index would be, use modulo on the index by the chunk size: 𝑐 ℎ 𝑢𝑛𝑘𝑃𝑜𝑠𝑖𝑡𝑖𝑜𝑛 = 𝑖𝑛𝑑𝑒𝑥 % 𝑐 ℎ 𝑢𝑛𝑘𝑆𝑖𝑧𝑒 We can then access the data via list [chunk][chunkPosition] (Where list is the main list inside the class).
Index-Accessing Example: private int convertIndexToChunk(int index) { return index / chunkSize; } private int convertIndexToChunkPos(int index) { return index % chunkSize; }
Index Example:
The following example demonstrates accessing an element at index 8 in a chunk list containing numbers 0 – 10 with chunk size 5. (See Figure 2)
HUNK LIST 6
Accessing the chunk: int(8 / 5) = 1
Accessing the chunk position:
Fig. 2:
Chunk 0 [0] [1] [2] [3] [4] 0 1 2 3 4
Chunk 1 [0] [1] [2] [3] [4] 5 6 7 Chunk 2 [0] [1] [2] [3] [4] 10
One issue with using indices in a chunk list, however, is the problem where items flow left (step down by index until falling in place with the rest of the list, as in the style of a linked list) within the chunk but do not migrate left from one to another (pulling items from the next chunk to fill the previous) if a chunk has an open slot. To implement so may hinder performance during removal. However, a very simple solution would be to use recursion, such as within a try-catch statement using the index + 1.
Example: The following example demonstrates a solution to the problem by counting up the index until an open position is found, or throwing an error if the index is unreachable or beyond the span of the list: public T get(int index) { if (index >= size()) throw new ArgumentOutOfRangeException(); try { return myList[convertIndexToChunk(index)][convertIndexToChunkPos(index)]; } catch (ArgumentOutOfRangeException) { return get(index + 1); } }
Should our data set grow marginally larger, we may need to resize our list. To do so HUNK LIST 7 however, means we’ll need to rebalance our list, which is especially important if the chunk size we’re changing to is smaller than the current one. We can make a temporary list containing all of our old items, change the chunk size, clear our old list, and then reflow our data back in. While somewhat costly performance-wise, this is an operation that should not be necessary to occur often. If the chunk size we want to adjust to is larger than the current one, however, we can simply leave the list as is and allow the elements to re-fill the chunks that are not yet at capacity. This change is a fairly simple operation and should result in either a time of Big-O (1), or at worst Big-O (C ^ 2 * N)
Example: public void setChunkSize(int newChunkSize) { if (newChunkSize > chunkSize) { chunkSize = newChunkSize; } else { var items = getList(); chunkSize = newChunkSize; clear(); foreach (var item in items) { add(items); } } } public List
Another option which we can implement into other methods, such as adding and removing elements, is to overload the method with an optimized version which allows us to choose between setting the chunk size to Sqrt(total list size) or 5% of the total size. This new operation, however, changes our operations to Big-O (C) at best and Big-O (Sqrt(C * N) * C ^ 2 * N) at worst. HUNK LIST 8
Example: public void setChunkSize(bool optimizeSqrtSize = false) { if (optimizeSqrtSize) setChunkSize((int) Math.Sqrt(size())); else setChunkSize((int)(size() * 0.05)); }
Element Operations
We should include the standard operations equivalent to an array list. Manipulating the data will work very similarly but will focus more on the data chunks and the usage of concurrency for each method.
Adding elements to a chunk list is simple; however, it does require that we check if each chunk is at capacity. Getting the size from the chunk should be Big-O (1), at worst Big-O (Sqrt(C * N), so this should not increase runtime marginally whatsoever. An element will naturally fall into the first open spot, or the first chunk that is not at capacity. (See Figure 3) If all chunks are at capacity, however, we need to add a new chunk to our list, then add the item it. The resulting time would be processed in the time of Big-O (log C).
Fig. 3:
0 1 2 3 4
5 6 7 8 9
Chunk 0
0 1 2 3 4
Chunk 1
5 6 7 8 9
Chunk 2 10
Example: public void add(T t) { foreach (List
Another option, to avoid a large imbalance of course, is to resize the list after our add operation. We can do this with an overload and the optimized setChunkSize method we created.
Example: public void add(T t, bool optimizeSqrtSize) { add(t); setChunkSize(optimizeSqrtSize); } Performing this operation may be quick, but in rare cases may be costly. Since we rely on two operations, this method yields the potential for Big-O (C), or at worst Big-O (log C * ((Sqrt(C * N) * C ^ 2 * N)). This is unlikely, however, and in a very large dataset may be beneficial in the long run – rather than manually resizing the list through trial-and-error.
Removing elements is one of the fastest computational operations in a chunk list. This is where we can start using multithreading to our advantage. To remove an element, we can use a parallel for loop to concurrently check each chunk for the item. We can use a binary search to get the index that we’re looking for. This is also where we need to be able to have access to the thread’s state when we’re looping through each chunk. If we only want to remove the first found instance of an element, we need to immediately break out of the parallel for loop. To remove all instances of an element within the list, we can still use a parallel for loop, and just call a removeAll method on each chunk. Given a more powerful computer, these events should both be fairly fast: our average time should be Big-O ((log C * log N) / P), and at worst we are only losing our HUNK LIST 10 divisor of processor cores at Big-O (log C * log N), based on the division of threads for each removal operation. We can also call for the list to remove an item at a particular index to create a removeAt method, which should run very quickly even still at Big-O (1) or at worst Big-O (C * N - I) where I is the index. To clear the entire list, we can simply call clear on the main list (containing the chunks).
Example: public void remove(T t) { Parallel.ForEach(myList, (currentList, state) => { int indx = currentList.BinarySearch(t); if (indx >= 0) { currentList.RemoveAt(indx); state.Break(); } }); } public void removeAll(T t) { Parallel.ForEach(myList, (currentList) => { for (int i = 0; i < currentList.Count; i++) { if (currentList[i].Equals(t)) { lock (_lock) { currentList.RemoveAt(i); } i--; } } }); } public void removeAt(int index) { if (index >= size()) throw new ArgumentOutOfRangeException(); try { myList[convertIndexToChunk(index)].RemoveAt(convertIndexToChunkPos(index)); } catch (ArgumentOutOfRangeException) { removeAt(index + 1); } }
Another overload we can add for optimization is one for the removeAll method – if we remove a lot of elements from a large list, we risk an imbalance as well. An overload that rebalances the list helps to solve this issue. This operation, like the optimized addition, may be potentially costly as well, with the average case being Big-O (C * (log C * N) / P) and the worst case being Big-O (C ^ 2 Sqrt(C * N) * log C * N ^ 2). Again, this case is very unlikely though. HUNK LIST 11
Example: public void removeAll(T t, bool optimizeSqrtSize) { removeAll(t); setChunkSize(optimizeSqrtSize); }
Searching for an element is also where chunk lists shine. Once again we can use concurrency to get the shortest possible runtime, as now we can use a parallel for loop not only on the list itself, but on each chunk. Essentially, we can check most items in the list at the exact same time, meaning our runtime will be marginally smaller than using a linear search at worst case, and in the best case, a binary search. This is the result of our parallel search form: by opening each chunk on a separate thread, our goal is for one of our chunks to be successfully binary searched, even completely through without having the same linear performance for the rest of the list. Resulting is our more-likely Big-O ((log C * log N) / P), but should we have to search the entirety of every chunk for the full list, we may fall into the computational span of linear time Big-O (C * N). Of course, this should also be lessened by the number of threads opened, preventing a completely consecutive search time.
Example: public bool contains(T t) { bool found = false; Parallel.ForEach(myList, (currentList, state) => { Parallel.ForEach(currentList, (currentItem) => { if (currentItem,Equals(t)) { found = true; state.Break(); } }); }); return found; }
Sorting our list is a fairly complex operation; to properly sort our list, we do have to make a temporary list containing all elements of our chunk list. To do otherwise would only HUNK LIST 12 sort the chunks, which is not ideal as we do not know which order they will be inserted in. Using our temporary list, we can clear our main list and simply reflow all of our items back in after sorting it.
Sorting Example: For this example, I simply used the sort method implemented within C • If the partition size is fewer than 16 elements, it uses an insertion sort algorithm. • If the number of partitions exceeds 2 * log N, where N is the range of the input array, it uses a heapsort algorithm. • Otherwise, it uses a quicksort algorithm. public void sort() { var items = getList(); items.Sort(); clear(); foreach (T item in items) { add(item); } } HUNK LIST 13
3. COMPLEXITY
We can find the computational complexities by comparing those of a standard abstract list and dividing up the data based on the equivalent methods and the chunks of data as individual lists, acting as the size of each sub-list containing its own number of elements. In instances where we see C * N for example, this would represent either the entire list, or log C * log N representing a divisional portion of the data for the sake of computation complexity.
Complexities are listed with the following variables: • C being the number of chunks currently in the list. • N being the number of elements per chunk. • P being the number of processors. • I being the index input for the operation.
OPERATION AVERAGE CASE WORST CASE
Add Θ(1) Θ(log C) Remove Θ((log C * log N) / P) Θ(log C * log N) RemoveAll Θ((log C * N) / P) Θ(log C * N) RemoveAt Θ(1) Θ(C * N - I) Set Θ(1) Θ(C * N - I) Get Θ(1) Θ(C * N - I)
These methods have been computed based primarily from instant computations or the computation complexity of the base method being used (e.g. the Sort method being derived from the built-in Sort method as mentioned in ). OPERATION AVERAGE CASE WORST CASE
GetList Θ(C ^ 2 * N) N/A Contains (Search) Θ((log C * log N) / P) Θ(C * N) Size (Count) Θ(C) Θ(Sqrt( C * N) SetChunkSize Θ(1) Θ(C ^ 2 * N) Sort Θ(C * N * log N) Θ(C * N ^ 2) HUNK LIST 14
4. USAGE
The biggest potential usage for the chunk list would definitely be for video games – any time a large amount of objects or map data would need to be contained or searched through, the data structure would provide the most efficient mean to load portions of data as well as find objects within the chunks.
Unit Test & Benchmarks
A working unit test can be found on the GitHub repository [4] comparing results of data computations using a chunk list, a chunk list with the chunk size set to the square root of the data quantity, and a standard array list. There are three main Unit Tests which utilize the same method of examination. • For the Array List Test, a loop populates an array list, chunk list A, and chunk list B, with 500,000 integers between 0 and 10. Chunk list A has a chunk size of 50,000 (1/10 th the sample size) and chunk list B has a chunk size of ~707 (Sqrt(Sample Size)). • For the Macro Chunk Size Test, 11 lists are populated with 500,000 integers of 0-10 with various chunk sizes: 10, 100, 500, 1000, 2500, 5000, 10000, 25000, 50000, 100000, 50000, and Sqrt(500,000). • Lastly, for the Micro Chunk Size Test, there are three lists: a list of chunk lists (A), a list of chunk lists (B), and a list of sample sets. A for loop which runs 30 times generates a random sample size between 100 and 10,000 and two chunk lists: one of chunk size (5% of sample size, A) and one of chunk size (Sqrt(sample size), B). A nested loop generates a random integer between 0 and 10 and adds it to the lists until their count reaches the sample size. HUNK LIST 15 The examination is as follows: • Check if the list contains ‘3’ • Check if the list contains ‘6’ • Check if the list contains ‘500’ • Return the list size • Sort the list • Remove ‘7’ • Remove All ‘3’ The Array List Test proved that chunk list A ran faster than the array list, but also that chunk list B ran faster than chunk list A. The Macro Chunk Size Test proved that on average, a chunk size of 5% the sample size was the fastest performing compared to the others. Finally, the Micro Chunk Size Test proved that the majority (83+%) of the time, a chunk size of 5% was faster than a chunk size of Sqrt(sample size). All tests can be found on the GitHub page on the ‘UnitTest.cs’ file, utilizing the xUnit framework for .NET unit testing.
A lot of data structures could potentially either replace or be replaced by the chunk list – in smaller amounts of data, an array list could of course be used, or even a binary tree if searching through the data is the key important part. The biggest selling point is of course the concurrency: any time speed is the biggest factor in manipulating data, the chunk list is a likely competitor compared to a graph or binary search tree. However, that isn’t to say that the chunk list could not be integrated together with another structure – in the case that a job requires multiple of a certain data structure, such as a stack, queue, linked-list, tree, graph, etc., a chunk list could be used as a capsule for a large set of structures, and enable the ability to search through multiple of them at one time. The concurrent nature of the chunk list can provide the tools necessary for scalability even in the most mundane form of storage, such as a bit list or array, and allow faster HUNK LIST 16 means of searching for pieces of data within each structure. One could even create a two or three-dimensional chunk list, containing objects of itself (such as that of a matrix).
As mentioned previously, Minecraft’s chunking system was a drawn upon concept in development of the data structure. The idea of chunking data into smaller sets in order to access the most important parts and use different threads and processor cores to manipulate data concurrently provided a baseline to the core design of the structure. Cryptocurrencies and other mathematical challenges such as Project Euler also were key to finding an importance in speed in data manipulation, as the usage in problem solving and video game optimization were too key ideas meant to be solved through the implementation of the data structure.
HUNK LIST 17
REFERENCES
APPENDIX
Method Headers & Complexities
OPERATION AVERAGE CASE WORST CASE
Constructor () : this(DEFAULT_SIZE) N/A N/A
Constructor (int chunkSize)
N/A N/A void
Add (T t)
Θ(1) Θ(log C) void
Add (T t,, bool optimizeSqrtSize)
Θ(C) Θ(log C * ((Sqrt(C * N) * C ^ 2 * N)) void
Remove (T t)
Θ((log C * log N) / P) Θ(log C * log N) void
RemoveAll (T t)
Θ((log C * N) / P) Θ(log C * N) void
RemoveAll (T t, bool optimizeSqrtSize)
Θ(C * (log C * N)) / P) Θ(C ^ 2 * Sqrt(C * N) * log C * N^2) void
RemoveAt (int index)
Θ(1) Θ(C * N - I) int
ConvertIndexToChunk (int index)
N/A N/A int
ConvertIndexToChunkPos (int index)
N/A N/A void
Set (int index, T t)
Θ(1) Θ(C * N - I) T
Get (int index)
Θ(1) Θ(C * N - I) List
GetList () Θ(C ^ 2 * N) N/A bool
Contains (Search) (T t)
Θ((log C * log N) / P) Θ(C * N) int
Size (Count) () Θ(C) Θ(Sqrt(C * N)) void
SetChunkSize (int size)
Θ(1) Θ(C ^ 2 * N) void
SetChunkSize (bool optimizeSqrtSize=0)
Θ(C) Θ((Sqrt(C * N) * C ^ 2) * N) void