Adaptive Rate Allocation for View-Aware Point-Cloud Streaming
TTECHNICAL REPORTAdaptive Rate Allocation for View-AwarePoint-Cloud Streaming
Mohammad HosseiniUniversity of Illinois at Urbana-Champaign (UIUC)September 8, 2017
In the context of view-dependent point-cloud streaming in a scene, our rateallocation is “adaptive” in the sense that it priorities the point-cloud modelsdepending on the camera view and the visibility of the objects and their distanceas described. The algorithm delivers higher bitrate to the point-cloud modelswhich are inside user’s viewport, more likely for the user to look at, or arecloser to the view camera or, while delivers lower quality level to the point-cloud models outside of a user’s immediate viewport or farther away from thecamera. For that purpose, we hereby explain the rate allocation problem withinthe context of multi- point-cloud streaming where multiple point-cloud modelsare aimed to be streamed to the target device, and propose a rate allocationheuristic algorithm to enable the adaptations within this context. To the bestof our knowledge, this is the first work to mathematically model, and propose arate allocation heuristic algorithm within the context of point-cloud streaming.The rate selection and allocation problem is the well-known binary Knap-sack optimization problem, for which one approach to tackle is to transmit asubset of the whole point clouds within the 360-degree environment. The bi-nary Knapsack problem is NP-hard, but efficient approximation algorithms canbe utilized (fully polynomial approximation schemes), so this approach is com-putationally feasible. However, using this method only a subset of the wholepoint clouds are selected, which is not desired since the user intends to receive1 a r X i v : . [ c s . MM ] N ov igure 1: An example point cloud prioritization in a viewport. Point clouds closer tothe camera are assigned highest priority ( C ) and therefore higher qualityrepresentation, while point clouds farther away are assigned lowest priority ( C ), andtherefore, lower quality representation. all the necessary point clouds. Our proposed algorithms select all necessarypoint clouds, however, with different bitrates according to the models’ prioritiesgiven the user’s view. This is a multiple-choice knapsack problem (MCKP) inwhich the items (point cloud models in our context) are organized into groupscorresponding to the items. Each group contains higher bitrate point cloudscorresponding to a model and lower-bitrate versions of the same point cloudmodel given the adaptation manifest that we have designed.There are n point cloud models T = { τ , τ , . . . , τ n } in the 360-degree en-vironment. The highest possible representation of each τ h ∈ T has a bitraterequirement of s τ h and a priority or importance coefficient of p τ h given it’s viewor distance. With view awareness feature, our algorithm assigns highest priority( C ) to the point clouds more important for the user’s view (within the user’simmediate viewport or closer), and lowest priority ( C ) to the point clouds (ei-ther outside of user’s immediate viewport or farther away). Figure 1 illustrateshow our prioritization approach is applied against point cloud models inside a3D scene. We assume the quality contribution of a point-cloud τ h is a simplefunction q τ h = p τ h × s τ h . The available bandwidth in every interval limits thetotal bitrate of all point clouds that can be received at the receiving device to2 , which serves as an available bandwidth budget. Our rate allocation heuristicis general, and can be employed based on any number of priority classes, or anytype of available resources, such as energy budget or available CPU processingpower. In this pilot study, we use three priority classes, and assume W to bethe available bandwidth.Let X = { x , x , . . . , x n } , be the set of point clouds which are received atthe device, serving as the output of running rate allocation algorithm. Each x i ∈ X corresponds to an original point cloud τ i ∈ T . Similarly, each x i has apriority coefficient p x i = p τ i depending on the view.We assume there are L number of point clouds representations availablegiven the manifest, with a representation of level k noted as R k (0 ≤ k ≤ L )and the bitrate of a point cloud τ i with representation R k noted as s R k τ i . Weassume the lowest bitrate corresponds to the representation with the highestID (i.e. R L ) which is determined as the minimum bitrate that is acceptablefor a user. In a similar way, the quality contribution of a point cloud x i is q x i = p x i × s x i . Let S be the total bitrate requirement of all point clouds, and W be the availablebandwidth budget. The minimum quality acceptable for users is given as therepresentation of level L noted as R L . Let C , C , and C be the class ofpoint clouds with the highest priority, medium priority, and lowest priority,respectively.For each point cloud τ i in T , we calculate q i as described previously. Thisis the contribution that τ i would make to the average quality of the 3D worldsystem if it were received at highest bitrate possible. We then calculate W min = (cid:80) s R L τ i which is the minimum bitrate that is needed to receive all point cloudsat their lowest bitrates. In the following, assume that W min ≤ W so the unusedbitrate budget would be W = W − W min .To determine the best bitrate for each point cloud, our algorithm sorts theprioritized list of point clouds by the global priority from the largest to thesmallest. For ease of notation in the following, suppose that the point clouds arere-indexed so that the sorted list of point clouds is τ , τ , . . . , τ n . If s τ − s R L τ ≤ W then there is enough unused budget to receive τ at highest bitrate ( R ), sothe point cloud x would have s x = s τ and would contribute q to the averagequality. This leaves an unused bandwidth budget of W = W − s R τ − s R L τ forthe remaining point clouds after x . The algorithm repeats for τ , τ , . . . untilsome point clouds τ (cid:96) cannot be received at highest bitrate within the remaining3ate allocation heuristic algorithm for point cloud streaming T : prioritized list of point clouds sorted from highest to lowest priority τ i : point cloud with highest bitrate s τ i x i : adapted point cloud with bitrate s x i L : Number of representation levels R L : Level L representationCalculate W min = (cid:80) s R L τ i %comment: minimum bitrate requirement for allpoint clouds ∀ τ i ∈ T : s x i ← s R L τ i %comment: assign R L (minimum bitrate) to all τ i ’s. W ← W − W min %comment: initialization while s τ i − s R L τ i ≤ W i − do %comment: i=1 initially. s x i ← s τ i W i ← W i − − ( s τ i − s R L τ i ) i ← i + 1 %comment: adapt next point cloud end while %comment: above loop repeats until a point cloud τ (cid:96) cannot be delivered athighest bitrate within the remaining bandwidth budget W (cid:96) − . (cid:96) ← i %comment: resulting from above loop.Find lowest L (cid:48) ≤ L such that s R L (cid:48) τ (cid:96) ≤ W (cid:96) − + s R L τ (cid:96) %comment: determines the highest bitrate possible atwhich τ (cid:96) can be received within remaining budget, by calculating the lowestrepresentation level L (cid:48) . s x (cid:96) ← s R L (cid:48) τ (cid:96) %comment: adapt τ (cid:96) and calculate s x (cid:96) budget W (cid:96) − . It then determines the highest possible bitrate at which it canbe received by calculating the lowest representation level L (cid:48) : L (cid:48) ≤ L such that s R L (cid:48) τ (cid:96) ≤ W (cid:96) − + s R L τ (cid:96) . The point cloud x (cid:96) will have bitrate s x (cid:96) = s R L (cid:48) τ (cid:96) and willcontribute q (cid:48) (cid:96) to the average quality of the whole. The remaining bandwidthbudget after streaming x (cid:96) will be W (cid:96) = W (cid:96) − − s R L (cid:48) τ (cid:96) . The algorithm repeatsthis process to determine the proper bitrates, amount of bandwidth budget, andquality contribution for each of the remaining point clouds x (cid:96) +1 , x (cid:96) +2 , . . . , x n .The algorithm needs a one-time implementation in the beginning of the ses-sion for the main process. Therefore it is implemented in real-time and doesnot provide any additional overhead during the runtime. It is implemented effi-ciently in O ( nlogn ) time and O ( n ) space and produces solutions close optimal.The approximation error depends on the difference between the bitrate chosenfor the first point cloud that cannot be received at highest bitrate (i.e. τ (cid:96) ) andthe remaining budget available to receive it.4 References • M. Hosseini and C. Timmerer, ”Dynamic Adaptive Point Cloud Stream-ing”, In Proceedings of the 23rd Packet Video Workshop, 2018. ACM,New York, 1–7. • M. Hosseini, V. Swaminathan, ”Adaptive 360 VR video streaming: Divideand conquer!”, IEEE International Symposium on Multimedia (ISM), SanJose, 2016. ••