Centralized and Distributed Age of Information Minimization with non-linear Aging Functions in the Internet of Things
aa r X i v : . [ c s . I T ] J a n Centralized and Distributed Age of Information Minimization withnon-linear Aging Functions in the Internet of Things
Taehyeun Park , Walid Saad , and Bo Zhou Wireless@VT, Bradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, USA,Emails: { taehyeun, walids, ecebo } @vt.edu Abstract —Resource management in Internet of Things (IoT)systems is a major challenge due to the massive scale andheterogeneity of the IoT system. For instance, most IoT appli-cations require timely delivery of collected information, which isa key challenge for the IoT. In this paper, novel centralized anddistributed resource allocation schemes are proposed to enableIoT devices to share limited communication resources and totransmit IoT messages in a timely manner. In the consideredsystem, the timeliness of information is captured using non-linearage of information (AoI) metrics that can naturally quantify thefreshness of information. To model the inherent heterogeneityof the IoT system, the non-linear aging functions are definedin terms of IoT device types and message content. To minimizeAoI, the proposed resource management schemes allocate thelimited communication resources considering AoI. In particular,the proposed centralized scheme enables the base station to learnthe device types and to determine aging functions. Moreover, theproposed distributed scheme enables the devices to share thelimited communication resources based on available informationon other devices and their AoI. The convergence of the proposeddistributed scheme is proved, and the effectiveness in reducingthe AoI with partial information is analyzed. Furthermore, theproposed resource management schemes with different numberof devices, activation probabilities, and outage probabilities areanalyzed in terms of the average instantaneous AoI. Simulationresults show that the proposed centralized scheme achievessignificantly lower average instantaneous AoI when compared tosimple centralized allocation without learning, while the proposeddistributed scheme achieves significantly lower average instanta-neous AoI when compared to random allocation. The resultsalso show that the proposed centralized scheme outperformsthe proposed distributed scheme in almost all cases, but thedistributed approach is more viable for a massive IoT.
Index Terms —Age of Information, Internet of Things, RadioResource Management
I. I
NTRODUCTION
The Internet of Things (IoT) is arguably the most importanttechnology of the coming decade [1]. However, the effectiveoperation of several IoT services, such as industrial moni-toring [2], health monitoring [3], drones [4], virtual reality[5], and vehicular network [6], requires timely and frequentcommunications. To maintain the proper performance of suchdiverse IoT applications, the base station (BS) must maintainthe most relevant information gathered from the IoT devicesat any given time.In addition to timely transmissions from the devices to theBS, another key challenge is to account for the distinctivecharacteristics of an IoT and its devices. One prominentproperty of an IoT is its massive scale as the number of devices
This research was supported by the Office of Naval Research (ONR) underMURI Grant N00014-19-1-2621. greatly outnumbers the available communication resources [7].Therefore, an appropriate allocation of the limited communi-cation resources among numerous IoT devices is necessary forthe deployment of an IoT and its services [8]. Furthermore,the IoT exhibits a high heterogeneity in terms of devicetypes, functions, messages, transmission requirements, andresource constraints [9]. The aforementioned IoT propertiespose challenges for timely uplink transmission in an IoT.To ensure the performance of time-sensitive IoT applicationsdespite the aforementioned challenges, a new informationtimeliness performance metric is needed as an alternative toconventional delay, reliability, and data rate.To evaluate the communication between the BS and the de-vices, the age of information (AoI), which is a metric that canquantify the relevance and the freshness of the information, isused [10], [11]. However, the AoI has different characteristicscompared to delay [10], because it explicitly considers packetgeneration time. The problem of AoI minimization in an IoThas unique challenges due to the characteristics of an IoT,including massive scale, limited communication resources, andIoT device heterogeneity. Largely, AoI minimization can bedone in a centralized way or in a distributed way. However, acentralized AoI minimization approach is not always viable foran IoT, because the energy constrained IoT devices may not beable to communicate frequently with BS. On the other hand, adistributed AoI minimization approach may require extensivedevice-to-device communication and could perform worsethan a centralized solution for some IoT scenarios. Therefore,both centralized and distributed AoI minimization must beinvestigated to compare their applicability and performancesin an IoT.
A. Existing Works
A number of recent works studied the problem of AoIminimization in wireless networks [10]–[34]. These studiesuse various approaches to minimize the AoI under differentconstraints and conditions. For instance, the works in [10]–[33] study a variety of scheduling policies for AoI mini-mization in different networks, including single hop broadcastnetwork [12], single-hop uplink communication [13], multi-hop uplink communication [14], ad-hoc networks [15], andALOHA-like random access [16]. The authors in [17] and[18] propose and analyze scheduling policies for the wirelessnetworks with known and unknown channel state information.The works in [10], [13], [15], and [19] introduce effectivescheduling policies to minimize the average AoI with networkconstraints, such as throughput requirement, physical con-1traint for sensing, spectrum sharing, and varying packet sizes.In [15], [20], and [21], the authors use online techniques, suchas reinforcement learning to perform AoI-minimal scheduling.Moreover, the authors in [21] and [22] analyze the perfor-mance of user scheduling for minimizing the average AoIin presence of multiple sources of information and proposea hybrid queueing system. The authors in [23] analyze thecoexistence of DSRC and WiFi networks as a game, in whichthe DSRC network minimizes the AoI and the WiFi networkmaximizes the throughput. For CSMA networks, the work in[24] optimizes the backoff time of each communication link tominimize the total average AoI, and the authors in [25] proposea sleep-wake scheduling to optimize the tradeoff between AoIminimization and energy consumption.The works in [26]–[29] address the problem of AoI min-imization using queueing-theoretic approaches. In [26], theauthors analyze the peak AoI in a multi-class queueing systemwith packets having heterogeneous service times and require-ments. The authors in [27]–[29] derive closed-form solutionsfor the average AoI and the peak AoI for different queueingmodels, including M/M/1, M/G/1, and M/G/1/1. In [30], theauthors consider a vehicular network with ultra-reliable low-latency communication and minimize the tail of the AoIdistribution. The peak AoI considering the packet deliveryfailure is analyzed in [31]. Moreover, the non-orthogonal mul-tiple access is compared against the conventional orthogonalmultiple access in terms of AoI minimization in [32]. Theauthors study the sampling policies to minimize the averageAoI with the joint status sampling in IoT [11] or with thenon-linear aging functions [33].Despite being interesting, the existing solutions in [10]–[34] do not consider some of the unique properties of anIoT, such as limited communication resources, massive scale,and high device heterogeneity. One of the key challenges inIoT is the massive scale of IoT coupled with highly limitedavailable communication resources. However, the works in[10], [12]–[16], [20], and [31]–[33] do not investigate therealistic IoT scenario in which the number of devices greatlyoutnumbers the communication resources. Furthermore, theinherent heterogeneity among IoT devices and the presenceof non-linear aging functions are not considered in [11]–[20],and [27]–[32]. Moreover, most of the prior works for AoIminimization in [10], [12]–[18], [21]–[29], and [33] only con-siders a centralized approach. Moreover, in [34], we studiedcentralized AoI minimization with non-linear aging functionsand proposed a centralized resource allocation scheme toenable the BS to consider different aging functions. However,the work in [34] only investigates a centralized approachfor AoI minimization and does not introduce a distributedresource allocation framework for an IoT with non-linear agingfunctions. A centralized approach may not always be suitablefor an IoT, because the frequent communication with BS is notviable for the energy constrained IoT devices. These importantchallenges for enhancing the AoI in an IoT have been largelyoverlooked in prior works [10]–[34].
B. Contributions
The main contributions of this paper are novel centralizedand distributed resource allocation frameworks that can beused to minimize the average instantaneous AoI for a massiveIoT with heterogeneous devices and non-linear aging func-tions. In particular, we capture the heterogeneity among IoTdevices using non-linear aging functions . Typically, the AoIis defined only in terms of time, and it is assumed to increaselinearly with a slope of [10]. However, the definition ofthe AoI can be broader such that the AoI can be a functionof completeness, validity, accuracy, currency, and utility [35],[36]. Under such a broader definition of the AoI, the agingfunction can be defined as an age penalty function or anage utility function [33], which can be an exponential, linear,or step function [35]. As such, we propose to capture theheterogeneity among IoT devices and messages by assigningdifferent aging functions based on the devices types, theIoT application, the message content, and the transmissionrequirement.For centralized AoI minimization, we propose a new priorityscheduling scheme with a learning perspective such that thedevice types and the aging functions can be determined. Fornon-linear aging functions, we show that using the future AoIfor priority scheduling achieves a lower average instantaneousAoI than using the current AoI. Simulation results for thecentralized approach show that the proposed priority schedul-ing scheme achieves . lower average instantaneous AoIwith high activation probability and . lower averageinstantaneous AoI with high outage probability than a simplepriority scheduling. In particular, our approach outperforms asimple priority scheduling and performs similar to a priorityscheduling with complete information on device types andaging functions.For the distributed AoI minimization, we formulate a mi-nority game [37], such that massive number of IoT devicescan share the limited available communication resources au-tonomously. Furthermore, a payoff function is designed toallow the messages with the highest AoI to transmit firstin a self-organizing manner. We then show the conditionsthat a resource allocation among IoT devices must satisfy toachieve a Nash equilibrium (NE). We propose a stochasticcrowd avoidance algorithm for the resource allocation gameand prove that the resource allocation using our proposedalgorithm converges to an NE with sufficient information andunder certain network parameters. Simulation results for thedistributed case show that the proposed algorithm is effectivein minimizing the AoI even if the devices only have thepartial information about other devices. The results showthat our game-based approach achieves . lower averageinstantaneous AoI with limited information and . loweraverage instantaneous AoI with high outage probability than arandom resource allocation. Moreover, after convergence, ourgame-based approach performs similar to the pre-determinedresource allocation with complete information.The centralized and the distributed AoI minimization2chemes are compared in terms of overhead, implementation,and requirements. In particular, the centralized AoI minimiza-tion has an overhead of uplink communication request, whilethe distributed AoI minimization has an overhead of device-to-device communication. Simulation results show that thedistributed AoI minimization achieves -fold higher averageinstantaneous AoI than the centralized AoI minimization in amassive IoT, where the communication resources are highlylimited and the devices only have partial information. In a lessconstrained IoT, simulation results show that the distributedAoI minimization achieves -fold higher average instanta-neous AoI than the centralized AoI minimization. Althoughthe centralized AoI minimization outperforms the distributedAoI minimization in terms of average instantaneous AoI, thedistributed approach may be more suitable for an IoT, becausethe centralized approach may not be practical or viable foran IoT. As such, our analysis clearly showcases the contrastsbetween the two solutions.The rest of this paper is organized as follows. Section II in-troduces the system model and the non-linear aging functions.Section III analyzes the AoI minimization with coexistence oflinear and non-linear aging functions. Section IV presents thecentralized and the distributed resource allocations in an IoT.Section V analyzes the simulation results, while Section VIdraws conclusions. II. S YSTEM M ODEL
Consider the uplink of a wireless IoT system consisting ofone BS serving N IoT devices. The IoT devices can transmittheir messages to the BS using the communication resourcesallocated by either a centralized or distributed resource al-location scheme. To transmit to the BS, the IoT devicesuse time-slotted orthogonal frequency-division multiple access(OFDMA). Here, R time-frequency resource blocks (RBs)are allocated to the IoT devices at each time slot. If morethan one IoT device use a given RB, none of the messagestransmitted using the given RB can be successfully decoded,which leads to transmission failures. This implies that at most R devices can transmit successfully to the BS at a time slot. Inan IoT where the number of devices N greatly outnumbers thenumber of RBs R , RB allocation is critical for the operationof IoT, and RB allocation can be done in a centralized or ina distributed way.Under a centralized resource allocation scheme, the BSallocates the RBs to the IoT devices such that a given RBis used by only one device. Therefore, duplicate RB usagewill not occur when using a centralized resource allocation.However, centralized resource allocation incurs an overheadrelated to the need that the devices request their uplink com-munication resources via a random access channel (RACH)[38]. Furthermore, the uplink communication resource requestusing RACH can fail resulting in transmission failure. Incontrast, when using a distributed resource allocation scheme,the devices decide which RB to use autonomously without anyintervention from the BS and without RACH. Although thereis no overhead related to the need for requesting uplink com- munication resources, distributed resource allocation incurs anoverhead related to the devices cooperating to avoid duplicateRB usage. Since there is no RACH request when performingdistributed resource allocation, no RACH request failures willhappen. However, the uplink transmission may fail because ofa duplicate RB usage.For both centralized and distributed resource allocations, theuplink transmission can also fail because of the RB outage.The RB outage is based on the signal-to-noise ratio (SNR)such that the transmission is considered to be a failure ifthe SNR is less than a given threshold ǫ ≥ . We considera stationary Rayleigh fading channel with additive whiteGaussian noise (AWGN), such that the statistical properties ofchannel do not change over time. Therefore, the SNR outageprobability is Pr ( S / σ ≤ ǫ ) , where the received signal power S is an exponentially distributed random variable and σ isthe variance of the AWGN. We assume that the IoT devicesonly know the distributional properties of the channel andthe AWGN at the receiver. Furthermore, we assume that alldevices transmit with the same transmit power as the devicesdo not know exact channel gain [5], [8], and [39]–[41]. Sincewe consider a Rayleigh fading channel, the received signalpower S is exponentially distributed, and the devices knowthe mean λ − of S . We assume that the transmit powers ofall IoT devices are equal, and, thus, the mean of the receivedsignal power will be the same for all devices. When an IoTdevice uses multiple RBs simultaneously, then the transmitpower will be equally divided among those RBs. For instance,if an IoT device i uses R i,t RBs simultaneously at time slot t , then the received signal power is S / R i,t . For an IoT device i using R i,t RBs simultaneously at time slot t , the outageprobability p i,t for device i at time slot t will be: p i,t = Pr (cid:18) S / R i,t σ ≤ ǫ (cid:19) . (1)Since S is exponentially distributed with mean λ − , S / Ri,t σ isexponentially distributed with mean ( R i,t σ λ ) − for a given R i,t and a known σ . Moreover, the outage probability p i,t can be interpreted as a cumulative distribution function of anexponential distribution. Since an exponential random variable S with mean λ − has a cumulative distribution function of Pr (cid:0) S ≤ ǫ (cid:1) = 1 − exp ( − λǫ ) for ǫ ≥ , the outage probability p i,t in (1) will be: p i,t = 1 − exp (cid:0) − (cid:0) R i,t σ λ (cid:1) ǫ (cid:1) . (2)For a successful uplink transmission, an RB must be used byonly one device, and the SNR must be higher than a giventhreshold ǫ .One prominent feature of an IoT is its massive scale. Inparticular, the number of IoT devices N greatly outnumbersthe number of RBs R . For an IoT scenario where N > R , theproblem of RB allocation among the IoT devices becomesmore challenging. Another prominent feature of an IoT isthe heterogeneity among the IoT devices. The IoT devicesare heterogeneous in terms of message types, transmission3equirements, and packet content. A metric that can be used todetermine which R out of the N devices will transmit and toquantify the freshness of the information in perspective of thedestination is AoI. Furthermore, the heterogeneity among theIoT devices and their messages can be captured by extendingthe definition of AoI to include the quality of information andintroducing non-linear aging functions. A. Age of Information
The AoI is a metric that quantifies the freshness of theinformation in the perspective of a destination [11], and thedefinition of the AoI is the time elapsed since the generationof a message that is most recently received at the BS. In priorart on the AoI, devices are commonly assumed to generatethe messages at will [42] and to update BS with a newmessage just in time [43]. The generate-at-will model forAoI implies that IoT devices can have messages to transmitto the BS at any time, and the just-in-time model for AoIimplies that IoT devices transmit new message to the BSimmediately after the successful transmission of the currentmessage. However, for an IoT, the devices are not always active and do not always have the messages to transmit tothe BS. In our model, a device has a message to transmitto the BS at a given time slot with an activation probability v a . If a device transmits to the BS unsuccessfully at a giventime slot, then the device retransmits the message immediatelyat following time slot without a random backoff time. Forthe distributed RB allocation, the proposed game is designedto give incentive to devices with low AoI to not transmit.The RACH phase of the centralized RB allocation can usethe game formulated for distributed RB allocation to achievea lower average instantaneous AoI. Meanwhile, the randombackoff time used in [23] and [44] does not consider differentaging functions, when the backoff time is determined for eachdevice. For instance, a long backoff time can be assigned to adevice with higher AoI and exponential aging function, whilea short backoff time can be assigned to a device with low AoIand linear aging function.Minimizing the AoI implies that the destination maintainsfresh information from the source. However, minimizing theAoI is different from simply minimizing delay [10]. TheAoI is measured by an aging function, and it is typicallyassumed that all devices have the same, linear aging functionwith a slope of [10]. However, by using different agingfunctions for different messages, the AoI can naturally capturethe inherent heterogeneity among IoT devices and messages.For instance, depending on the device type and the messagecharacteristics, the aging function can be assigned appropri-ately. If the device is a simple sensor transmitting a time-insensitive update messages, the appropriate aging function isa linear aging function. On the other hand, if the device isan industrial monitoring sensor transmitting a time-sensitivestatus report, the appropriate aging function is an exponentialaging function. By using different aging functions, the AoIcaptures both the freshness of information and the value ofinformation [33], [35], [36]. To model the heterogeneous messages, we consider thecoexistence of linear aging function and exponential agingfunction. In particular, with a linear aging function, the AoIfrom IoT device i at the beginning of time slot t ∈ Z + is: a i ( t ) = t − δ i ( t ) , (3)where δ i ( t ) is time slot at which the most recent messagefrom device i received by the BS was generated. With anexponential aging function, the AoI from IoT device i at thebeginning of time slot t ∈ Z + is: b i ( t ) = 2 t − δ i ( t ) − . (4)Although a specific linear and exponential aging functionsare considered, our proposed centralized and distributed ap-proaches to minimize the AoI are not limited to these agingfunctions only, and any type of aging function can be used.In a scenario where all IoT devices have the same linearaging function with slope , the IoT devices and the BScan easily determine the AoI of the messages by simplycounting the number of time slots passed since the mostrecently received message was generated [10]. However, inour system model where different aging functions coexist, theaging function of a given message is determined by the contentof the message [35]. For instance, the aging function of amessage whose content is critical would be the exponentialaging function b i ( t ) , while the aging function of a messagewhose content is normal would be the linear aging function a i ( t ) [33]. This implies that the BS cannot determine theaging function directly before receiving it, and, thus, theBS cannot compute the AoI of the messages. Therefore, thedevices determine the aging function of their own messageand compute the current AoI.To capture the heterogeneity among the IoT devices, theIoT devices are classified into different types based on theprobabilistic properties of their messages. A typical IoT devicewould not always have a time-sensitive message to send toBS. Additionally, a device that usually sends time-insensitivemessages may sometimes have a critical message to send. Inour model, we consider two types of devices. Type devicesare more likely to have linearly aging messages than expo-nentially aging messages. In other words, type devices havethe messages with aging function a i ( t ) with probability m and have messages with aging function b i ( t ) with probability (1 − m ) with > m > . . Type devices are morelikely to have exponentially aging messages than linearly agingmessages. In other words, type devices have messages withaging function b i ( t ) with probability m and have messageswith aging function a i ( t ) with probability (1 − m ) with >m > . . We assume that the characteristics of devices types,such as m and m , are known to the BS, but the BS doesnot know the type of a given device. Although having differenttypes of messages realistically models the heterogeneity of theIoT devices, this makes the RB allocation more challenging,because the messages transmitted by a given device may havedifferent aging functions. With non-linear aging functions andheterogeneous device types, the problem of AoI minimization4s different from AoI minimization with only linear agingfunction and homogeneous devices. Hence, next we investigatethe problem of non-linear AoI minimization with coexistenceof different aging functions and heterogeneous devices.III. N ON - LINEAR A O I M
INIMIZATION
To minimize the average instantaneous AoI, the devices withhighest AoI are permitted to transmit to the BS, and the AoIof different devices are compared to decide which devicesare allocated the RBs in a massive IoT. Without coexistenceof different aging functions, comparing AoI from differentdevices to allocate the limited RBs is simple. If all deviceshave same linear aging function a i ( t ) , then a i ( τ ) > a h ( τ ) implies a i ( τ + β ) > a h ( τ + β ) for any positive integers τ and β given that devices i and h do not transmit successfully tothe BS. Therefore, in a massive IoT with N > R , comparingcurrent AoI with t = τ at time slot τ can be used to decidewhich R out of N devices transmit and to minimize theaverage instantaneous AoI. Furthermore, using current AoIwith t = τ or using future AoI with t = τ + β at time slot τ isequivalent, when all devices have same linear aging function a i ( t ) .When a linear aging function a i ( t ) and an exponential agingfunction b i ( t ) coexist, it is insufficient to only compare the AoIof different devices to minimize the average instantaneous AoI,and RBs must be allocated considering the aging functions tominimize the average instantaneous AoI. For instance, evenif a device i with b i ( t ) has lower AoI than a device h with a h ( t ) , device i will eventually have higher AoI than device h ,because b i ( t ) increases faster than a h ( t ) . Therefore, the AoIcomparison must take aging functions into account, and oneway to consider the aging functions is to compare the futureAoI.The optimization problem to minimize the average instan-taneous AoI at time slot τ is: min n N X i ∈ n z ′ i ( τ ) + X i n ,i ∈ N z i ( τ ) (5)s.t. n ⊆ N , (6) | n | = R, (7)where N = { , · · · , N } is set of all devices, z ′ i ( τ ) is agingfunction of device i such that δ i ( τ ) = δ i ( τ − is updated, and z i ( τ ) is aging function of device i such that δ i ( τ ) = δ i ( τ − . n is a set of all devices allocated the RBs at ( τ − , while N − n is a set of all devices not allocated the RBs at ( τ − .Next, for the problem of instantaneous AoI minimization, weprove that performing RB allocation based on the future AoIachieves a lower average instantaneous AoI than an alternativethat is based on the current AoI. Proposition 1.
In a massive IoT where
N > R , if there aredifferent aging functions a i ( t ) and b i ( t ) , comparing the futureAoI with t = τ + β for some positive integer β at time slot τ to determine the RB allocation achieves a lower averageinstantaneous AoI than comparing the current AoI with t = τ . Proof.
See Appendix A. (cid:4)
From Proposition 1, we observe that allocating RBs to thedevices with highest future AoI results in a lower overallaverage instantaneous AoI of all devices at all time slots thanallocating RBs to the devices with highest current AoI. It isimportant to note that the current AoI at the time slot ofsuccessful transmission is used to compute the average in-stantaneous AoI, and the future AoI is only used to determinethe RB allocation. In Proposition 1, the value of the positiveinteger β in determining which future AoI to use for the RBallocation is a design parameter, and different values of β have different effects. If higher values of β are used, deviceswith exponentially aging messages are more likely to beallocated the RBs than devices with linearly aging messages.This implies that devices with exponentially aging messagesare allocated the RBs even when their current AoI is low,while devices with linearly aging messages are not allocatedthe RBs even when their current AoI is high. Therefore, theexponentially aging messages are being transmitted beforethe linearly aging messages. In other words, with highervalues of β , the average instantaneous AoI of IoT deviceswith exponentially aging messages is lower, while the averageinstantaneous AoI of IoT devices with linearly aging messagesis higher. Furthermore, using higher values of β for the futureAoI does not necessarily achieve a lower average instantaneousAoI, and β can be chosen depending on how much theexponentially aging messages are prioritized over the linearlyaging messages. In our system model, the centralized and thedistributed RB allocations use future AoI with β = 1 .We consider that the IoT devices may have messagesrequiring multiple RBs to successfully transmit to the BS.When the messages take multiple RBs to successfully trans-mit, δ i ( t ) in (3) and (4) will represent the time slot duringwhich the most recent message from device i , which is fullyreceived by BS, was generated. In this case, the devices mustdetermine how to transmit the messages taking multiple RBs.In particular, if a device i has a message that requires n i RBs totransmit, then the message may be transmitted simultaneouslyby using n i RBs at a given time slot, consecutively by using RB each time for n i time slots, or jointly by using bothsimultaneous and consecutive transmissions. The simultaneoustransmission may complete the transmission at once reducingthe AoI, but it has high R i,t and outage probability p i,t (2). Aconsecutive transmission achieves its lowest outage probability p i,t with R i,t = 1 , but it takes the largest number of timeslots to completely transmit increasing the AoI. Therefore,the problem of AoI minimization must consider the outageprobability. The optimization problem to minimize the averageinstantaneous AoI for a device i with a linearly aging messagerequiring n i RBs at time slot τ is: min R a i τ + τ + n i − X j = τ (1 − p i,j ) − , (8)s.t. X τ + n i − j = τ R i,j = n i , (9)5 ≤ R i,j ≤ R ∀ j, (10)where R = [ R i,τ , ..., R i,τ + n i − ] . Since R i,τ ∈ Z + , | R | = n i ,and n i is typically small for the IoT devices [45], the so-lution space of optimization problem (8) is finite and small.Therefore, the optimization problem can be solved easily usingany discrete optimization method, such as combinatorial opti-mization. In particular, for any n i , the optimization problem(8) can be mapped as a directed graph with (1 − p i,j ) − as weights to find a shortest path. When a device uses R i,t RBs simultaneously, (1 − p i,t ) is the probability of successfultransmission given that duplicate RB selection did not occur.Taking (1 − p i,t ) as the success probability in a geometricdistribution, the expected number of time slots needed for thesuccessful transmission when using R i,t RBs simultaneously is (1 − p i,t ) − , which is the mean of the geometric distribution.Therefore, P τ + n i − j = τ (1 − p i,t ) − is the expected number oftime slots needed to transmit a message requiring n i RBs. Ifa device i has an exponentially aging message, b i ( t ) replaces a i ( t ) in (8). The solution to the optimization problem (8)determines the number of RBs R i,τ that a device i shouldbe allocated with at time slot τ so that the average instan-taneous AoI is minimized. Furthermore, the solution to theoptimization problem is used for centralized and distributedapproaches for the RB allocation.IV. R ESOURCE B LOCK A LLOCATION
In a massive IoT with
N > R , RB allocation is achallenging problem especially given the high heterogeneityamong IoT devices and the messages. RB allocation in anIoT can be done in a centralized way or in a distributed way.Moreover, RB allocation schemes can achieve a lower averageinstantaneous AoI by allocating the limited RBs to IoT deviceswith higher future AoI. Centralized RB allocation schemeis based on a priority scheduling improved with maximumlikelihood to determine the aging functions and to learn thedevice types. The proposed distributed RB allocation schemein Algorithm 1 is designed to enable only the devices withsufficiently high future AoI to transmit, and the proposedstochastic crowd avoidance algorithm is proved to convergeto an NE of the formulated game.
A. Centralized RB Allocation
For centralized RB allocation, the BS allocates the RBsto the IoT devices. In a time slot. the centralized approachhas two phases. The active devices request for the RB usingRACH in the first phase. If an active device is allocated anRB, the active device transmits its message to the BS in thesecond phase. Although there will be no duplicate RB usagecausing a transmission failure in the second phase, there maybe RACH preamble collision causing an RB request failurein the first phase. Therefore, using centralized RB allocationscheme, a device fails to transmit to the BS because of theRACH preamble collision, the outage based on SNR, and thelack of RB allocation.We let P be the number of RACH preambles and N t be the number of active devices at time slot t . The probability ofthe RACH preamble collision c t at time slot t is: c t = 1 − (cid:18) P − P (cid:19) N t − , (11)which is the probability of more than one active device usinga given RACH preamble. If a device fails to request for anRB at time t with probability c t , then this is equivalent toa transmission failure. However, if a device i successfullyrequests for an RB at time t , IoT device i sends the informationabout its current AoI C i and the necessary number of RBs R i,t at time slot t . After gathering the information from the devices,the BS determines the RB allocation based on the future AoIof devices to minimize the average instantaneous AoI.In the second phase, the BS allocates the RBs to the activedevices using a priority scheduling based on the future AoI.The priority scheduling allocates the R RBs to at most R activedevices with the highest future AoI, minimizing the averageinstantaneous AoI. If a device i with high future AoI has R i,t > , then IoT device i may be allocated more than RBat time slot t . The problem in the second phase is determiningthe future AoI using the received current AoI because of thecoexistence of different aging functions. In other words, theBS does not know the aging function of an active device i ,and the BS must determine the aging function to computethe future AoI F i , which is used for the priority schedulingscheme to achieve a lower average instantaneous AoI as shownin Proposition 1. However, the BS can determine the currentaging function of an active device i from C i .The BS can determine that the aging function of an activedevice i is a i ( t ) , when C i cannot be derived using b i ( t ) .The possible values of AoI using a i ( t ) are { , , , , ... } , andthe possible values of the AoI using b i ( t ) are { , , , , ... } .Therefore, the values of AoI that are only possible using a i ( t ) are { , , , , ... } . If the received current age C i from an activedevice i is one of { , , , , ... } , then the aging function is a i ( t ) , and, thus, the future AoI F i of device i is C i + 1 . TheBS can also determine the aging functions of active devices byusing regression. For an active device i , the BS can use the AoIfrom the most recently received uplink request from device i and C i to determine if the aging function is a i ( t ) or b i ( t ) .After determining the aging functions of active devices, theBS can compute the future AoI F i of active devices and learnthe device types, which are used for the priority schedulingscheme with learning. The BS can use either of the twomethods to accurately determine the current aging functions,but it is not always possible to use these methods. When bothmethods are not possible to use in a given time slot due toRACH preamble collision, the BS uses the expected value of F i for the priority scheduling scheme.The BS can compute the expected value of F i of an activedevice i by learning the device type of device i . The BS canlearn the type of a device i by using the previous data fromthe instances that the BS was able to determine the agingfunction of messages from device i . In particular, the BS canuse a maximum likelihood to determine the device types. We6et S be a set of all device types and O i be a vector ofaging functions that a device i had that the BS was able todetermine exactly. Furthermore, we let k i,f be the number oftimes that the BS determined device i to have aging function f .Assuming that the aging functions of a device i are determinedindependently, the learned type H i ∈ S of a device i is: H i = argmax s ∈S Pr( O i | s ) = argmax s ∈S Y f ∈F Pr( f | s ) k i,f , (12) = argmax s ∈S X f ∈F k i,f ln(Pr( f | s )) , (13)where F is a set of all aging functions. For our model, thevalues of Pr( f | s ) for any f ∈ F and s ∈ S are known.In particular, Pr( a i | s = 1) = m , Pr( b i | s = 1) = 1 − m , Pr( a i | s = 2) = 1 − m , and Pr( b i | s = 2) = m .Therefore, the maximum likelihood in (13) can be solved bythe BS directly.Once the device type of a device i is learned, the expectedfuture AoI E [ F i ] of device i can be computed. For our model,if H i = 1 , then the expected future AoI E [ F i | H i = 1] is: E [ F i | H i = 1] = m ( C i + 1) + (1 − m )(2 C i ) . (14)If H i = 2 , then the expected future AoI E [ F i | H i = 2] is: E [ F i | H i = 2] = (1 − m )( C i + 1) + m (2 C i ) . (15)The expected value of F i is used for the priority schedulingscheme only if the exact value of F i cannot be determined.Priority scheduling determines the RB allocation among theactive IoT devices based on the future AoI F i . In a time slot τ , the RBs are allocated first to the devices with the highest F i . Furthermore, for a device i with highest F i at time slot τ ,the number of RBs allocated to device i is R i,τ . If some ofthe active devices have same F i , then the RBs are allocatedfirst to the devices whose type is more likely to have a fasteraging function. For instance, if a device i is device type , adevice j is device type , and F i = F j , then device j has apriority over device i , because device j is more likely to haveexponentially aging messages. The RBs are allocated to theactive devices until all RBs are allocated or all active devicesare allocated the RBs.One of the major problems with priority scheduling isthe infinite blocking of low-priority tasks. However, since thepriority depends on the future AoI, the priority of low-prioritymessages increases with time. Therefore, regardless of theaging function or the device type, the messages will eventuallybe allocated the RBs to transmit to the BS. One of thelimitations of centralized RB allocation scheme is the overheadrelated to the RB request via RACH. With the higher numberof active devices N t at time t , the probability of the RACHpreamble collision c t becomes significant, and, thus, moretransmission failures occur. Furthermore, with centralized RBallocation scheme, a frequent communication between thedevices and the BS is required, which may not be viable for theIoT devices [46]. However, the main advantage of centralizedRB allocation scheme is that the RBs are fully utilized. Algorithm 1
Priority scheduling based on future AoI at timeslot t .1 : Receive values C i and R i,t , and initialize R .2 : Compute F i or E [ F i ] for each C i .3 : for j = 1 , , · · · Z ← set of type devices with j -th highest valueamong F i .5 : Z ← set of type devices with j -th highest valueamong F i .6 : for all i ∈ Z if R > ,8 : Allocate min ( R, R i,t ) RBs to device i .9 : R ← R − min ( R, R i,t ) . end if
10 : end for
11 : for all i ∈ Z
12 : if R > ,13 : Allocate min ( R, R i,t ) RBs to device i .14 : R ← R − min ( R, R i,t ) . end if
15 : end for end for
B. Distributed RB Allocation
Distributed RB allocation enables the devices to allocate theRBs in a self-organizing manner without any intervention fromthe BS. Since a duplicate RB selection results in transmissionfailures, the active devices must choose the RBs such thatno other device is choosing the same RB. Furthermore, thedistribution RB allocation can be modeled as one-to-oneassociation between the RBs and the devices. The behaviorof the devices wanting to choosing an RB alone can beformulated as a minority game [47].A suitable minority game for distributed RB allocation inan IoT is the Kolkata paise restaurant (KPR) game [37].The KPR game is a repeated game in which the customerssimultaneous go to one of the restaurants, which can onlyserve one customer each. Additionally, the cost of going to arestaurant is the same for all restaurants, and a customer canonly go to one restaurant at any given time. In the KPR game,the players are the customers, whose action in each iterationis to choose one of the restaurants. The payoff of a givenplayer depends on the utility of the chosen restaurants andthe number of players choosing the same restaurant. Givenplayers, actions, and payoffs in a game, one important stablesolution is an NE. A vector of actions is an NE if no player canachieve a higher payoff by a unilateral change of action. Forthe KPR game, the existence of an NE depends on the utilitiesof the restaurants [37], and an NE is when all customers go todifferent restaurants and none of the customers have the utilityof . If an NE exists in the KPR game, it coincides with thesocially optimal solution, which is when all restaurants arebeing utilized.The fundamental structure of the KPR game can be readilyextended for our IoT model. The customers can be modeledas IoT devices, and the restaurants can be modeled as the7Bs. Furthermore, the cost of using any of the RBs is same.However, there are significant differences between the KPRgame and the IoT game. In the KPR game, the number ofcustomers and the number of restaurants are the same, andeach customer goes to one of the restaurants at every iteration.In the IoT game, the number of devices N and the number ofRBs R may not be the same, and not all devices are active andneed to use the RBs at each time slot. The most significantdifference is the payoff in the case of duplicate RB selection.When multiple customers choose the same restaurant in theKPR game, one of those customers is randomly chosen to getthe full payoff, while other customers with duplicate selectionget a zero payoff. However, in the IoT game, all devicesthat choose the same RB get a zero payoff because of thetransmission failures.For our AoI minimization, the players are the N IoTdevices, and their action is to transmit using the RBs or notto transmit. We let R be the set of R RBs and let x i ( t ) bethe action of device i at time slot t . If x i ( t ) ∈ R , then device i transmits using the RB in x i ( t ) at time slot t . If x i ( t ) = 0 ,then device i does not transmit at time slot t . at each time slot,the payoff of each device depends on the actions of all devices.If a device transmits successfully by using an RB alone, thenthe payoff is ρ . Furthermore, a successful transmission usingany of the RBs has the same payoff of ρ . If a device transmitsunsuccessfully due to a duplicate RB usage, then the payoffis − γ . The transmission failure has a negative payoff, becausethe energy is consumed for the transmission without success.Additionally, the transmission failure using any of the RBshas the same payoff of − γ , and ρ and γ are positive numberssuch that ρ > γ .To minimize the AoI of the devices, active devices withlower AoI must not transmit, while the active devices withhigh AoI need to transmit. Using a distributed RB allocationscheme, the devices must know the AoI of other devices todetermine if their own AoI is high enough to transmit. Weassume that the active devices broadcast their own future AoI F i to other devices within the communication range r c , andthe communication resource for this broadcast is pre-allocated.Moreover, we assume that device-to-device communicationlinks are orthogonal to the uplink communication as donein [48]–[52]. Similar to the overhead related to the RACHuplink request for centralized RB allocation scheme, thecommunication between the devices to share the AoI can beseen as the overhead for distributed RB allocation scheme.However, depending on r c , the active devices may not know F i of all other active devices. We let α i be the active statusof a device i such that α i = 0 implies that device i is inactiveand α i = 1 implies that device i is active. We let A be avector that captures the future AoI F i of all active devices,and A i be a vector of the future AoI F i of the active deviceswithin r c of an active device i . With r c sufficiently large, | A i | = | A | for all devices, where | A | is the cardinality of A .For an active device i , if F i is higher than κ -th highest AoIin A i , then the active device i transmits. κ determines if F i issufficiently higher than the future AoI of other active devices, and we let A i ( κ ) be the κ -th highest AoI in A i . Moreover, κ should ensure that the number of transmitting devices T t attime slot t is equal to R such that all transmitting devices canbe allocated with the RB. If T t is higher than R , then thereare at least two active devices with transmission failure, whichcauses the average instantaneous AoI to increase. If T t is lessthan R , then there are some of the RBs not used by any ofthe active devices, which may cause the average instantaneousAoI to increase. However, T t = R is difficult to achieve when r c is not sufficiently large and | A i | < | A | .When | A i | = | A | , the devices have full informationon the future AoI F i of the active devices. Moreover, thedevices know all active devices. Since the devices have fullinformation of F i , an active device i transmits if F i ≥ A i ( R ) ,and an active device i does not transmit if F i < A i ( R ) .Therefore, κ = R when | A i | = | A | for all i . This ensuresthat R active devices transmit, while | A | − R devices do nottransmit. Therefore, the number of transmitting devices T t attime slot t is equal to R . To formulate this decision to transmitor not to transmit into the payoff, the payoff y full when anactive device i does not transmit is: y full ( A i ) = ( ρ + η ) θ R + ( A i ( R ) − F i − η ) − ( γ + η ) θ R + ( F i − A i ( R )) , (16)where η is a real number in (0 , and θ R + is an indicatorfunction such that: θ R + ( x ) = (cid:26) if x ∈ [0 , ∞ ) , if x [0 , ∞ ) . (17)The function of η in payoff functions is to ensure that thepayoff functions are used as intended and only appropriateindicator function is activated. It is important to note that y full ( A i ) does not depend on actions of the players, becausethe decision to transmit or not to transmit only depends onfuture AoI. Given the payoff in (16), the R active devices with F i ≥ A i ( R ) transmit, because their payoff of not transmittingis − ( γ + η ) . The | A | − R active devices with F i < A i ( R ) donot transmit, because their payoff of not transmitting is ρ + η .Therefore, IoT devices with R highest future AoI transmit,while other devices do not transmit.In a more realistic scenario where IoT devices do not havefull information of F i , | A i | < | A | , and the devices do notknow all active devices. It is difficult to make only R activedevices to transmit at time slot t , and, thus, κ is designed tomake T t ≈ R . For the case of | A i | < | A | , κ is: κ = (cid:24) R | A | | A i | (cid:25) , (18)where κ = R in the case of A = A i . However, with partialinformation, the active devices do not know A in (18). | A | is the number of active devices, and the expected number ofnewly active devices is N v a . However, with previously activedevices yet to transmit successfully, | A | is typically greaterthan N v a . Therefore, in (18), | A | can be estimated with N v a ζ ,where ζ is a design parameter to consider the number ofpreviously active devices yet to transmit successfully. With8igher values of ζ , the number of transmitting devices T t attime slot t is smaller, while T t is bigger with smaller valuesof ζ . Therefore, with approximation for | A | , κ is: κ = (cid:24) RN v a ζ | A i | (cid:25) . (19)For an active device i , κ in (19) approximates if F i issufficiently higher than the future AoI of other active devicesbased on the percentile of F i on the known vector of futureAoI A i . With | A i | < | A | , the payoff y act when an activedevice i does not transmit is: y act ( A i ) = ( ρ + η ) θ R + ( A i ( κ ) − F i − η ) − ( γ + η ) θ R + ( F i − A i ( κ )) . (20)It is important to note that y act ( A i ) is equal to y full ( A i ) , whenthe active devices have full information with | A i | = | A | and κ = R . Similar to y full ( A i ) (16), with the payoff y act ( A i ) ,the active devices with sufficiently high F i satisfying F i ≥ A i ( κ ) transmit, while the active devices with F i such that F i < A i ( κ ) do not transmit. Moreover, y act ( A i ) also does notdepend on actions of the players.In the IoT game, the payoff needs to consider the inactivedevices. The inactive devices with α i = 0 do not transmit asthey do not have messages to transmit. The payoff y nt whenan device i does not transmit is: y nt ( A i , α i ) = (1 − α i )( ρ + η ) + α i y act ( A i ) . (21)With payoff y nt ( A i , α i ) for not transmitting, the inactivedevices with α i = 0 do not transmit as they get payoffof ρ + η . The active devices with α i = 1 decide to trans-mit or to not transmit based on κ and future AoI. We let x ( t ) = [ x ( t ) , x ( t ) , · · · , x N ( t )] be a vector of all actions of N devices at time slot t . For a given x ( t ) , the payoff function y i ( x ( t ) , A i , α i ) for a device i at time slot t is: y i ( x ( t ) , A i , α i )= ρ if x i ( t ) = x j ( t ) ∀ j = i, x i ( t ) = 0 , − γ if ∃ j = i s.t. x j ( t ) = x i ( t ) = 0 ,y nt ( A i , α i ) if x i ( t ) = 0 . (22)In a simple game where N = R = 2 with v a = 1 , the payoffsof two devices at each time slot is summarized in Table I.NE in this IoT game is x ( t ) = [1 , or x ( t ) = [2 , . Forsimple IoT game, NE is when two devices choose differentRBs and get the payoff of ρ . If one device deviates from NEand the other device does not deviate from NE, the devicedeviating from NE gets the lower payoff of − γ or − ( γ + η ) .Furthermore, in the IoT game, NE implies that a duplicate RBselection does not occur. An NE for a more general case ofthe IoT game can be found with certain conditions. Theorem 1.
For the IoT game with N players with action x i ( t ) ∈ { , R} , payoff function y i ( x ( t )) , and | A i | = | A | forall i , any vector of actions x ( t ) such that at most R activedevices with F i ≥ A i ( R ) transmit, the rest of the devices donot transmit, and each of the RBs is used by at most one deviceis an NE. Table II O T GAME WITH N = R = 2 . x ( t ) = 1 x ( t ) = 2 x ( t ) = 0 x ( t ) = 1 ( − γ, − γ ) ( ρ, ρ ) ( − ( γ + η ) , ρ ) x ( t ) = 2 ( ρ, ρ ) ( − γ, − γ ) ( − ( γ + η ) , ρ ) x ( t ) = 0 ( ρ, − ( γ + η )) ( ρ, − ( γ + η )) ( − ( γ + η ) , − ( γ + η )) Proof.
See Appendix B. (cid:4)
There are many sets of actions that satisfy the conditionsdescribed in Theorem 1, and, thus, NE in the IoT game is notunique. For instance, when the number of devices is equalto the number of RBs with v a = 1 in an IoT, an NE iswhen each RB is used by one device, and, thus, the numberof NEs in that particular IoT game is N ! . There are manyNEs, because the payoff of successful transmission does notdepend on which RB is used. Although there are many NEs,the expected payoffs of devices at any given NE are the same,and, thus, one of those NEs is chosen with distributed RBallocation algorithm discussed in Section IV-B1.
Similar tothe simple IoT game in Table I, an NE for our IoT gameimplies that the number of transmitting devices T t is equal tothe number of RBs R and that a duplicate RB selection doesnot occur. Furthermore, at an NE, a transmitting device hasone of the R highest AoI, and a device that is not transmittingis inactive or has low AoI. This is because the payoff functionwith | A i | = | A | is designed to only allow the messages havingthe R highest AoI to transmit. Therefore, the convergence ofa distributed RB allocation algorithm to an NE reduces theaverage instantaneous AoI. However, when the devices onlyhave partial information such that | A i | < | A | , x ( t ) describedin the Theorem 1 is not necessarily an NE, because κ is notnecessarily equal to R .In addition to the NE, another solution concept is a sociallyoptimal solution in which the overall payoff of an IoT gameis maximized. In other words, a vector of actions is a sociallyoptimal solution when the sum of payoffs of all devices ismaximized. In an IoT game, a socially optimal solution iswhen the devices with highest future AoI F i fully utilize RBswithout any duplicate RB selection. Therefore, similar to anNE in KPR game [37], an NE in IoT game coincides with asocially optimal solution. A performance metric that can beused to describe an NE and a socially optimal solution is aservice rate s r , which is the percentage of RBs that are used byone device. NE in Theorem 1 has a service rate of , whichimplies that all RBs are used by one device, or the highestpossible service rate of T t / R .With the number of transmitting devices T t approximatelyequal to R via IoT game design, a distributed RB allocation al-gorithm is necessary to enable the transmitting devices to shareRBs autonomously. Moreover, with existence of a sociallyoptimal NE in our IoT game, the convergence of a distributedRB allocation algorithm to an NE is crucial in minimizingthe average instantaneous AoI. Therefore, to evaluate differentRB allocation algorithms, the convergence to an NE and theservice rates are analyzed. The service rate is an importantmetric to determine convergence to an NE, because a vector of9 lgorithm 2 SCA for device i at time t .1 : Receive X i , P i , A i , and L .2 : if x i ( t − ∈ R , y i ( x ( t − ρ , and α i = 1 ,3 : x i ( t ) ← x i ( t − .4 : else if x i ( t − ∈ R , y i ( x ( t − ρ , and α i = 0 ,5 : j ← one neighboring device chosen with F j P Fh ∈ A i F h .6 : x j ( t ) ← x i ( t − .7 : else if x i ( t − ∈ R and y i ( x ( t − − γ ,8 : z ← with probability X i ( x i ( t − − .9 : if z = 1 , x i ( t ) ← x i ( t − .10 : else x i ( t ) ← randomly chosen from L . end if .11 : else if x i ( t −
1) = 0 ,12 : x i ( t ) ← randomly chosen from L .13 : end if. actions must achieve service rate of or highest service rateto be an NE. Furthermore, the service rate is an importantperformance metric for the AoI, because the higher servicerate implies that more devices are transmitting successfullyat each time slot, reducing the average instantaneous AoI.Therefore, a distributed RB allocation algorithm must achievea service rate of or increase the service rate as high aspossible, because a high service rate is required to achieve alow average instantaneous AoI.
1) Stochastic Crowd Avoidance:
We propose a stochasticcrowd avoidance (SCA) algorithm that enables the devices toavoid using the RBs that are used by many devices stochas-tically and to choose an RB for a successful transmission, asshown in Algorithm 2. For the SCA algorithm, the devicesneed to share more information in addition to their F i andcan perform a channel sensing to determine the RBs thatwere not used at a previous time slot [23]–[25], and [53].At time slot t , the devices that transmitted at the time slot t − share their previous actions x i ( t − ∈ R and theprevious payoff y i ( x ( t − of their transmission. We let X i be the vector of actions of the transmitting devices attime slot t − that a device i knows and P i be the vectorof payoffs of the transmitting devices at time slot t − thata device i knows. Furthermore, we let X i ( x ) with x ∈ R be the number of x in X i . In other words, X i ( x ) is thenumber of devices that chose the RB x at time slot t − thata device i knows. Learning from X i and P i , the proposedSCA algorithm enables a transmitting device i at time slot t to not use the RBs that are being used successfully by otherdevices and to avoid using the contended RBs stochastically.Using SCA algorithm, at time slot t , a transmitting device i determines its RB usage based on x i ( t − and y i ( x ( t − .If the transmission at time slot t − is successful such that x i ( t − ∈ R and y i ( x ( t − ρ , then transmitting device i uses the same RB x i ( t ) = x i ( t − . If the transmissionat time slot t − is unsuccessful such that x i ( t − ∈ R and y i ( x ( t − − γ , then transmitting device i uses thesame RB x i ( t ) = x i ( t − with probability X i ( x i ( t − − or chooses an RB from a set L uniformly randomly. L is a set of the RBs that were not used by any of the device attime slot t − determined with channel sensing. If there is notransmission at time slot t − such that x i ( t −
1) = 0 , thenthe transmitting device i chooses an RB from L uniformlyrandomly. In the case where v a < , a device that transmittedsuccessfully at time slot t − may no longer be active at timeslot t . In this case, a device i with α i = 0 and y i ( x ( t − ρ chooses a neighboring device j with α j = 1 with a probabilityproportional to F j , such that the probability is F j / P Fh ∈ A i F h .In our SCA algorithm, the RB that is used successfullyat time slot t − is also used successfully at time slot t if v a = 1 or if there is an active neighboring device.Moreover, with r c sufficiently large, the expected number oftransmitting devices using an RB that is used by more thanone device at the previous time slot t − is at the currenttime slot t . This is because the probability of choosing thesame RB after a transmission failure is / X i ( x i ( t − . Thedevices avoid to use the same RB stochastically even after atransmission failure, because the strict crowd avoidance causesthe crowding in other RBs resulting in more transmissionfailures. Furthermore, when a device i chooses some otherRB such that x i ( t ) = x i ( t − , device i chooses from aset of RBs L that were not used at time slot t − . This isto avoid using the RBs that are either used successfully orcrowded, both of which cause the transmission failures. WithSCA algorithm design to avoid duplicate RB selection, nextwe prove that the proposed SCA algorithm converges to anNE under certain IoT system parameters. Theorem 2.
When N devices are always active with fullinformation and use out of N RBs to transmit with negligibleoutage probability p i,t at each time slot, the vector of actions x ( t ) converges to an NE using SCA.Proof. See Appendix C. (cid:4)
Under the conditions in Theorem 2, x ( t ) converges to anNE, and this implies that the service rate increases to . Ingeneral, SCA algorithm increases the service rate, because anRB, which is used by device at previous time slot t − , isstill used by device at current time slot t . Therefore, SCAalgorithm is effective in reducing the average instantaneousAoI. However, SCA is susceptible to the high outage prob-ability p i,t based on the SNR. This is because SCA cannotdistinguish the transmission failure due to the duplicate RBselection and the transmission failure due to the outage basedon the SNR. Furthermore, with partial information | A i | < | A | ,the devices also have partial information of X i and P i , and,thus, the devices cannot choose x i ( t ) accurately in the case oftransmission failure.In a massive IoT with N > R and partial information,the vector of actions x ( t ) does not converge to an NE usingSCA, because it is not possible to achieve service rate of .The service rate cannot be , because the transmitting devicesare changing every time slot and a duplicate RB selectionis inevitable. Since the service rate cannot be , the averageinstantaneous AoI in a massive IoT is higher than the average10nstantaneous AoI in an ideal IoT described in Theorem 2.However, in a massive IoT, the proposed SCA algorithmstill enables the transmitting devices to stochastically avoidduplicate RB selection with available information. Therefore,the proposed SCA algorithm still increases the service rateand reduces the average instantaneous AoI. However, in thatcase, it does not reach an NE but rather a sub-optimal,heuristic solution. To evaluate performance of the proposedSCA algorithm, next we study a random RB selection fordistributed RB allocation scheme.
2) Random RB Selection:
One way for the transmittingdevices to determine their RB usage is via random selection. Inother words, the actions x i ( t ) of the transmitting devices arechosen uniformly random in R . This random RB selectionis used as a baseline. Even with | A i | = | A | for all i , therandom RB selection is highly unlikely to achieve x ( t ) suchthat each of the RBs is used by at most one device, which isthe requirement of x ( t ) to be an NE. Furthermore, the servicerate s r using random RB selection for the transmitting devicesis low. Proposition 2.
At a time slot t , the service rate s r with T t transmitting devices using random RB selection is: s r = T t R (cid:18) R − R (cid:19) T t − , (23)and, for a massive IoT with N increasing to infinity, the servicerate s r is: lim N →∞ s r = T t R − (cid:18) − T t R (cid:19) (24) Proof.
See Appendix D. (cid:4)
Under the conditions in Theorem 2, when N devices arealways active and use out of N RBs, the number oftransmitting devices T t is equal to N . In this case, the servicerate using random RB selection is always less than evenwith N = R ≥ . On the other hand, for a massive IoTwith N > R , the service rate using random RB selectionexponentially decreases to as N increases. Therefore, therandom RB selection is not suitable for a massive IoT inwhich the number of devices N outnumbers the number ofRBs R , Furthermore, with low service rate, the probability ofa successful transmission is low for a device, and, thus, theaverage instantaneous AoI is high.V. S IMULATION R ESULTS AND A NALYSIS
For our simulations, we consider a rectangular area withwidth w and length l within which the N devices are deployedfollowing a Poisson point process. We let w = l = 10 m and R = 50 with a MHz frequency band [54], while the numberof devices N will be varied for analysis. We choose a time slotduration of ms [55] and expected value of SNR of dBwith λ = . and σ = 0 . . To vary outage probability p i,t ,different values of ǫ are used. Moreover, a device is assumedto be of type with probability . with m = 0 . , while adevice is assumed to be of type with probability . with m = 0 . . The average of the current AoI C i at the time Activation probability v a A v e r age i n s t an t aneou s age o f i n f o r m a t i on Proposed priority scheduling with learningPriority scheduling without learningPriority scheduing with full information
Figure 1. Average instantaneous AoI using centralized RB allocation schemeswhile varying v a . slot of successful transmission is the performance metric fordifferent RB allocation schemes, and their performances areanalyzed with varying v a and p i,t .For the centralized RB allocation scheme, the number ofRACH preambles P for the uplink transmission request is [56]. Three different kinds of priority scheduling areanalyzed. Priority scheduling without learning [12]–[15] doesnot learn the device types, and, hence, this scheme is used forbaseline comparison. The proposed priority scheduling withlearning learns the device types using maximum likelihood andinformation on F i . Priority scheduling with full informationassumes that the BS always knows the types of all devices,and, hence, this scheme is used for optimal performance com-parison. All three priority scheduling algorithms are analyzedwith N = 500 , while varying v a and p i,t .Fig. 1 shows the average instantaneous AoI of the devicesusing centralized RB allocation schemes for different values ofthe activation probability v a with p i,t = 0 . and ǫ = 1 . Theaverage instantaneous AoI for the no learning case quickly in-creases to . and then increases slowly above , while theaverage instantaneous AoI for both learning and full informa-tion quickly increases to . and then increases slowly above . As v a increases, the average instantaneous AoI increasesfor all priority scheduling algorithms, because more devicesare transmitting and RACH preamble collisions are more likelyto occur. However, for v a > . , the average instantaneous AoIflattens and increases at a much slower rate with increasing v a for all priority scheduling algorithms, because all RBs are fullysaturated. Moreover, there is a significant difference betweenpriority scheduling with learning and without learning. Afterthe average instantaneous AoI flattens, the difference of theaverage instantaneous AoI between priority scheduling withlearning and without learning is constantly about . Thisimplies that the proposed priority scheduling scheme withlearning achieves about . lower average instantaneousAoI when compared to simple priority scheduling scheme.Hence, learning the device types is important to decrease the11 Activation probability v a A v e r age i n s t an t aneou s age o f i n f o r m a t i on Proposed priority scheduling with learningPriority scheduling without learningPriority scheduing with full information
Figure 2. Average instantaneous AoI using centralized RB allocation schemeswith different transmit powers while varying v a . average instantaneous AoI for priority scheduling. However,there is an insignificant difference between priority schedulingwith learning and with full information, which implies that thelearning is effective in learning the device types.Fig. 2 shows the average instantaneous AoI of the deviceswith different transmit powers using centralized RB allocationschemes for different values of the activation probability v a with p i,t = 0 . and ǫ = 1 . The devices have differenttransmit powers such that their SNR values are uniformlydistributed random variables from . dB to . dB, andthe only difference between Fig. 1 and Fig. 2 is assumptionon the transmit powers of devices. The overall trends ofthe average instantaneous AoI for all three centralized RBallocation schemes are similar to the trends shown in Fig. 1.However, a notable difference is the average instantaneous AoIafter the curves flatten. After the average instantaneous AoIflattens, the average instantaneous AoI increases slowly above for priority scheduling without learning, increases slowly to for priority scheduling with learning, and increases slowlyto . for priority scheduling with full information. Withdifferent SNR values randomly assigned to the devices, somedevices have higher outage probability and other devices havelower outage probability compared to the devices in Fig. 1.With exponentially aging messages, an increase in the averageinstantaneous AoI from devices with higher outage probabilityoutweighs a decrease in the average instantaneous AoI fromdevices with lower outage probability. Therefore, there is aslight increase in average instantaneous AoI when the deviceshave different transmit powers. However, similar to Fig. 1, thelearning can still effectively decrease the average instantaneousAoI, and priority scheduling with learning performs similarto priority scheduling with full information even when thedevices have different transmit powers.Fig. 3 shows the average instantaneous AoI of the devicesusing centralized RB allocation schemes for different valuesof the SNR outage probability p i,t with v a = 0 . and varying ǫ from to . Unlike in Fig. 1, the average instantaneous SNR outage probability p i,t A v e r age i n s t an t aneou s age o f i n f o r m a t i on Proposed priority scheduling with learningPriority scheduling without learningPriority scheduing with full information
Figure 3. Average instantaneous AoI using centralized RB allocation schemeswhile varying p i,t . AoI does not flatten and increases at about the same rate.With p i,t = 0 . , the average instantaneous AoI withoutlearning is . , and the average instantaneous AoI withlearning is . . With p i,t = 0 . , the average instantaneousAoI without learning is . , and the average instantaneousAoI with learning is . . Therefore, the difference betweenthe average instantaneous AoI with learning and withoutlearning increases as p i,t increases. Furthermore, with high p i,t , the proposed priority scheduling scheme with learningachieves about . lower average instantaneous AoI whencompared to simple priority scheduling scheme. For a higher p i,t and more frequent transmission failures, the AoI of theexponentially aging messages becomes much higher than theAoI of the linearly aging messages. In this case, the learningscheme, which enables the BS to accurately identify themessages aging faster, becomes more crucial in reducingthe average instantaneous AoI. Furthermore, the differencebetween the average instantaneous AoI with learning and withfull information also increases as p i,t increases. This is becauseit becomes increasingly difficult to learn the device types as p i,t increases.For the distributed RB allocation scheme, the commu-nication range r c determines the information A i that thedevices have. For the given dimensions of the deployment area, r c ≥ m is sufficiently large such that | A i | = | A | for anydevice i . SCA algorithm is compared against two algorithms,which are the random RB selection and the pre-determinedRB selection. The pre-determined RB selection scheme [37]is known as the dictator’s solution in the KPR game, and theRB usage for a device i is pre-determined based on the rankof F i . For instance, if F i is κ -th highest in A i for κ ≤ R ,device i uses a specific RB as previously agreed among IoTdevices. The pre-determined RB allocation scheme requiresfull information | A i | = | A | for all i and always achieves aservice rate s r of . While the pre-determined RB selectionis used for optimal performance comparison, the random RBselection is used for baseline comparison. To analyze different12
10 15 20 25 30
Time (ms) S e r v i c e r a t e s r Random selectionProposed SCA, v a = .25 Proposed SCA, v a = .35 Proposed SCA, v a = .45 Activation probability v a A v e r age i n s t an t aneou s age o f i n f o r m a t i on Pre-determined selectionProposed SCA
Figure 4. Average instantaneous AoI and service rate using distributed RBallocation schemes while varying v a . distributed RB allocation schemes, the activation probability v a , the SNR outage probability p i,t , and the communicationrange r c are varied. In addition to the average instantaneousAoI, the service rate s r is evaluated for different distributedRB allocation schemes.Fig. 4 shows the average instantaneous AoI and the servicerate of the devices using distributed RB allocation schemesfor different values of the activation probability v a with p i,t = 0 . , ǫ = 1 , r c = 10 m, and N = 200 . It isimportant to note that the expected number of newly activedevices at a given time slot is N v a , and, thus, the numberof active devices N t outnumbers the number of RBs R forhigh values of v a , simulating a massive IoT. Moreover, with r c = 10 m, SCA only has partial information such that | A i | < | A | for all i . The service rate converges to . for SCA with v a = 0 . , . for SCA with v a = 0 . , . for SCA with v a = 0 . , and . for random RBallocation with v a = 0 . . As v a increases from . to . , the average instantaneous AoI increases from . to . using SCA. As N t increases with increasing v a , thetransmission failure is more likely to occur due to the duplicateRB selection, because R is fixed. Therefore, as v a increases, s r decreases, and the average instantaneous AoI increases.The pre-determined RB allocation scheme achieves a muchlower average instantaneous AoI compared to SCA, becausethe pre-determined RB allocation scheme requires and uses thefull information. Furthermore, the average instantaneous AoIwith random RB allocation is multiple orders of magnitudehigher than the average instantaneous AoI with SCA. There-fore, in a massive IoT with partial information, the proposedSCA algorithm is the most suitable algorithm to achieve lowaverage instantaneous AoI as it balances between having fullinformation and performing arbitrary allocations.Fig. 5 shows the average instantaneous AoI and the ser-vice rate of the devices with different transmit powers usingdistributed RB allocation schemes for different values of theactivation probability v a with p i,t = 0 . , ǫ = 1 , r c = 10 Time (ms) S e r v i c e r a t e s r Random selectionProposed SCA, v a = .25 Proposed SCA, v a = .35 Proposed SCA, v a = .45 Activation probability v a A v e r age i n s t an t aneou s age o f i n f o r m a t i on Pre-determined selectionProposed SCA
Figure 5. Average instantaneous AoI and service rate using distributed RBallocation schemes with different transmit powers while varying v a . Time (ms) S e r v i c e r a t e s r Pre-determined selectionRandom selectionProposed SCA, p i,t = .01 Proposed SCA, p i,t = .03 Proposed SCA, p i,t = .05 SNR outage probability p i,t A v e r age i n s t an t aneou s age o f i n f o r m a t i on Pre-determined selectionRandom selectionProposed SCA
Figure 6. Average instantaneous AoI and service rate using distributed RBallocation schemes while varying p i,t . m, and N = 200 . The devices have different transmit powerssuch that their SNR values are uniformly distributed randomvariables from . dB to . dB, and the only differencebetween Fig. 4 and Fig. 5 is assumption on the transmit powersof devices. The difference in the service rates between Fig. 4and Fig. 5 is insignificant. However, there is a notable increasein the average instantaneous AoI in Fig. 5 compared to Fig.4. Some devices have higher outage probability and otherdevices have lower outage probability, because the deviceshave different transmit powers. With an exponential agingfunction, an increase in the average instantaneous AoI withhigher outage probability is more significant than a decreasein the average instantaneous AoI with lower outage probability.Therefore, similar to Fig. 2, there is a slight increase in theaverage instantaneous AoI when the devices have differenttransmit powers. Even when the devices have different transmitpowers, the proposed SCA algorithm is still the most suitablealgorithm to achieve low average instantaneous AoI in amassive IoT with partial information.13
10 15 20
Time (ms) S e r v i c e r a t e s r Pre-determined selectionRandom selectionProposed SCA, r c = 1 mProposed SCA, r c = 5 mProposed SCA, r c = 10 mProposed SCA, r c = 15 m Communication range r c (m) A v e r age i n s t an t aneou s age o f i n f o r m a t i on Pre-determined selectionRandom selectionProposed SCA
Figure 7. Average instantaneous AoI and service rate using distributed Rballocation schemes while varying r c . Fig. 6 shows the average instantaneous AoI and the servicerate of the devices using distributed RB allocation schemesfor different values of the SNR outage probability p i,t with r c = 10 m, v a = 1 , N = R = 50 , and varying ǫ from to .It is important to note that the number of active devices N t isequal to R with N = R and v a = 1 , and this is the conditionconsidered in Theorem 2. The service rate converges to . for SCA with p i,t = 0 . , . for SCA with p i,t = 0 . , . for SCA with p i,t = 0 . , and . for random RB allocation.As p i,t increases from . to . , the average instantaneousAoI increases from . to . using SCA, while the averageinstantaneous AoI increases from . to . using randomRB allocation. With high p i,t , the proposed SCA algorithmachieves about . lower average instantaneous AoI whencompared to random RB allocation. Since T t = N with v a = 1 and N = R , the theoretical value of s r (23) forrandom RB allocation case matches the simulated value of s r in Fig. 6. As p i,t increases, the converged value of s r forSCA decreases, because the proposed SCA assumes that thetransmission failures are caused by duplicate RB selection.Therefore, using SCA, a device i stochastically avoids to usean RB even when there was no duplicate RB selection and thetransmission failure is caused by SNR outage. Furthermore,the difference between the average instantaneous AoI usingSCA and random RB allocation decreases as p i,t increases,because increasing p i,t has a more negative impact on SCAthan on random RB allocation. However, it is important tonote that SCA with low p i,t converges quickly to the servicerate of with low p i,t as discussed in Theorem 2.Fig. 7 shows the average instantaneous AoI and the servicerate of the devices using distributed RB allocation schemesfor different values of the communication range r c with p i,t = 0 . , ǫ = 1 , v a = 1 , and N = R = 50 . Thecommunication range r c determines the amount of information A i that the devices have, and r c = 15 m implies that thedevices have full information | A i | = | A | for all i . The servicerate converges to . for SCA with r c = 15 m, . for Future age parameter A v e r age i n s t an t aneou s age o f i n f o r m a t i on Pre-determined selectionRandom selectionProposed SCA
Future age parameter A v e r age i n s t an t aneou s age o f i n f o r m a t i on Proposed priority scheduling with learningPriority scheduling without learningPriority scheduling with full information
Figure 8. Average instantaneous AoI using centralized and distributed RBallocation schemes while varying β in a massive IoT. SCA with r c = 10 m, . for SCA with r c = 5 m, . for SCA with r c = 1 m, and . for random RB allocation.As r c increases from m to m, the average instantaneousAoI decreases from . to . using SCA, while the averageinstantaneous AoI decreases from . to . using randomRB allocation. With low r c , the proposed SCA algorithmachieves about . lower average instantaneous AoI whencompared to random RB allocation. Similar to the Fig. 6.the theoretical and simulated values of s r for random RBallocation are matched. As r c increases, the convergence valueof s r for SCA increases, because the devices have moreinformation A i with higher r c . As r c increases sufficientlysuch that | A i | = | A | for all i , the service rate converges to as discussed in Theorem 2. However, SCA with only partialinformation can still achieve s r close to . Moreover, as r c increases, the average instantaneous AoI using SCA convergesto the average instantaneous AoI using pre-determined RBallocation scheme.Next, the proposed centralized and distributed RB allocationschemes are compared in a massive IoT with N > R and inan ideal IoT described in Theorem 2. To analyze different RBallocation schemes, the activation probability v a and the SNRoutage probability p i,t are varied,Fig. 8 shows the average instantaneous AoI of the devicesusing centralized and distributed RB allocation schemes fordifferent values of β with p i,t = 0 . , ǫ = 1 , r c = 10 m, N = 100 , and v a = 1 . This simulates a massive IoTas the number of devices N greatly outnumbers the numberof RBs R . For the centralized RB allocation scheme, as β in-creases, the average instantaneous AoI with learning decreasesfrom . to . , and the average instantaneous AoI withfull information decreases from . to . . However, theaverage instantaneous AoI without learning does not changesignificantly. With many type devices frequently transmittingexponentially aging messages, high values of β can effectivelyreduce the average instantaneous AoI of type devices asBS learns the device types. However, without learning the14 .1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Activation probability v a A v e r age i n s t an t aneou s age o f i n f o r m a t i on SCA with r c = 5 SCA with r c = 10 SCA with r c = 15 Priority Scheduling
Figure 9. Average instantaneous AoI using centralized and distributed RBallocation schemes while varying v a in a massive IoT. device types, high values of β are ineffective in reducing theaverage instantaneous AoI. For the distributed RB allocationscheme, as β increases, the average instantaneous AoI withrandom selection decreases from . to . , and the averageinstantaneous AoI with proposed SCA decreases from . to . . This is because high values of β enable the exponentiallyaging messages to be transmitted before their AoI increasesgreatly due to duplicate RB selection. However, the averageinstantaneous AoI with pre-determined selection does notchange significantly, because most duplicate RB selectioncan be avoided with given r c and pre-determined selection. β affects centralized and distributed RB selection schemesdifferently depending on other parameters of the IoT.Fig. 9 shows the average instantaneous AoI of the devicesusing centralized and distributed RB allocation schemes fordifferent values of the activation probability v a with p i,t =0 . , ǫ = 1 , and N = 200 . This simulates a massive IoTas the number of devices N greatly outnumbers the numberof RBs R . As v a increases, the average instantaneous AoIconverges to for SCA with r c = 5 m, for SCA with r c = 10 m, and for SCA with r c = 15 m. On the otherhand, for priority scheduling, the average instantaneous AoIincreases from . to . as v a increases from . to . .With high v a , the proposed SCA algorithm with r c = 15 machieves about -fold higher average instantaneous AoI whencompared to the proposed priority scheduling with learning.For almost any values of v a , centralized RB allocation withpriority scheduling performs much better than distributed RBallocation with SCA in terms of the average instantaneous AoI.However, centralized RB allocation scheme requires the BS todictate the RB allocation for all devices, and, thus, centralizedRB allocation scheme may not be viable for some of theIoT. Moreover, similar to Fig. 1, the average instantaneousAoI flattens after a certain value of v a , because all RBs arefully saturated. There is a performance gap between SCA withdifferent values of r c , because r c is directly related to theamount of information that the devices have. With higher r c SNR outage probability p i,t A v e r age i n s t an t aneou s age o f i n f o r m a t i on SCA with r c = 5 SCA with r c = 10 SCA with r c = 15 Priority Scheduling
Figure 10. Average instantaneous AoI using centralized and distributed RBallocation schemes while varying p i,t in a massive IoT. and more information for the devices, SCA is more effectivein reducing the average instantaneous AoI.Fig. 10 shows the average instantaneous AoI of the devicesusing centralized and distributed RB allocation schemes fordifferent values of the SNR outage probability p i,t with v a = 0 . , N = 200 , and varying ǫ from to . Similarto Fig. 9, this simulates a massive IoT. When p i,t = 0 . . theaverage instantaneous AoI is . using SCA with r c = 5 m, . using SCA with r c = 10 m, . using SCA with r c = 15 m, and . using priority scheduling. With high p i,t ,the proposed SCA algorithm with r c = 15 m achieves about -fold higher average instantaneous AoI when compared tothe proposed priority scheduling with learning. Similar toFig. 9, centralized RB allocation with priority schedulingperforms much better than distributed RB allocation with SCAin terms of the average instantaneous AoI for all values of p i,t . It is interesting to note that the difference in the averageinstantaneous AoI between SCA algorithms increases as p i,t increases. This implies that SCA with less information ismore severely affected by increasing p i,t than SCA with moreinformation.Fig. 11 shows the average instantaneous AoI of the devicesusing centralized and distributed RB allocation schemes fordifferent values of the SNR outage probability p i,t with v a = 1 , N = R = 50 , and varying ǫ from to . Thissimulates an ideal IoT for SCA as some of the conditions forNE convergence in Theorem 2 are satisfied. When p i,t = 0 . .the average instantaneous AoI is . using SCA with r c = 5 m, . using SCA with r c = 10 m, . using SCA with r c = 15 m, and . using priority scheduling. With high p i,t ,the proposed SCA algorithm with r c = 15 m achieves about -fold higher average instantaneous AoI when compared to theproposed priority scheduling with learning. It is interestingto note that even in an ideal IoT for SCA, priority schedulingwith learning performs better than SCA in terms of the averageinstantaneous AoI for most values of p i,t . Moreover, similarto Fig. 10, the difference in the average instantaneous AoI15 .02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 SNR outage probability p i,t A v e r age i n s t an t aneou s age o f i n f o r m a t i on SCA with r c = 5 SCA with r c = 10 SCA with r c = 15 Priority Scheduling
Figure 11. Average instantaneous AoI using centralized and distributed RBallocation schemes while varying p i,t in an ideal IoT. between SCA algorithms increases as p i,t increases.From our simulations, we observe that both priority schedul-ing and SCA are susceptible to high SNR outage probabil-ity p i,t as the average instantaneous AoI increases withoutflattening as p i,t increases. This is because the SNR outageprobability is directly related to the transmission failures.However, the average instantaneous AoI increases slowly aftera certain value of the activation probability v a , because theRBs are fully saturated. Since increasing v a with fixed N is equivalent to increasing N with fixed v a , the averageinstantaneous AoI also flattens for the case in which onlythe number of devices N increases. Although centralizedRB allocation scheme outperforms distributed RB allocationscheme in most cases, SCA can still achieve a high servicerate s r and low average instantaneous AoI only with partialinformation. Furthermore, communication range r c and infor-mation availability are critical to the performance of SCA.VI. C ONCLUSION
In this paper, we have proposed centralized and distributedapproaches for allocating the limited communication resourcesbased on the aging function and the current AoI of IoT devices.In the presence of both linear and exponential aging functions,we have shown that comparing the future AoI achieves alower average instantaneous AoI at the BS than comparing thecurrent AoI. For the centralized approach, we have introduceda priority scheduling scheme with learning, which enables theBS to allocate the limited RBs to the heterogeneous devicesbased on their future AoI. For the distributed approach, wehave formulated the problem of autonomously allocating thelimited RBs to the devices using game theory, and we havedesigned payoff functions to encourage the devices with highAoI to transmit, while discouraging the devices with low AoIto not transmit. Furthermore, we have proposed a novel SCAalgorithm such that the heterogeneous devices can allocatethe RBs in a self-organizing manner to avoid the duplicateRB selection and to minimize the AoI. We have proved the conditions that a vector of actions in the IoT game mustsatisfy to achieve an NE. Furthermore, we have proved that theactions of devices using our proposed SCA algorithm convergeto an NE, if the devices have sufficient information undercertain network parameters. Simulation results have shownthat the average instantaneous AoI is an increasing functionof the activation probability and the SNR outage probability.Moreover, the simulation results have shown that the servicerate is an increasing function of the communication range anda decreasing function of the activation probability and theSNR outage probability. We have compared our centralizedand distributed RB allocation schemes, and we have shownthat our centralized RB allocation scheme outperforms ourdistributed RB allocation scheme in most cases. However, ourproposed SCA algorithm has shown to be effective in reducingthe AoI and increasing the service rate only with partialinformation. With high SNR outage probability, the proposedpriority scheduling scheme with learning has shown to achieveabout . lower average instantaneous AoI when comparedto simple priority scheduling scheme. Furthermore, with highSNR outage probability, the proposed SCA algorithm hasshown to achieve about . lower average instantaneousAoI when compared to random RB allocation.A CKNOWLEDGMENT
This research was supported by the U.S. Office of NavalResearch (ONR) under Grant N00014-19-1-2621.A
PPENDIX
A. Proof of Proposition 1
Without a loss of generality, let there be an IoT device i withaging function a i ( t ) and an IoT device h with aging function b h ( t ) . Moreover, the proof only considers the case with N = 2 and R = 1 . It is sufficient to only consider N = 2 , becausecomparing the AoI of N > devices is equivalent to doingpairwise AoI comparison N ( N − / times. It is unnecessary toconsider the cases of R = 0 and R ≥ . If N = 2 and R = 0 ,none of the devices can be allocated an RB. If N = 2 and R ≥ , all devices can be allocated the RBs. Therefore, theAoI comparison to determine RB allocation is unnecessary.At time slot τ , the current AoI of devices i and h must beone of the following cases: a i ( τ ) > b h ( τ ) , a i ( τ ) = b h ( τ ) , or a i ( τ ) < b h ( τ ) . If a i ( τ ) ≤ b h ( τ ) , then a i ( τ + β ) ≤ b h ( τ + β ) for any positive integer β . At time slot τ , the current AoIcomparison with t = τ and the future AoI comparison with t = τ + β are equivalent, because device h is allocatedwith an RB for both. However, if a i ( τ ) > b h ( τ ) , the futureAoI of devices i and h can be any of the following cases: a i ( τ + β ) > b h ( τ + β ) , a i ( τ + β ) = b h ( τ + β ) , or a i ( τ + β ) < b h ( τ + β ) . For the case of a i ( τ ) > b h ( τ ) and a i ( τ + β ) ≥ b h ( τ + β ) , the current AoI comparison with t = τ and the future AoI comparison with t = τ + β are equivalent,because device i is allocated with an RB for both.Comparing the current AoI and the future AoI are differentif a i ( τ ) > b h ( τ ) and a i ( τ + β ) < b h ( τ + β ) , becausecomparing the current AoI allocates the RB to device i , while16omparing the future AoI allocates the RB to device h . Whenone device is allocated an RB at time slot τ and the otherdevice is allocated an RB at time slot τ + β , the RB allocationbased on current AoI achieves the average instantaneousAoI of . a i ( τ ) + 2 β b h ( τ )) , and the RB allocation basedon future AoI achieves the average instantaneous AoI of . a i ( τ )+ β + b h ( τ )) . For any β ∈ Z + , comparing the averageinstantaneous AoI of two cases is: a i ( τ ) + 2 β b h ( τ )2 > a i ( τ ) + β + b h ( τ )2 , (25) (2 β − b h ( τ ) > β, (26) b h ( τ ) > β β − . (27)Even when one device is allocated with an RB at time slot τ and the other device is allocated with an RB one time slot laterat time slot τ + 1 , the current AoI comparison yields higheraverage instantaneous AoI than the future AoI comparison.Since β β − = 1 with β = 1 , b h ( τ ) cannot be less than orequal to 1, because the condition of a i ( τ ) > b h ( τ ) and a i ( τ +1) < b h ( τ + 1) cannot be satisfied. Therefore, at time slot τ ,comparing the future AoI with t = τ + β to determine theRB allocation achieves lower average instantaneous AoI thancomparing the current AoI with t = τ . B. Proof of Theorem 1
With x ( t ) satisfying the given conditions, then at most R active devices with sufficiently high values of F i aretransmitting successfully, while rest of the devices are nottransmitting. Assuming that all other devices do not changetheir action, an active device i that is transmitting successfullywith sufficiently high F i cannot change its action x i ( t ) toget higher payoff than its current payoff of ρ . If device i uses some other RB, then its transmission may fail due tothe duplicate RB usages, getting the payoff of − γ , or itstransmission may succeed, getting the same payoff of ρ . Ifdevice i does not transmit, then the payoff is − ( γ + η ) as F i is greater than A i ( R ) . Therefore, the active devices thatare transmitting successfully with sufficiently high F i do notchange their action.The devices that are not transmitting may be active or in-active. An inactive device that is not transmitting has a payoffof ( ρ + η ) , which is higher than the payoff of transmittingsuccessfully. Therefore, the inactive devices do not transmit.With x ( t ) satisfying the given conditions, then active deviceswith F i < A i ( R ) are not transmitting. With y i ( x ( t )) , theactive devices with F i < A i ( R ) have the payoff of ( ρ + η ) ,which is higher than the payoff of transmitting successfully.Therefore, the active devices with F i < A i ( R ) do not changetheir action from not transmitting.With the design of payoff function y i ( x ( t )) (22), inactivedevices and active devices with F i < A i ( R ) have thehighest payoff of ( ρ + η ) by not transmitting. Moreover, with | A i | = | A | , at most R active devices with F i ≥ A i ( R ) havehigher payoff from transmitting successfully than from nottransmitting. With x ( t ) such that each of the RBs is used byat most one device, the active devices with F i ≥ A i ( R ) have the highest payoff of ρ by transmitting successfully. Therefore,with y i ( x ( t )) , any vector of actions ( x )( t ) such that at most R active devices with F i ≥ A i ( R ) transmit, rest of the devicesdo not transmit, and each of the RBs is used by at most onedevice is an NE. C. Proof of Theorem 2
At time slot t = 1 , x ( t ) is initialized as a random RBselection. For N = R and v a = 1 , T t = N and the service rate s r is ( ( R − / R ) N − . Therefore, from the initial RB allocation,the expected number of RBs that are used by one device is Rs r . With SCA and v a = 1 , an RB that is used by one deviceat time slot t = 1 is used by the same device at time slot t = 2 . Furthermore, with SCA, the RBs that are used by morethan one device at the time slot t = 1 are expected to beused by one device at the time slot t = 2 . Therefore, at timeslot t = 2 , E [ |L| ] = R ( ( R − / R ) N , which is the expectednumber of RBs that are used by none of the devices at timeslot t = 1 . E [ |L| ] also is the expected number of devices thatare competing to use the RBs in L , because the devices useat most RB at each time slot. Since the devices choose theRBs in L randomly, the RB selection at time slot t = 2 isequivalent to the random RB selection with the number ofdevices and RBs equal to R ( ( R − / R ) N . Moreover, the sameanalysis done for t = 1 can be done with t = 2 .Expanding to the general case, at time slot t , the expectednumber of RBs that are used by none of the devices and theexpected number of devices competing to use the RBs in L is R ( ( R − / R ) N ( t − . As t increases to infinity, the expectednumber of RBs that are used by none of the devices andthe expected number of devices competing to use the RBsin L decrease to . With SCA, this implies that the numberof RBs each used by one device increases to R as t increasesto infinity. Furthermore, with N = R and | A i | = | A | for all i , any active device i satisfies F i ≥ A i ( R ) . The action space x ( t ) converged using SCA is such that all R active devicestransmit and each of the RBs is used by one device. Therefore, x ( t ) converged using SCA is an NE. D. Proof of Proposition 2
When the T t transmitting devices use random RB selection,the service rate is equivalent to the probability of an RB beingused by only one transmitting device. Therefore, the servicerate s r is: s r = (cid:18) T t (cid:19) R (cid:18) − R (cid:19) T t − = T t R (cid:18) R − R (cid:19) T t − . (28)When the number of IoT devices N increases to infinity ina massive IoT, the number of transmitting devices T t alsoincreases to infinity with fixed v a , and the service rate s r is: lim T t →∞ s r = lim T t →∞ T t R (cid:18) − T t / R T t (cid:19) T t − , (29) = lim T t →∞ T t R − (cid:18) − T t / R T t (cid:19) T t = T t R − (cid:18) − T t R (cid:19) . (30)17 EFERENCES[1] W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems:Applications, trends, technologies, and open research problems,”
IEEENetwork , pp. 1–9, Oct. 2019.[2] L. D. Xu, W. He, and S. Li, “Internet of things in industries: A survey,”
IEEE Transactions on Industrial Informatics , vol. 10, no. 4, pp. 2233–2243, Nov. 2014.[3] M. Hassanalieragh, A. Page, T. Soyata, G. Sharma, M. Aktas, G. Mateos,B. Kantarci, and S. Andreescu, “Health monitoring and managementusing internet of things (IoT) sensing with cloud-based processing:Opportunities and challenges,” in
Proc of IEEE International Conferenceon Services Computing , New York, USA, Jun. 2015, pp. 285–292.[4] M. Mozaffari, A. T. Z. Kasgari, W. Saad, M. Bennis, and M. Debbah,“Beyond 5G with UAVs: Foundations of a 3D wireless cellular network,”
IEEE Transactions on Wireless Communications , vol. 18, no. 1, pp. 357–372, Jan. 2019.[5] M. Chen, W. Saad, and C. Yin, “Echo-liquid state deep learning for ◦ content transmission and caching in wireless VR networks with cellular-connected UAVs,” IEEE Transactions on Communications , vol. 67,no. 9, pp. 6386–6400, Sep. 2019.[6] Y. Zhang, M. Chen, N. Guizani, D. Wu, and V. C. M. Leung, “SOVCAN:Safety-oriented vehicular controller area network,”
IEEE Communica-tions Magazine , vol. 55, no. 8, pp. 94–99, Aug. 2017.[7] Z. Dawy, W. Saad, A. Ghosh, J. G. Andrews, and E. Yaacoub, “To-ward massive machine type cellular communications,”
IEEE WirelessCommunications , vol. 24, no. 1, pp. 120–128, Feb. 2017.[8] S. F. Abedin, M. G. R. Alam, S. M. A. Kazmi, N. H. Tran, D. Niyato,and C. S. Hong, “Resource allocation for ultra-reliable and enhancedmobile broadband IoT applications in fog network,”
IEEE Transactionson Communications , vol. 67, no. 1, pp. 489–502, Jan. 2019.[9] T. Park and W. Saad, “Distributed learning for low latency machine typecommunication in a massive internet of things,”
IEEE Internet of ThingsJournal , vol. 6, no. 3, pp. 5562–5576, Jun. 2019.[10] S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often shouldone update?” in
Proc. of IEEE International Conference on ComputerCommunications , Florida, USA, Mar. 2012, pp. 2731–2735.[11] B. Zhou and W. Saad, “Joint status sampling and updating for minimiz-ing age of information in the internet of things,”
IEEE Transactions onCommunications , vol. 67, no. 11, pp. 7468–7482, Nov. 2019.[12] I. Kadota, A. Sinha, E. Uysal-Biyikoglu, R. Singh, and E. Modiano,“Scheduling policies for minimizing age of information in broadcastwireless networks,”
IEEE/ACM Transactions on Networking , vol. 26,no. 6, pp. 2637–2650, Dec. 2018.[13] I. Kadota, A. Sinha, and E. Modiano, “Scheduling algorithms foroptimizing age of information in wireless networks with throughputconstraints,”
IEEE/ACM Transactions on Networking , vol. 27, no. 4,pp. 1359–1372, Aug. 2019.[14] R. Talak, S. Karaman, and E. Modiano, “Minimizing age-of-informationin multi-hop wireless networks,” in
Proc. of 55th Annual AllertonConference on Communication, Control, and Computing , Illinois, USA,Oct. 2017, pp. 486–493.[15] S. Leng and A. Yener, “Age of information minimization for wirelessad hoc networks: A deep reinforcement learning approach,” in
Proc. ofIEEE Global Communications Conference , Hawaii, USA, Dec. 2019,pp. 1–6.[16] R. D. Yates and S. K. Kaul, “Status updates over unreliable multiaccesschannels,” in
Proc. of International Symposium on Information Theory ,Aachen, Germany, Jun. 2017, pp. 331–335.[17] R. Talak, I. Kadota, S. Karaman, and E. Modiano, “Scheduling policiesfor age minimization in wireless networks with unknown channel state,”in
Proc. of IEEE International Symposium on Information Theory ,Colorado, USA, Jun. 2018, pp. 2564–2568.[18] R. Talak, S. Karaman, and E. Modiano, “Optimizing age of informationin wireless networks with perfect channel state information,” in
Proc. of16th International Symposium on Modeling and Optimization in Mobile,Ad Hoc, and Wireless Networks , Shanghai, China, May 2018, pp. 1–8.[19] B. Zhou and W. Saad, “Minimum age of information in the internetof things with non-uniform status packet sizes,”
IEEE Transactions onWireless Communications , vol. 19, no. 3, pp. 1933–1947, Mar. 2020.[20] R. Talak, S. Karaman, and E. Modiano, “Distributed scheduling al-gorithms for optimizing information freshness in wireless networks,”in
Proc. of IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications , Kalamata, Greece, Jun. 2018,pp. 1–5.[21] Y. Sun, E. Uysal-Biyikoglu, and S. Kompella, “Age-optimal updates ofmultiple information flows,” in
Proc. of IEEE Conference on ComputerCommunications Workshops , Hawaii, USA, Apr. 2018, pp. 136–141.[22] R. D. Yates and S. K. Kaul, “The age of information: Real-timestatus updating by multiple sources,”
IEEE Transactions on InformationTheory , vol. 65, no. 3, pp. 1807–1827, Mar. 2019.[23] S. Gopal and S. K. Kaul, “A game theoretic approach to DSRC andWiFi coexistence,” in
Proc. of International Conference on ComputerCommunications Workshops , Hawaii, USA, Apr. 2018, pp. 565–570.[24] A. Maatouk, M. Assaad, and A. Ephremides, “On the age of informationin a CSMA environment,”
IEEE/ACM Transactions on Networking ,vol. 28, no. 2, pp. 818–831, Apr. 2020.[25] A. M. Bedewy, Y. Sun, R. Singh, and N. B. Shroff, “Optimizinginformation freshness using low-power status updates via sleep-wakescheduling,” arXiv preprint arXiv:1910.00205 , 2019.[26] L. Huang and E. Modiano, “Optimizing age-of-information in a multi-class queueing system,” in
Proc. of IEEE International Symposium onInformation Theory , Hong Kong, China, Jun. 2015, pp. 1681–1685.[27] M. Costa, M. Codreanu, and A. Ephremides, “On the age of informationin status update systems with packet management,”
IEEE Transactionson Information Theory , vol. 62, no. 4, pp. 1897–1910, Apr. 2016.[28] Y. Inoue, H. Masuyama, T. Takine, and T. Tanaka, “The stationarydistribution of the age of information in FCFS single-server queues,” in
Proc. of IEEE International Symposium on Information Theory , Aachen,Germany, Jun. 2017, pp. 571–575.[29] E. Najm, R. Yates, and E. Soljanin, “Status updates through m/g/1/1queues with harq,” in
Proc. of IEEE International Symposium onInformation Theory , Aachen, Germany, Jun. 2017, pp. 131–135.[30] M. K. Abdel-Aziz, C. Liu, S. Samarakoon, M. Bennis, and W. Saad,“Ultra-reliable low-latency vehicular networks: Taming the age of in-formation tail,” in
Proc. of IEEE Global Communications Conference ,Abu Dhabi, UAE, Dec. 2018, pp. 1–7.[31] K. Chen and L. Huang, “Age-of-information in the presence of error,”in
Proc. of IEEE International Symposium on Information Theory ,Barcelona, Spain, Jul. 2016, pp. 2579–2583.[32] A. Maatouk, M. Assaad, and A. Ephremides, “Minimizing the ageof information: NOMA or OMA?” in
Proc. of IEEE Conference onComputer Communications Workshops , Paris, France, Apr. 2019, pp.102–108.[33] Y. Sun and B. Cyr, “Sampling for data freshness optimization: Non-linear age functions,”
Journal of Communications and Networks , vol. 21,no. 3, pp. 204–219, Jun. 2019.[34] T. Park, W. Saad, and B. Zhou, “On the minimization of non-linear ageof information in the internet of things,” submitted to Proc. of IEEEInternational Conference on Communications , pp. 1–6, Jun. 2021.[35] A. Even and G. Shankaranarayanan, “Utility-driven assessment of dataquality,”
SIGMIS Database , vol. 38, no. 2, p. 75–93, May 2007.[36] S. Ioannidis, A. Chaintreau, and L. Massoulie, “Optimal and scalabledistribution of content updates over a mobile social network,” in
Proc.of IEEE International Conference on Computer Communications , Riode Janeiro, Brazil, Apr. 2009, pp. 1422–1430.[37] A. S. Chakrabarti, C. B. K, A. Chatterjee, and M. Mitra, “The KolkataPaise Restaurant Problem and Resource Utilization,”
Physica A: Statis-tical Mechanics and its Applications , vol. 388, no. 12, pp. 2420–2426,Jun. 2009.[38] 3GPP, “Evolved universal terrestrial radio access (E-UTRA) and evolveduniversal terrestrial radio access network (E-UTRAN) (release 15),” TS36.300 v15.8.0, Jan. 2020.[39] X. Liu, Z. Qin, Y. Gao, and J. A. McCann, “Resource allocation inwireless powered iot networks,”
IEEE Internet of Things Journal , vol. 6,no. 3, pp. 4935–4945, Jun. 2019.[40] G. Hattab and D. Cabric, “Energy-efficient massive iot shared spectrumaccess over uav-enabled cellular networks,”
IEEE Transactions onCommunications , vol. 68, no. 9, pp. 5633–5648, Sep. 2020.[41] Q. Abbas, S. Zeb, S. A. Hassan, R. Mumtaz, and S. A. R. Zaidi,“Joint optimization of age of information and energy efficiency in iotnetworks,” in
Proc. of IEEE Vehicular Technology Conference , Antwerp,Belgium, May 2020, pp. 1–5.[42] Y. Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff,“Update or wait: How to keep your data fresh,”
IEEE Transactions onInformation Theory , vol. 63, no. 11, pp. 7492–7508, Nov. 2017.
43] R. D. Yates, E. Najm, E. Soljanin, and J. Zhong, “Timely updatesover an erasure channel,” in
Proc. of IEEE International Symposiumon Information Theory , Aachen, Germany, Jun. 2017, pp. 316–320.[44] A. Mazin, M. Elkourdi, and R. D. Gitlin, “Comparison of slotted aloha-NOMA and CSMA/CA for M2M communications in iot networks,” in
Proc. of IEEE Vehicular Technology Conference , Illinois, USA, Aug.2018, pp. 1–5.[45] A. Sivanathan, D. Sherratt, H. H. Gharakheili, A. Radford, C. Wije-nayake, A. Vishwanath, and V. Sivaraman, “Characterizing and classi-fying iot traffic in smart cities and campuses,” in
Proc. of InternationalConference on Computer Communications Workshops , Georgia, USA,May 2017, pp. 559–564.[46] W. Ejaz, M. Naeem, A. Shahid, A. Anpalagan, and M. Jo, “Efficientenergy management for the internet of things in smart cities,”
IEEECommunications Magazine , vol. 55, no. 1, pp. 84–91, Jan. 2017.[47] T. Park and W. Saad, “Kolkata paise restaurant game for resourceallocation in the internet of things,” in
Proc. of Asilomar Conferenceon Signals, Systems, and Computers , California, USA, Nov. 2017, pp.1774–1778.[48] O. Bello and S. Zeadally, “Intelligent device-to-device communicationin the internet of things,”
IEEE Systems Journal , vol. 10, no. 3, pp.1172–1182, Sep. 2016.[49] J. Lianghai, B. Han, M. Liu, and H. D. Schotten, “Applying device-to-device communication to enhance IoT services,”
IEEE CommunicationsStandards Magazine , vol. 1, no. 2, pp. 85–91, Jul. 2017.[50] T. Zhang, H. Fan, J. Loo, and D. Liu, “User preference aware cachingdeployment for device-to-device caching networks,”
IEEE Systems Jour-nal , vol. 13, no. 1, pp. 226–237, Mar. 2019.[51] I. O. Sanusi, K. M. Nasr, and K. Moessner, “A device to device(D2D) spectrum sharing scheme for wireless industrial applications,”in
European Conference on Networks and Communications , Valencia,Spain, Jun. 2019, pp. 353–357.[52] J. Li, G. Lei, G. Manogaran, G. Mastorakis, and C. X. Mavromous-takis, “D2d communication mode selection and resource optimizationalgorithm with optimal throughput in 5g network,”
IEEE Access , vol. 7,pp. 25 263–25 273, Feb. 2019.[53] S. Gopal, S. K. Kaul, R. Chaturvedi, and S. Roy, “A non-cooperative multiple access game for timely updates,” arXiv preprintarXiv:2001.08850 , 2020.[54] K. L. V. S. P. Sakuru and M. Visali, “Power control based resourceallocation in LTE uplinks,” in
Proc. of International Conference onCommunications and Signal Processing , Melmaruvathur, India, Apr.2015, pp. 579–582.[55] G. Hampel, C. Li, and J. Li, “5G ultra-reliable low-latency communica-tions in factory automation leveraging licensed and unlicensed bands,”
IEEE Communications Magazine , vol. 57, no. 5, pp. 117–123, May2019.[56] M. Ali, E. Hossain, and D. Kim, “LTE/LTE-A random access for massivemachine-type communications in smart cities,”
IEEE CommunicationsMagazine , vol. 55, no. 1, pp. 76–83, Jan. 2017., vol. 55, no. 1, pp. 76–83, Jan. 2017.