Delay Optimal Event Detection on Ad Hoc Wireless Sensor Networks
aa r X i v : . [ c s . N I] M a y Delay Optimal Event Detection on Ad HocWireless Sensor Networks
K. Premkumar, Venkata K. Prasanthi M., and Anurag KumarDept. of Electrical Communication Engineering,Indian Institute of Science, Bangalore, INDIAemail: [email protected], [email protected], [email protected]
We consider a small extent sensor network for event detection, in which nodes take samplesperiodically and then contend over a random access network to transmit their measurement packetsto the fusion center. We consider two procedures at the fusion center to process the measurements.The Bayesian setting is assumed; i.e., the fusion center has a prior distribution on the changetime. In the first procedure, the decision algorithm at the fusion center is network–oblivious andmakes a decision only when a complete vector of measurements taken at a sampling instant isavailable. In the second procedure, the decision algorithm at the fusion center is network–aware and processes measurements as they arrive, but in a time causal order. In this case, the decisionstatistic depends on the network delays as well, whereas in the network–oblivious case, the decisionstatistic does not depend on the network delays. This yields a Bayesian change detection problemwith a tradeoff between the random network delay and the decision delay; a higher sampling ratereduces the decision delay but increases the random access delay. Under periodic sampling, in thenetwork–oblivious case, the structure of the optimal stopping rule is the same as that without thenetwork, and the optimal change detection delay decouples into the network delay and the optimaldecision delay without the network. In the network–aware case, the optimal stopping problem isanalysed as a partially observable Markov decision process, in which the states of the queues anddelays in the network need to be maintained. A sufficient statistic for decision is found to be thenetwork–state and the posterior probability of change having occurred given the measurementsreceived and the state of the network. The optimal regimes are studied using simulation.Categories and Subject Descriptors: C.2.3 [
Computer-Communication Networks ]: Net-work Operations—
Network monitoring ; I.2.8 [
Artificial Intelligence ]: Problem Solving, ControlMethods, and Search—
Control theory
General Terms: Algorithms, Design, PerformanceAdditional Key Words and Phrases: Optimal change detection over a network, detection delay,cross–layer design of change detectionThis is an expanded version of a paper that was presented in IEEE SECON 2006. This work wassupported in part by grant number 2900 IT from the Indo-French Center for the Promotion ofAdvanced Research (IFCPAR), and in part by a project from DRDO, Government of India.Permission to make digital/hard copy of all or part of this material without fee for personalor classroom use provided that the copies are not made or distributed for profit or commercialadvantage, the ACM copyright/server notice, the title of the publication, and its date appear, andnotice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish,to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.c (cid:13)
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY, Pages 1–39. · FusionCentre
Fig. 1. An ad hoc wireless sensor network with a fusion center is shown. The small circles arethe sensor nodes (“motes”), and the lines between them indicate wireless links obtained after aself-organization procedure.
1. INTRODUCTION
A wireless sensor network is formed by tiny, untethered devices (“motes”) that cansense, compute and communicate. Sensor networks have a wide range of appli-cations such as environment monitoring, detecting events, identifying locations ofsurvivors in building and train disasters, and intrusion detection for defense andsecurity applications. For factory and building automation applications, there is in-creasing interest in replacing wireline sensor networks with wireless sensor networks,due to the potential reduction in costs of engineering, installation, operations, andmaintenance [Honeywell Inc] [ISA].Event detection is an important task in many sensor network applications. Ingeneral, an event is associated with a change in the distribution of a related quan-tity that can be sensed. For example, the event of a gas leakage at any joints in apipe causes a change in the distribution of pressure at the joint and hence can bedetected with the help of pressure sensors. In this paper, we limit our discussionto the centralized fusion model (see Figure 1), in which each mote, in an event de-tection network, senses and sends some function of its observations (e.g., quantizedsamples) to the fusion center at a particular rate. The fusion center, by appropri-ately processing the sequence of values it receives, makes a decision regarding thestate of nature, i.e., it decides whether a change has occurred or not.Our problem is that of minimizing the mean detection delay (the delay betweenthe event occurring and the detection decision at the fusion center) with a boundon the probability of false alarm. We consider a small extent network in which allthe sensors have the same coverage , i.e., when the change in distribution occurs itis observed by all the sensors and the statistics of the observations are the same atall the sensors. N sensors synchronously sample their environment at a particularrate. Synchronized operation across sensors is practically possible in networks suchas 802.11 WLANs and Zigbee networks since the access point and the PAN coordi-nator, respectively, transmit beacons that provide all nodes with a time reference.Based on the measurement samples, the nodes send certain values (e.g., quantizedsamples) to the fusion center. Each value is carried by a packet, which is transmit-ted using a contention–based multiple access mechanism. Thus, our small extent ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. X1 X2 D2 (1) D2 (2) D2 (3) t t t t t ttt sampling instantsvectors of valuesto be sent by the N sensors Fig. 2. The sensors take samples periodically at instants t , t , · · · , and prepare to send to thefusion center a vector of values X h = h X (1) h , X (2) h , · · · , X ( N ) h i at t h . Each is queued as a packet inthe queue of the respective node. Due to multiple access delays, the packets arrive with randomdelays at the fusion center; for example, for X , the delays D (1)2 , D (2)2 , D (3)2 , for the packets fromsensors 1 , network problem is a natural extension of the standard change detection problem(see [Veeravalli 2001] and the references therein) to detection over a random accessnetwork. The problem of quickest event detection problem in a large extent network(where the region of interest is much larger than the sensing coverage of any sensor)is considered by us in [Premkumar et al. 2009]. Also, a small extent network canbe thought of as a cluster in a large extent network and that the decision makercan be thought of as a cluster head.In this setting, due to the multiple access network delays between the sensornodes and the fusion center, several possibilities arise. In Figure 2 we show thatalthough the sensors take samples synchronously, due to random access delays thevarious packets sent by the sensors arrive at the fusion center asynchronously. Asshown in the figure, the packets generated due to the samples taken at time t arrive at the fusion center with a delay of D (1)2 , D (2)2 , D (3)2 , etc. It can even happenthat a packet corresponding to the samples taken at time t can arrive before oneof the packets generated due to the samples taken at time t .Figure 3 depicts a general queueing and decision making architecture in the fusioncenter. All samples are queued in per–node queues in a sequencer. The way thesequencer releases the packets gives rise to the following three cases, the first twoof which we study in this paper .(1) The sequencer queues the samples until all the samples of a “batch” (a batchis the set of samples generated at a sampling instant) are accumulated; it thenreleases the entire batch to the decision device. The batches arrive to thedecision maker in a time sequence order. The decision maker processes thebatches without knowledge of the state of the network (i.e., reception timesat the fusion center, and the numbers of packets in the various queues). Wecall this, Network Oblivious Decision Making ( NODM ). In factory and buildingautomation scenarios, there is a major impetus to replace wireline networksbetween sensor nodes and controllers. In such applications, the first step couldbe to retain the fusion algorithm in the controller, while replacing the wireline
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · fusion centre MakerDecisionNetworkRandom AccessandSensors Sequencer
Fig. 3. A conceptual block diagram of the wireless sensor network shown in Figure 1. The fusioncenter has a sequencing buffer which queues out–of–sequence samples and delivers the samples tothe decision maker in time–order, as early as possible, batch–wise or sample–wise. network with a wireless ad hoc network. Indeed, we show that this approachis optimal for
NODM , provided the sampling rate is appropriately optimized.(2) The sequencer releases samples only in time–sequence order but does not waitfor an entire batch to accumulate. The decision maker processes samples asthey arrive. We call this,
Network Aware Decision Making ( NADM ). In
NADM ,whenever the decision maker receives a sample, it has to roll back its decisionstatistic to the sampling instant, update the decision statistic with the receivedsample and then update the decision statistic to the current time slot. Thedecision maker makes a Bayesian update on the decision statistic even if itdoes not receive a sample in a slot. Thus,
NADM requires a modification in thedecision making algorithm in the fusion center.(3) The sequencer does not queue any samples. The fusion center acts on the valuesfrom the various sampling instants as they arrive, possibly out of order. Theformulation of such a problem would be an interesting topic for future research.
Our Contributions:
We find that, in the existing literature on sequential changedetection problems (see discussion on related literature below), it has been assumedthat, at a sampling instant, the samples from all the sensors reach the fusion centerinstantaneously. As explained above, however, in our problem the delay in detectionis not only due to the detection procedure requiring a certain amount of samples tomake a decision (which we call decision delay ), but also due to the random packetdelay in the multiple access network (which we call network delay ). We work witha formulation that accounts for both these delays, while limiting ourselves to theparticular fusion center behaviours explained in cases (1) and (2) above.In Section 2, we discuss the basic change detection problem and setup the model.In Section 3, we formulate the change detection problem over a random accessnetwork in a way that naturally includes the network delay. We show that in thecase of
NODM , the problem objective decouples into a part involving the networkdelay and a part involving the optimal decision delay, under the condition thatthe sampling instants are periodic. Then, in Section 4, we consider the specialcase of a network with a star topology, i.e., all nodes are one hop away from thefusion center and provide a model for contention in the random access network.
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. In Section 5, we formulate the
NADM problem where we process the samples asthey arrive at the fusion center, but in a time causal manner. The out–of–sequencepackets are queued in a sequencing buffer and are released to the decision maker asearly as possible. We show in the
NADM case that the change–detection problemcan be modeled as a Partially Observable Markov Decision Process (POMDP).We show that a sufficient statistic for the observations include the network–state (which include the queue lengths of the sequencing buffer, network–delays) and theposterior probability of change having occurred given the measurements receivedand the network states. As usual, the optimal policy can be characterised via aBellman equation, which can then be used to derive insights into the structure of thepolicy. We show that the optimal policy is a threshold on the posterior probabilityof change and that the threshold, in general, depends on the network state . Finally,in Section 6 we compare, numerically, the mean detection delay performance of
NODM and a simple heuristic algorithm motivated by
NADM processing. We showthe tradeoff between the sampling rate r and the mean detection delay. Also, weshow the tradeoff between the number of sensors and the mean detection delay. Related Literature:
The basic mathematical formulation in this paper is anextension of the classical problem of sequential change detection in a Bayesianframework. The centralized version of this problem was solved by Shiryaev (see[Shiryaev 1978]). The decentralized version of the problem was introduced by Tennyand Sandell [Tenny and Sandell 1981]. In the decentralized setting, Veeravalli[Veeravalli 2001] provided optimal decision rules for the sensors and the fusioncenter, in the context of conditionally independent sensor observations and a quasi-classical information structure. For a large network setting, Niu and Varshney [Niuand Varshney 2005] studied a simple hypothesis testing problem and proposed a counting rule based on the number of alarms. They showed that, for a sufficientlylarge number of sensors, the detection performance of the counting rule is closeto that of the optimal rule. In a recent article on anomaly detection in wirelesssensor networks [Rajasegarar et al. 2008], Rajasegarar et al. have provided a surveyof statistical and machine learning based techniques for detecting various types ofanomalies such as sensor faults, security attacks, and intrusions. In [Aldosari andMoura 2004] the authors consider the problem of decentralized binary hypothesistesting, where the sensors quantize the observations and the fusion center makes abinary decision between the two hypotheses.
Remark:
In the existing literature on the topic of optimal sequential eventdetection in wireless sensor networks, to the best of our knowledge there has been noprior formulation that incorporates multiple access delay between the sensing nodesand the fusion center. Interestingly, in this paper we introduce, what can be calleda cross layer formulation involving sequential decision theory and random accessnetwork delays.
In particular, we encounter the fork–join queueing model (see,for example, [Baccelli and Makowski 1990]) that arises in distributed computingliterature.
2. THE BASIC CHANGE DETECTION PROBLEM
In this section, we introduce the model for the basic change detection problem. Thenotation, we follow, is given here.
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · ) t Slots t b+1b Sampling instants k k+1k−1
Beginning of Slot k
Sampling interval of /r slots Fig. 4. Time evolves over slots. The length of a slot is assumed to be unity. Thus, slot k representsthe interval [ k, k + 1) and the beginning of slot k represents the time instant k . Samples are takenperiodically every 1 /r slots, starting from t = 1 /r . K~ t K~ UT~ U~t t t t t ttt
U U U U U
Tchange time detection timewithout n/w delay detection timewith n/w delaynetwork delaycoarse sampling delay
Fig. 5. Change time and the detection instants with and without network delay are shown. Thecoarse sampling delay is given by t K − T where t K is the first sampling instant after change, andthe network delay is given by U ˜ K − t ˜ K . • Time is slotted and the slots are indexed by k = 0 , , . . . . We assume that thelength of a slot is unity and that slot k refers to the interval [ k, k + 1). Thus, thebeginning of slot k indicates the time instant k (see Figure 4). • N sensors are synchronously sampling at the rate r samples/slot, i.e., the sensorsmake an observation every 1 /r slots and send their observations to the fusioncenter. Thus, for example, if r = 0 .
1, then a sample is taken by a sensor every10 th slot. We assume that 1 /r is an integer. The sampling instants are denoted t , t , . . . (see Figure 5). Define t = 0; note that the first sample is taken at t = 1 /r . • The vector of network delays of the batch b is denoted by D b = h D (1) b , D (2) b , · · · , D ( N ) b i where D ( i ) b ∈ { , , , · · · } is the network delay in slots, of the i th component ofthe b th batch (sampled at t b = b/r ). Also, note that D ( i ) b >
1, as it requires onetime slot for the transmission of a packet to the fusion center after a successfulcontention. • The state of nature Θ ∈ { , } . 0 represents the state “before change” and ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. “after change” . It is assumed that the change time T (measured in slots), is geometrically distributed i.e., P ( T = 0) = ρ and, for k > , P ( T = k | T >
0) = p (1 − p ) ( k − . (1)The value of 0 for T accounts for the possibility that the change took place beforethe observations were made. • The vector of outputs from the sensor devices at the b th batch is denoted by X b = h X (1) b , X (2) b , · · · , X ( N ) b i where X ( i ) b ∈ X is the b th output at the i th sensor. Given the state of nature, X ( i ) b are assumed to be (conditionally) independent across sensors and i.i.d. oversampling instants with probability distributions F ( x ) and F ( x ) before and afterthe change respectively. X corresponds to the first sample taken. In this work,we do not consider the problem of optimal processing of the sensor measurementsto yield the sensor outputs, e.g., optimal quantizers (see [Veeravalli 2001]). • Let S b , b >
1, be the state of nature at the b th sampling instant and S the stateat time 0. Then S b ∈ { , } , with P ( S = 1) = ρ = 1 − P ( S = 0) S b evolves as follows. If S b = 0 for b >
0, then S b +1 = (cid:26) p r − p r )where p r = 1 − (1 − p ) /r . Further, if S b = 1, then S b +1 = 1. Thus, if S = 0,then there is a change from 0 to 1 at the K th sampling instant, where K isgeometrically distributed. For b > P ( K = b ) = p r (1 − p r ) b − Each value to be sent to the fusion center by a node is inserted into a packetwhich is queued for transmission. It is then transmitted to the fusion center byusing a contention based multiple access protocol. A node can directly transmitits observation to the fusion center or route it through other nodes in the system.Each packet takes a time slot to transmit. The MAC protocol and the queuesevolve over the same time slots. The fusion center makes a decision about thechange depending on whether
Network Oblivious ( NODM ) processing or
NetworkAware ( NADM ) processing is employed at the fusion center. In the case of
NODM processing, the decision sequence (also called as action sequence ), is A u , u >
0, with A u ∈ { stop and declare change (1) , take another sample (0) } , where u is a time in-stant at which a complete batch of N samples corresponding to a sampling instant isreceived by the fusion center. In the case of NADM processing, the decision sequenceis A k , k >
0, with A k ∈ { stop and declare change (1) , take another sample (0) } , i.e.,a decision about the change is taken at the beginning of every slot. ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · X k(1) X k(2) X k(N) . . . . MACsvc.rate1 N σ sampling . . . . Transmitter buffer 1 N Maker
Decision fusion centreSequencer
Fig. 6. A sensor network model of Figure 3 with one hop communication between the sensornodes and the fusion center. The random access network along with the sequencer is a fork–joinqueueing model.
3. NETWORK OBLIVIOUS DECISION MAKING (NODM)
From Figure 2, we note that although all the components of a batch b are generatedat t b = b/r , they reach the fusion center at times t b + D ( i ) b , i = 1 , , · · · , N . In NODM processing, the samples, which are successfully transmitted, are queuedin a sequencing buffer as they arrive (see Figure 6) and the sequencer releases a(complete) batch to the decision maker, as soon as all the components of a batcharrive. The decision maker makes a decision about the change at the time instantswhen a (complete) batch arrives at the fusion center. In the Network Oblivious(
NODM ) processing, the decision maker is oblivious to the network and processesthe batch as though it has just been generated (i.e., as if there is no network, hencethe name
Network Oblivious Decision Making ). We further define (see Figure 5) • U b , ( b > X b • e K ∈ { , , . . . } : the batch index at which the decision takes place, if there wasno network delay. e K = 0 means that the decision 1 ( stop and declare change ) istaken before any batch is generated • e T = t e K : the random time (a sampling instant) at which the fusion center stopsand declares change, if there was no network delay • e U = U e K : the random time slot at which the fusion center stops and declareschange, in the presence of network delay • D b = U b − t b : Sojourn time of the b th batch, i.e., the time taken for all thesamples of the b th batch to reach the fusion center. Note that D b is given bymax { D ( i ) b : i = 1 , , · · · , N } . Thus, the delay of the batch e K at which thedetector declares a change is U e K − t e K = e U − e T We define the following detection metrics.
Mean Detection Delay defined as the expected number of slots between thechange point T and the stopping time instant e U in the presence of coarse sampling and network delays, i.e., Mean Detection Delay = E h(cid:16) e U − T (cid:17) { e T > T } i . ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. Mean Decision Delay defined as the expected number of slots between the changepoint T and the stopping time instant e T in the (presence of coarse sampling delayand in the) absence of network delay, i.e., Mean Decision Delay = E h(cid:16) e T − T (cid:17) { e T > T } i .With the above model and assumptions, we pose the following NODM problem:
Minimize the mean detection delay with a bound on the probability of false alarm ,the decision epochs being the time instants when a complete batch of N componentscorresponding to a sampling instant is received by the fusion center. In Section 5,we pose the problem of making a decision at every slot based on the samples asthey arrive at the fusion center. Motivated by the approach in [Veeravalli 2001] weuse the following formulation for a given sampling rate r min E h ( e U − T ) { e T > T } i (2)such that P (cid:16) e T < T (cid:17) α where α is the constraint on the false alarm probability. T~ U~T
Fig. 7. Illustration of an event of false alarm with e T < T , but e U > T
Remark 3.1
Note that if α > − ρ , then the decision making procedure can bestopped and an alarm can be raised even before the first observation. Thus, weassume that α < − ρ . Remark 3.2
Note that here we consider P (cid:16) e T < T (cid:17) as the probability of falsealarm and not P (cid:16) e U < T (cid:17) , i.e., a case as shown in Figure 7 is considered a falsealarm. This can be understood as follows: when the decision unit detects a changeat slot e U , the measurements that triggered this inference would be carrying the“time stamp” e T , and we infer that the change actually occurred at or before e T .Thus if e T < T , it is an error.We write the problem defined in Eqn. 2 asmin Π α E h ( e U − T ) { e T > T } i (3)where Π α is the set of detection policies for which P (cid:16) e T < T (cid:17) α . Theorem 1
If the sampling is periodic at rate r and the batch sojourn time process D b , b > , is stationary with mean d ( r ) , then min Π α E h ( e U − T ) { e T > T } i = ( d ( r ) + l ( r ))(1 − α ) − ρ · l ( r ) + 1 r min Π α E h e K − K i + where l ( r ) is the delay due to (coarse) sampling. ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Remark 3.3
For example in Figure 5, the delay due to coarse sampling is t − T , e K − K = 3 − U − t . The stationarity assumptionon D b , b >
1, is justifiable in a network in which measurements are continuouslymade, but the detection process is started only at certain times, as needed.
Proof:
The following is a sketch of the proof (the details are in the Appendix –I) min Π α E h ( e U − T ) { e T > T } i = min Π α (cid:26) E h ( e U − e T ) { e T > T } i + E h e T − T i + (cid:27) = min Π α (cid:26) E [ D ] (cid:16) − P (cid:16) e T < T (cid:17)(cid:17) + E h e T − T i + (cid:27) where we have used the fact that under periodic sampling, the queueing systemevolution and the evolution of the statistical decision problem are independent,i.e., e K is independent of { D , D , . . . } and E [ D ] is the mean stationary queueingdelay (of a batch). By writing E [ D ] = d ( r ) and using the fact that the problemmin Π α E h e T − T i + is solved by a policy π ∗ α ∈ Π α with P (cid:16) e T < T (cid:17) = α , the problembecomes d ( r )(1 − α ) + min Π α E h e T − T i + = ( d ( r ) + l ( r ))(1 − α ) − ρ · l ( r ) + 1 r min Π α E h e K − K i + where l ( r ) is the delay due to sampling. Notice that min Π α E h e K − K i + is the basicchange detection problem at the sampling instants. Remark 3.4
It is important to note that the independence between e K and { D , D , . . . } arises from periodic sampling. Actually this is conditional independence given therate of the periodic sampling process. If, in general, one considers a model in whichthe sampling is at random times (e.g., the sampling process randomly alternatesbetween periodic sampling at two different rates or if adaptive sampling is used)then we can view it as a time varying sampling rate and the asserted independencewill not hold.We conclude that the problem defined in Eqn. 2 effectively decouples into the sumof the optimal delay in the original optimal detection problem, i.e., r min Π α E h e K − K i + as in [Veeravalli 2001], a part that captures the network delay, i.e., d ( r )(1 − α ), anda part that captures the sampling delay, i.e., l ( r )(1 − α ) − ρl ( r ).
4. NETWORK DELAY MODEL
From Theorem 1, it is clear that in
NODM processing, the optimal decision deviceand the queueing system are decoupled. Thus, one can employ an optimal sequentialchange detection procedure (see [Shiryaev 1978]) for any random access network(in between the sensor nodes and the fusion center). Also,
NODM is oblivious tothe random access network (in between the sensor nodes and the fusion center)and processes a received batch as though it has just been generated. In the caseof
NADM (which we describe in Section 5), the decision maker processes samples,
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. FusionCentre
Fig. 8. A sensor network with a star topology with the fusion center at the hub. The sensor nodesuse a random access MAC to send their packets to the fusion center. keeping network–delays into account, thus requiring the knowledge of the networkdynamics. In this section, we provide a simple model for the random access network,that facilitates the analysis and optimisation of
NADM . N sensors form a star topology (see Figure 8) ad hoc wireless sensor networkwith the fusion center as the hub. They synchronously sample their environmentat the rate of r samples per slot periodically. At sampling instant t b = b/r , sensornode i generates a packet containing the sample value X ( i ) b (or some quantizedversion of it). This packet is then queued first-in-first-out in the buffer behind theradio link. It is as if each sample is a fork operation that puts a packet into eachsensor queue (see Figure 6).The sensor nodes contend for access to the radio channel, and transmit packetswhen they succeed. The service is modeled as follows. As long as there are packetsin any of the queues, successes are assumed to occur at the constant rate of σ (0 <σ <
1) per slot, with the intervals between the successes being i.i.d., geometricallydistributed random variables, with mean 1 /σ . If, at the time a success occurs, thereare n nodes contending (i.e., n queues are nonempty) then the success is ascribedto any one of the n nodes with equal probability.The N packets corresponding to a sample arrive at random times at the fusioncenter. If the fusion center needs to accumulate all the N components of eachsample then it must wait for that component of every sample that is the last todepart its mote. This is a join operation (see Figure 6).It is easily recognized that our service model, in the case of NODM is the discretetime equivalent of generalized processor sharing (GPS – see, for example, [Kumaret al. 2004]), which can be called the FJQ-GPS (fork-join queue (see [Baccelli andMakowski 1990]) with GPS service). In the case of
NADM , the service model is justthe GPS.In IEEE 802.11 networks and IEEE 802.15.4 networks, if appropriate parametersare used, then the adaptive backoff mechanism can achieve a throughput that isroughly constant over a wide range of n , the number of contending nodes. This Note that
Theorem 1 is more general and does not assume a star topology.
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · η vs n for different transmission rates (C) in RTS−CTSNumber of nodes (n)−−−> A gg r e g a t e t h r oughpu t η ( M bp s ) −−−> AnalysisSimulationC= 11 Mbps C= 5.5 Mbps C= 2.2 Mbps
Fig. 9. The aggregate saturation throughput η of an IEEE 802.11 network plotted against thenumber of nodes in the network, for various physical layer bit rates: 2.2 Mbps, 5.5 Mbps, and11 Mbps . The two curves in each plot correspond to an analysis and an NS–2 simulation. η η Fig. 10. The aggregate saturation throughput η of an IEEE 802.15.4 star topology network plottedagainst the number of nodes in the network. Throughput obtained with default backoff parametersis shown on the left and that obtained with backoff multiplier = 3, is shown on the right. Thetwo curves in each plot correspond to an analysis and an NS–2 simulation. is well known for the CSMA/CA implementation in IEEE 802.11 wireless LANs;see, for example, Figure 9 [Kumar et al. 2008]. For each physical layer rate, thenetwork service rate remains fairly constant with increasing number of nodes. FromFigure 10 (taken from [Singh et al. 2008]) it can be seen that with the default backoffparameters, the saturation throughput of a star topology IEEE 802.15.4 networkdecreases with the number of nodes N , but with the backoff multiplier = 3, thethroughput remains almost constant from N = 10 to N = 50 [Singh et al. 2008];thus, in the latter case our GPS model can be applicable. Theorem 2
The stationary delay D is a proper random variable with finite meanif and only if N r < σ .Proof:
See Appendix – II.Thus, for the FJQ–GPS queueing system to be stable, the sampling rate r is chosensuch that r < σN . ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY.
5. NETWORK AWARE DECISION MAKING (NADM)
In Section 3, we formulated the problem of
NODM quickest change detection overa random access network, and showed that (when the decision instants are U k , asshown in Figure 5) the optimal decision maker is independent of the random accessnetwork, under periodic sampling. Hence, the Shiryaev procedure, which is shownto be delay optimal in the classical change–detection problem (see [Shiryaev 1978]),can be employed in the decision device independently of the random access network.It is to be noted that the decision maker in the NODM case, waits for a completebatch of N samples to arrive, to make a decision about the change. Thus, the meandetection delay of the NODM has a network–delay component corresponding to abatch of N samples. In this section, we provide an alternative mechanism of fusionat the decision device called Network Aware Decision Making ( NADM ), in whichthe fusion algorithm does not wait for an entire batch to arrive, and processes thesamples as soon as they arrive, but in a time–causal manner.We now describe the processing in
NADM . Whenever a node (successfully) trans-mits a sample across the random access network, it is delivered to the decisionmaker if the decision maker has received all the samples from all the batches gen-erated earlier. Otherwise, the sample is an out–of–sequence sample, and is queuedin the sequencing buffer. It follows that, whenever the (successfully) transmittedsample is the last component of the batch that the decision maker is looking for,then the head of line (HOL) components, if any, in the queues of the sequencingbuffer are also delivered to the decision maker. This is because, these HOL samplesbelong to the next batch that the decision maker should process. The decisionmaker makes a decision about the change at the beginning of every time slot , irre-spective of whether it receives a sample or not. In
NADM , whenever the decisionmaker receives a sample, it takes into account the network–delay of the samplewhile computing the decision statistic. The network–delay is a part of the state ofthe queueing system which is available to the decision maker. Thus, unlike NODM, the state of the queueing system also plays a role in decision making .In Section 5.1, we define the state of the queueing system. In Section 5.2, wedefine the dynamical system whose change of state (from 0 to 1) is the subject ofinterest to us. We define the state of the dynamical system as a tuple that containsthe queueing state , the state of nature , and a delayed state of nature . The delayedstate of nature is included in the state of the system so that the (delayed) sensor–observations that the decision maker receives at time instant k + 1 depend only onthe state, the control, and the noise of the system at time instant k , a propertywhich is desirable to define a sufficient statistic (see page 244, [Bertsekas 2000a]).We explain the evolution of the state of the dynamical system in Section 5.3. InSection 5.4, we formulate the NADM change detection problem and we find a suf-ficient statistic for the observations in Section 5.5. In Section 5.6, we provide theoptimal decision rule for the
NADM change detection problem.
Recall the notation introduced in Section 2. Time progresses in slots, indexed by k = 0 , , · · · ; the beginning of slot k is the time instant k . Also, the time instant ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · t Slots t b+1b t B k k k+1 ∆ k λ k Fig. 11. At time k , the decision maker expects samples (or processes samples) from batch B k .Also, at time k , λ k is the number of slots to go for the next sampling instant and ∆ k is the numberof slots back at which batch B k is generated. t Slots t k B k k+1k−1 B k−1 r k−1 B Last component of batchreached at time k k k+1 ∆ = 0 ∆ = 0 ∆ = 0 Bk t Fig. 12. Illustration of a scenario in which ∆ k = 0. If the last component from batch B k − isreceived at k , and if there is no sampling instant between t B k − and k , then ∆ k = 0. Also, note inthis case that ∆ k = ∆ k +1 = · · · = ∆ t Bk = 0. In this scenario, at time instants k, k + 1 , · · · , t B k ,all the queues at the sensor nodes and at the sequencer are empty, and at time instant t B k +, allsensor node queues have one packet which is generated at t B k . just after the beginning of time slot is denoted by k + . Recall that the nodestake samples at the instants 1 /r , 2 /r , 3 /r , · · · . We define the state of the queueingsystem here. Note that the queueing system evolves over slots. • λ k ∈ { , , · · · , /r } denotes the number of time slots to go for the next samplinginstant, at the beginning of time slot k (see Figure 11). Thus, λ k := 1 r − (cid:18) k mod 1 r (cid:19) . (4)Thus, λ = r , λ = r − , · · · , and at the sampling instants t b , λ t b = r . • B k ∈ { , , , · · · } denotes the index of the batch that is expected to be or isbeing processed by the decision maker at the beginning of time slot k . Note B = B = · · · = B /r = 1. Also, note that the batch B k is generated at timeinstant B k /r . • ∆ k ∈ { , , , · · · } denotes the delay in number of time slots between the timeinstants k and B k /r (see Figure 11).∆ k := max (cid:26) k − B k r , (cid:27) . (5) Note that the notation t + denotes a time embedded to the right of t and is different from thenotation ( x ) + . Recall that ( x ) + := max { x, } . ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. slotsA new sample is taken here ifk+1k Departure w.p. if L ( i ) k λ k +1 = 1 /r σN k L ( i ) k > L ( i ) k +1 = L ( i ) k − { M k = i } + { λ k +1 =1 /r } Fig. 13. The evolution of L ( i ) k from time slot k to time slot k + 1. If during time slot k , node i transmits (successfully) a packet to the fusion center (i.e., M k = i ), then that packet is flushed outof its queue at the end of time slot k . Also, a new sample is generated (every 1 /r slots) exactlyat the beginning of a time slot. Thus, L ( i ) k +1 , the queue length of sensor node i just after thebeginning of time slot k + 1 (i.e., at ( k + 1)+) is given by L ( i ) k +1 = L ( i ) k − { M k = i } + { λ k +1 =1 /r } . Note that the batches of samples taken after B k /r and up to (including) k arequeued either in the sensor node queues or in the sequencing buffer in the fusioncenter. If at time k , the fusion center receives a sample which is the last samplefrom batch B k − , then B k = B k − + 1. If the sampling instant of the B k th batchis later than k (i.e., B k /r > k ), then ∆ k = 0 (up to time B k /r at which instant,a new batch is generated). This corresponds to the case, when all the samplesgenerated up to time slot k , have already been processed by the decision maker(see Figure 12). In particular, ∆ = ∆ = · · · = ∆ r − = 0. • L ( i ) k ∈ { , , , · · · } denotes the queue length of the i th sensor node just afterthe beginning of time slot k (i.e., at time instant k +). The vector of queuelengths is L k = [ L (1) k , L (2) k , · · · , L ( N ) k ]. Let N k ∈ { , , , · · · , N } be the numberof non–empty queues at the sensor nodes, just after the beginning of time slot k . N k := N X i =1 { L ( i ) k > } i.e., the number of sensor nodes that contend for slot k is N k . Hence, using thenetwork model we have provided in Section 4, the evolution of L ( i ) k (see Figure 13)is given by the following: L ( i )0 = 0 L ( i ) k +1 = L ( i ) k + { λ k +1 =1 /r } w . p . N k = 0 ,L ( i ) k + { λ k +1 =1 /r } w . p . (1 − σ ) if N k > , max { L ( i ) k − , } + { λ k +1 =1 /r } w . p . σN k if N k > . Note that when all the samples generated up to time slot k have already beenprocessed by the decision maker and k is not a sampling instant, i.e., ∆ k = 0 and λ k = 1 /r , then L k = (as there are no outstanding samples in the system). Fore.g., L = L = · · · = L /r − = . Also, note that just after sampling instant t b , L ( i ) t b > ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · slotsk+1kA new sample is received by the fusion center here if w.p. if W ( i ) k W ( i ) k +1 M k = j L ( j ) k > M k = j σN k Fig. 14. The evolution of W ( i ) k from time slot k to time slot k + 1. If a sample from node i istransmitted (successfully) during time slot k (i.e., M k = i ), then it is received by the fusion centerat the end of time slot k (i.e. at ( k + 1) − ). If this sample is from batch B k , it is passed on to thedecision maker. Otherwise, it is queued in the sequencing buffer, in which case W ( i ) k +1 = W ( i ) k + 1.On the other hand, if a sample from some other node j is transmitted (successfully) during timeslot k (i.e., M k = j = i ), and if this sample is the last component to be received from batch B k by the fusion center, then the HOL packet of the i th sequencing queue, if any, is also delivered tothe decision maker. Thus, in this case, W ( i ) k +1 = max { W ( i ) k − , } . Note that W ( i ) k +1 refers to thequeue length corresponding to node i at the sequencer, at the beginning of time slot k + 1. • M k ∈ { , , , · · · , N } denotes the index of the node that successfully transmitsin slot k . M k = 0 means that there is no successful transmission in slot k . Thus,from the network model we have provided in Section 4, we note that M k = . p . N k = 00 w . p . (1 − σ ) if N k > j w . p . σN k if L ( j ) k > , j = 1 , , · · · , N • W ( i ) k ∈ { , , , · · · } denotes the queue length of the i th sequencing buffer at time k . The vector of queue lengths is given by W k = [ W (1) k , W (2) k , · · · , W ( N ) k ]. Notethat W k = if ∆ k = 0, i.e., the sequencing buffer is empty if there are nooutstanding samples in the system. In particular, W = W = · · · = W r = .The evolution of W ( i ) k is explained in Figure 14. If a sample from node i of a batchlater than B k is successfully transmitted during slot k , then W ( i ) k +1 = W ( i ) k + 1.If a sample from node j of batch B k is successfully transmitted and if it is thelast sample to be received from batch B k , then the queue lengths of sequencingbuffer are decremented by 1, i.e., W ( i ) k +1 = max { W ( i ) k − , } . • R ( i ) k ∈ { , } denotes whether the sample X ( i ) B k has been received and processedby the decision maker at time k . R ( i ) k = 0 means that the sample X ( i ) B k has notyet been received by the decision maker and R ( i ) k = 1 means that the sample X ( i ) B k has been received and processed by the decision maker. The vector of R ( i ) k s isgiven by R k = [ R (1) k , R (2) k , · · · , R ( N ) k ]. Note that, if R ( i ) k = 0, W ( i ) k = 0, i.e., the i th sequencing buffer is empty if the sample expected by the decision maker hasnot yet been transmitted. Also note that when ∆ k = 0, R k = , as the samplesfrom the current batch B k have yet to be generated or have just been generated.We now relate the queue lengths L ( i ) k and W ( i ) k . Note that at the beginning oftime slot k , j k /r k batches have been generated so far, of which B k − ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. completely received by the decision maker. In batch B k , i th sample is received bythe decision maker if R ( i ) k = 1. Hence, at time k , B k − R ( i ) k samples generatedby node i have been processed by the decision maker and the remaining samplesare in the sensor and sequencing queues. Thus, we have L ( i ) k + W ( i ) k = (cid:22) k /r (cid:23) − ( B k − − R ( i ) k (6)= (cid:22) k − B k /r + 1 /r /r (cid:23) − R ( i ) k = j ∆ k /r k + 1 − R ( i ) k if k > B k /r − R ( i ) k if k = B k /r − R ( i ) k if k < B k /r Recalling the definition of ∆ k , we write the above Eqn. as, L ( i ) k + W ( i ) k = j ∆ k /r k + 1 − R ( i ) k if ∆ k >
01 if ∆ k = 0 , λ k = 1 /r k = 0 , λ k = 1 /r. (7)Note that in the above Eqn. ∆ k = 0 , λ k = 1 /r (or equivalently k = B k /r ),corresponds to the case when the samples of batch B k have just been taken andall the samples from all previous batches have been processed. Thus, in this case L ( i ) k = 1 (as W ( i ) k = 0). In the case of ∆ k = 0 , λ k = 1 /r (or equivalently k < B k /r ),all the samples from all previous batches have been processed and a new samplefrom batch B k is not taken yet. Thus, in this case L ( i ) k = 0 (and W ( i ) k = 0). Hence,given Q k = [ λ k , B k , ∆ k , W k , R k ], the queue lengths L ( i ) k s can be computed as L ( i ) k = φ L ( i ) ( Q k ) := j ∆ k /r k + 1 − R ( i ) k − W ( i ) k if ∆ k >
01 if ∆ k = 0 , λ k = 1 /r k = 0 , λ k = 1 /r . (8)Also , N k = φ N ( Q k ) := N X i =1 { φ L ( i ) ( Q k ) > } . (9)Thus, the state of the queueing system at time k , can be expressed as Q k =[ λ k , B k , ∆ k , W k , R k ]. Note that the decision maker can observe the state Q k per-fectly . The evolution of the queueing system is explained in the next subsection. The evolution of the queueing system from time k to time k + 1 depends onlyon M k , the success/no–success of contention on the random access channel. Notethat the evolution of λ k is deterministic and that of ∆ k depends on B k . Hence,to describe the evolution of Q k , it is enough to explain the evolution of B k , W k ,and R k for various cases of M k . Let Y k +1 ∈ {∅} ∪ (cid:0) ∪ Nn =1 X n (cid:1) denote the vector ofsamples received, if any, by the decision maker at the beginning of slot k + 1 (i.e.,the decision maker can receive a vector of n samples where n ranges from 0 to N ). ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · At the beginning of time slot k + 1, the following possibilities arise: • No successful transmission:
This corresponds to the case i) when all thequeues are empty at the sensor nodes ( N k = 0), or ii) when some queues arenon–empty at the sensor nodes ( N k > M k = 0 and thedecision maker does not receive any sample, i.e., Y k +1 = ∅ . In this case, it isclear that B k +1 = B k , W k +1 = W k , and R k +1 = R k . • Successful transmission of node j ’s sample from a later batch: Thiscorresponds to the case, when the decision maker has already received the j thcomponent of the current batch B k (i.e., R ( j ) k = 1) and that it has not receivedsome sample, say i = j , from the batch B k (i.e., R ( i ) k = 0, for some i ). Thereceived sample (is an out–of–sequence sample and) is queued in the sequencingbuffer ( W ( j ) k +1 = W ( j ) k +1). Thus, in this case, M k = j and the decision maker doesnot receive any sample, i.e., Y k +1 = ∅ . In this case, it is clear that B k +1 = B k , W k +1 = W k + e j , and R k +1 = R k . • Successful transmission of node j ’s current sample which is not thelast component of the batch B k : This corresponds to the case when thedecision maker has not received the j th component of the batch B k before timeslot k ( R ( j ) k = 0), and that it has received all the samples that are generatedearlier than that of the successful sample. Also, the fusion center is yet to receivesome other component of batch B k (i.e., P Ni =1 R ( i ) k < N − M k = j and the decision maker receives the sample Y k +1 = X ( j ) B k . In this case,it is clear that B k +1 = B k , W k +1 = W k , and R k +1 = R k + e j . • Successful transmission of node j ’s current sample which is the lastcomponent of the batch B k : This corresponds to the case when the decisionmaker has not received the j th component of the batch B k before time slot k ( R ( j ) k = 0), and that it has received all the samples that are generated earlierthan that of the successful sample. Also, this sample is the last component ofbatch B k , that is received by the fusion center. (i.e., P Ni =1 R ( i ) k = N − head of line (HOL) components (which correspond to the batch index B k + 1), if any, to the decision maker and the queues are decremented by one( W ( i ) k +1 = max { W ( i ) k − , } ). Thus, M k = j and the decision maker receives thevector of samples Y k +1 = h X ( j ) B k , X ( i ′ ) B k +1 , X ( i ′ ) B k +1 , · · · , X ( i ′ n − ) B k +1 i where W ( i ) k > i ∈ { i ′ , i ′ , · · · i ′ n − } , and W ( i ) k = 0 for i / ∈ { i ′ , i ′ , · · · i ′ n − } . In this case, B k +1 = B k + 1, W k +1 = W k − e i ′ − e i ′ − · · · − e i ′ n − , and R k +1 = e i ′ + e i ′ + · · · + e i ′ n − .Thus, the state of the queueing system at time k + 1 can be described by Q k +1 = φ Q ( Q k , M k ):= [ φ λ ( Q k , M k ) , φ B ( Q k , M k ) , φ ∆ ( Q k , M k ) , φ W ( Q k , M k ) , φ R ( Q k , M k )] . In the next subsection, we provide a model of the dynamical system whose statehas the state of nature Θ k as one of its constituents. The quickest detection ofchange of Θ k from 0 to 1 (at a random time T ) is the focus of this paper. ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. Let Θ k ∈ { , } , k >
0, be the state of nature at the beginning of time slot k .Recall that T is the change point, i.e., for k < T , Θ k = 0 and for k > T , Θ k = 1,and that the distribution of T is given in Eqn. 1. The state Θ k is observed onlythrough the sensor measurements, but these are delayed. We will formulate theoptimal NADM change detection problem as a partially observable Markov decisionprocess (POMDP) with the delayed observations. The approach and the terminol-ogy used here is in accordance with the stochastic control framework in [Bertsekas2000a]. At time k , a sample, if any, that the decision maker receives is generatedat time B k /r < k (i.e., samples arrive at the decision maker with a network–delayof ∆ k = k − B k r slots). To make an inference about Θ k from the sensor measure-ments, we must consider the vector of states of nature that corresponds to thetime instants k − ∆ k , k − ∆ k + 1 , · · · , k . We define the vector of states at time k , Θ k := [Θ k − ∆ k , Θ k − ∆ k +1 , · · · , Θ k ]. Note that the length of the vector dependson the network–delay ∆ k . When ∆ k > Θ k = [Θ Bkr , Θ Bkr +1 , · · · , Θ k ], and when∆ k = 0, Θ k is just [Θ k ].Consider the discrete–time system, which at the beginning of time slot k is de-scribed by the state Γ k = [ Q k , Θ k ] , where we recall that Q k = (cid:20) λ k , B k , ∆ k , W k , R k (cid:21) , Θ k = [Θ k − ∆ k , Θ k − ∆ k +1 , · · · , Θ k ] . Note that Γ = (cid:2)(cid:2) r , , , (cid:3) , Θ (cid:3) . At each time slot k , we have the following setof controls { , } where 0 represents “ take another sample ”, and 1 represents “ stopand declare change ”. Thus, at time slot k , when the control chosen is 1, the stateΓ k +1 is given by a terminal absorbing state t ; when the control chosen is 0, thestate evolution is given by Γ k +1 = [ Q k +1 , Θ k +1 ], where Q k +1 = φ Q ( Q k , M k ) , Θ k +1 = (cid:2) Θ k + { T = k +1 } (cid:3) , if ∆ k +1 = 0 (cid:2) Θ k − ∆ k , Θ k − ∆ k +1 , · · · , Θ k , Θ k + { T = k +1 } (cid:3) , if ∆ k +1 = ∆ k + 1 h Θ k − ∆ k + r , Θ k − ∆ k + r +1 , · · · , Θ k , Θ k + { T = k +1 } i , if ∆ k +1 = ∆ k + 1 − r . =: φ Θ (cid:0) Θ k , Q k , M k , { T = k +1 } (cid:1) (10)where it is easy to observe that Θ k + { T = k +1 } = Θ k +1 . When ∆ k +1 = ∆ k +1, the batch B k is still in service, and hence, in addition to the current stateΘ k +1 = Θ k + { T = k +1 } , we need to keep the states Θ k − ∆ k , Θ k − ∆ k +1 , · · · , Θ k .Also, when ∆ k +1 = ∆ k + 1 − r , then the batch index is incremented, and hence,the vector of states that determines the distribution of the observations sampledat or after B k +1 /r and before k + 1 is given by h Θ k − ∆ k + r , Θ k − ∆ k + r +1 , · · · , Θ k ,Θ k + { T = k +1 } (cid:3) .Define O k := { T = k +1 } , and define N k := [ M k , O k ] be the state–noise duringtime slot k . The distribution of state–noise N k given the state of the discrete– ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · time system Γ k is given by P (cid:0) M k = m, O k = o Γ k = [ q , θ ] (cid:1) and is the productof the distribution functions, P (cid:0) M k = m Γ k = [ q , θ ] (cid:1) and P (cid:0) O k = o Γ k = [ q , θ ] (cid:1) .These distribution functions are provided in Appendix – III.The problem is to detect the change in the state Θ k as early as possible bysequentially observing the samples at the decision maker. We now formulate the
NADM change–detection problem in which the observationsfrom the sensor nodes are sent over a random access network to the fusion centerand the fusion center processes the samples in the
NADM mode.In Section 5.3, we defined the state Γ k = [ Q k , Θ k ] on which we formulate the NADM change detection problem as a POMDP. Recall that at the beginning of slot k , the decision maker receives a vector of sensor measurements Y k and observes thestate Q k of the queueing system. Thus, at time k , Z k = [ Q k , Y k ] is the observationof the decision maker about the state of the dynamical system Γ k .Let A k ∈ { , } be the control (or action) chosen by the decision maker afterhaving observed Z k at k . Recall that 0 represents “ take another sample ” and 1represents the action “ stop and declare change ”. Let I k = (cid:2) Z [0: k ] , A [0: k − (cid:3) be the information vector that is available to the decision maker, at the beginning of timeslot k . Let τ be a stopping time with respect to the sequence of random variables I , I , · · · . Note that A k = 0 for k < τ and A k = 1 for k > τ . We are interested inobtaining a stopping time τ (with respect to the sequence I , I , · · · ) that minimizesthe mean detection delay subject to a constraint on the probability of false alarm.min E (cid:2) ( τ − T ) + (cid:3) (11)such that P ( τ < T ) α Note that in the case of NADM, at any time k , a decision about the change is madebased on the information I k (which includes the batch index we are processing andthe delays). Thus, in the case of NADM , false alarm is defined as the event { τ < T } and, hence, τ > T is not classified as a false alarm even if it is due to pre–changemeasurements only. However, in the case of NODM , this is classified as a false alarmas the decision about the change is based on the batches received until time k .Let c be the cost per unit delay in detection. We are interested in obtaining astopping time τ ∗ that minimizes the expected cost (Bayesian risk) given by C ( c, τ ∗ ) = min τ E (cid:2) { Θ τ =0 } + c · ( τ − T ) + (cid:3) = min τ E " { Θ τ =0 } + c · τ − X k =0 { Θ k =1 } = min τ E " g τ (Γ τ , A τ ) + τ − X k =0 g k (Γ k , A k ) = min τ E " ∞ X k =0 g k (Γ k , A k ) (12) The notation Z [ k : k ] := Z k , Z k +1 , · · · , Z k ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. where, as defined earlier, Γ k = [ Q k , Θ k ]. Let θ = [ θ δ , θ δ − , · · · , θ , θ ]. We definefor k τ g k ([ q , θ ] , a ) = , if θ = 0 , a = 01 , if θ = 0 , a = 1 c, if θ = 1 , a = 00 , if θ = 1 , a = 1 (13)and for k > τ , g k ( · , · ) := 0. Recall that A k = 0 for k < τ and A k = 1 for k > τ .Note that A k , the control at time slot k , depends only on I k . Thus, every stoppingtime τ , corresponds to a policy µ = ( µ , µ , · · · ) such that A k = µ k ( I k ), with A k = 0 for k < τ and A k = 1 for k > τ . Thus, Eqn. 12 can be written as C ( c, τ ∗ ) = min µ E " ∞ X k =0 g k (Γ k , A k ) = min µ ∞ X k =0 E [ g k (Γ k , A k )] (by monotone convergence theorem)= min µ ∞ X k =0 E [ g k (Γ k , µ k ( I k ))] (14)Since Θ k is observed only through I k , we look at a sufficient statistic for I k in thenext subsection. In Section 5.2, we have illustrated the evolution of the queueing system Q k and wehave shown in different scenarios, the vector Y k received by the decision maker.Recall from Section 5.2 that Y k +1 = ∅ , if M k = 0 , ∅ , if M k = j > , R ( j ) k = 1 ,Y k +1 , , if M k = j > , R ( j ) k = 0 , N P i =1 R ( i ) k < N − Y k +1 , , Y k +1 , , · · · , Y k +1 ,n ] , if M k = j > , R ( j ) k = 0 , N P i =1 R ( i ) k = N − , N P i =1 { W ( i ) k > } = n. Note that Y k +1 , corresponds to X ( M k ) B k . The last part of the above equation corre-sponds to the last pending sample of the batch B k arriving at the decision makerat time k + 1, with some samples from batch B k + 1 (= B k +1 ) also being re-leased by the sequencer. In this case, the state of nature at the sampling instantof the batch B k +1 = B k + 1 is Θ k − ∆ k +1 /r . Note that Θ k − ∆ k +1 /r is a componentof the vector Θ k as k − ∆ k + 1 /r = ( B k + 1) /r < k . Thus, the distribution of ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Y k +1 , , Y k +1 , , · · · , Y k +1 ,n is given by f Y k +1 , ( · ) = (cid:26) f ( · ) , if Θ k − ∆ k = 0 f ( · ) , if Θ k − ∆ k = 1 and f Y k +1 ,i ( · ) = (cid:26) f ( · ) , if Θ k − ∆ k +1 /r = 0 f ( · ) , if Θ k − ∆ k +1 /r = 1 , i = 1 , , · · · , n. Thus, at time k + 1, the current observation Y k +1 depends only on the previousstate Γ k , previous action A k , and the previous noise of the system N k . Thus, asufficient statistic is (cid:2) P (cid:0) Γ k = [ q , θ ] I k (cid:1)(cid:3) [ q , θ ] ∈S (see page 244, [Bertsekas 2000a])where S is the set of all states of the dynamical system defined in Sec. 5.3. Let q = [ λ, b, δ, w , r ]. Note that P (cid:0) Γ k = [ q , θ ] I k (cid:1) = P (cid:0) Γ k = [ q , θ ] I k − , Q k , Y k (cid:1) = { Q k = q } · P (cid:0) Θ k = θ I k − , Q k = q , Y k (cid:1) = { Q k = q } · P (cid:0) [Θ k − δ , Θ k − δ +1 , · · · , Θ k − , Θ k ] = [ θ δ , θ δ − , · · · , θ , θ ] I k − , Q k = q , Y k (cid:1) = { Q k = q } · P (cid:0) Θ k − δ = θ δ I k − , Q k = q , Y k (cid:1) · δ Y j =1 P (cid:0) Θ k − δ + j = θ δ − j Θ k − δ + j ′ = θ δ − j ′ , j ′ = 0 , , · · · , j − , I k − , Q k = q , Y k (cid:1) (15)Observe that P (cid:0) Θ k − δ + j = θ δ − j Θ [ k − δ : k − δ + j − , Θ k − δ + j − = 0 , I k − , Q k = q , Y k (cid:1) = (cid:26) − p, if θ δ − j = 0 p, if θ δ − j = 1and P (cid:0) Θ k − δ + j = θ δ − j Θ [ k − δ : k − δ + j − , Θ k − δ + j − = 1 , I k − , Q k = q , Y k (cid:1) = (cid:26) , if θ δ − j = 01 , if θ δ − j = 1 . This is because given Θ k − δ , the events { Θ k − δ + j = θ δ − j } , { I k − , Q k = q , Y k } areconditionally independent. Thus, Eqn. 15 can be written as P (cid:0) Γ k = [ q , θ ] I k (cid:1) = { Q k = q } · P (cid:0) Θ k − δ = 1 I k − , Q k = q , Y k (cid:1) , if θ = { Q k = q } · P (cid:0) Θ k − δ = 0 I k − , Q k = q , Y k (cid:1) · (1 − p ) δ − j − p, if θ = [0 , · · · , , |{z} θ j , · · · , { Q k = q } · P (cid:0) Θ k − δ = 0 I k − , Q k = q , Y k (cid:1) · (1 − p ) δ , if θ = (16) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. Define e Θ k := Θ k − ∆ k , and defineΨ k := P (cid:16) e Θ k = 1 I k − , Q k = [ λ, b, δ, w , r ] , Y k (cid:17) = P (cid:0) Θ k − δ = 1 I k − , Q k = [ λ, b, δ, w , r ] , Y k (cid:1) Π k := P (cid:0) Θ k = 1 I k − , Q k = [ λ, b, δ, w , r ] , Y k (cid:1) = P (cid:0) T k I k − , Q k = [ λ, b, δ, w , r ] , Y k (cid:1) . (17)Thus, Eqn. 16 can be written as P (cid:0) Γ k = [[ λ, b, δ, w , r ] , θ ] I k (cid:1) = { Q k =[ λ,b,δ, w , r ] } · Ψ k , if θ = { Q k =[ λ,b,δ, w , r ] } · (1 − Ψ k ) · (1 − p ) δ − j − p, if θ = [0 , · · · , , |{z} θ j , · · · , { Q k =[ λ,b,δ, w , r ] } · (1 − Ψ k ) · (1 − p ) δ , if θ = (18)We now find a relation between Π k and Ψ k in the following Lemma. Lemma 1
The relation between the conditional probabilities Π k and Ψ k is givenby Π k = Ψ k + (1 − Ψ k ) (cid:0) − (1 − p ) δ (cid:1) (19) Proof.
See Appendix – IV.From Eqn. 18 and Lemma 1, it is clear that a sufficient statistic for I k is ν k =[ Q k , Π k ]. Also, we show in Appendix – V that ν k can be computed recursively,i.e., when A k = 0, ν k +1 = [ Q k +1 , Π k +1 ] = [ Q k +1 , φ Π ( ν k , Z k +1 )], and when A k = 1, ν k +1 = t , a terminal state. Thus, ν k can be thought of as entering into a terminating(absorbing) state t at τ (i.e., ν k = [ Q k , Π k ] for k < τ and ν k = t for k > τ ).Since ν k is sufficient, for every policy µ k there corresponds a policy e µ k such that µ k ( I k ) = e µ k ( ν k ) (see page 244, [Bertsekas 2000a]). τ Let Q be the set of all possible states of the queueing system, Q k . Thus the statespace of the sufficient statistic is N = ( Q × [0 , ∪ { t } . Recall that the actionspace is A = { , } . Define the one–stage cost function e g : N × A → R + as follows.Let ν ∈ N be a state of the system and let a ∈ A be a control. Then, e g ( ν, a ) = ν = t c · π if ν = [ q , π ] , a = 01 − π if ν = [ q , π ] , a = 1 . Note from Eqn. 13 for k τ that E [ g k (Θ k , A k )] = E [ g k (Θ k , µ k ( I k ))]= E (cid:20) E (cid:20) g k (Θ k , µ k ( I k )) I k (cid:21)(cid:21) = E [ e g ( ν k , e µ k ( ν k ))] ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · and for k > τ , E [ g k (Θ k , A k )] = 0= E [ e g ( t , · )]Since, { ν k } is a controlled Markov process, and the one–stage cost function e g ( · , · ),the transition probability kernel for A k = 1 and for A k = 0 (i.e., P (cid:0) Z k +1 ν k (cid:1) ),do not depend on time k , and the optimization problem defined in Eqn. 14 is overinfinite horizon, it is sufficient to look for an optimal policy in the space of stationaryMarkov policies (see page 83, [Bertsekas 2000b]). Thus, the optimization problemdefined in Eqn. 14 can be written as C ( c, τ ∗ ) = min e µ ∞ X k =0 E (cid:2)e g (cid:0) ν k , e µ k ( ν k ) (cid:1)(cid:3) = ∞ X k =0 E (cid:2)e g (cid:0) ν k , e µ ∗ ( ν k ) (cid:1)(cid:3) . (20)Thus, the optimal total cost is given by J ∗ ([ q , π ]) = ∞ X k =0 E (cid:20)e g (cid:0) ν k , e µ ∗ ( ν k ) (cid:1) ν = [ q , π ] (cid:21) . (21)The solution to the above problem is obtained following the Bellman’s equation, J ∗ ([ q , π ]) := min (cid:26) − π, cπ + E (cid:20) J ∗ ( Q k +1 , φ Π ( ν k , Z k +1 )) ν k = [ q , π ] (cid:21)(cid:27) . (22)where the function φ Π ( ν k , Z k +1 ) is provided in Appendix – V. Remark 5.1
The optimal stationary Markov policy (i.e., the optimum stoppingrule τ ) in general depends on Q . Hence, the decision delay and the queueing delayare coupled, unlike in the NODM case.We characterize the optimal policy in the following theorem.
Theorem 3
The optimal stopping rule τ ∗ is a network–state dependent thresholdrule on the a posteriori probability Π k , i.e., there exist thresholds γ ( q ) such that τ = inf { k > k > γ ( Q k ) } (23) Proof.
See Appendix–VI.In general, the thresholds γ ( Q k )s (i.e., optimum policy) can be numerically ob-tained by solving Eqn. 22 using value iteration method (see pp. 88–90, [Bertsekas2000b]). However, computing the optimal policy for the NADM procedure is hardas the state space is huge even for moderate values of N . Hence, we resort to asuboptimal policy based on the following threshold rule, which is motivated by thestructure of the optimal policy. τ = inf { k > k > γ } (24) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. where γ is chosen such that P ( τ < T ) = α is met.Thus, we have formulated a sequential change detection problem when the sensorobservations are sent to the decision maker over a random access network, andthe fusion center processes the samples in the NADM mode. The information fordecision making now needs to include the network state Q k (in addition to thesamples received by the decision maker); we have shown that [ Q k , Π k ] is sufficientfor the information history I k . Also, we have provided the structure for the optimalpolicy. Since, obtaining the optimal policy is computationally hard, we gave asimple threshold based policy, which is motivated by the structure of the optimalpolicy.
6. NUMERICAL RESULTS
Minimizing the mean detection delay not only requires an optimal decision rule atthe fusion center but also involves choosing the optimal values of the sampling rate r , and the number of sensors N . To explore this, we obtain the minimum decisiondelay for each value of the sampling rate r numerically, and the network delay viasimulation. Consider a sensor network with N nodes. For a given probability of false alarm,the decision delay (detection delay without the network–delay component) decreaseswith increase in sampling rate. This is due to the increase in the number of samplesthat the fusion center receives within a given time. But, as the sampling rateincreases, the network delay increases due to the increased packet communicationload in the network. Therefore it is natural to expect the existence of a samplingrate r ∗ , with r ∗ < σ/N , (the sampling rate should be less than σ/N , for thequeues to be stable; see Theorem 2) that optimizes the tradeoff between these twocomponents of detection delay. Such an r ∗ , in the case of NODM can be obtainedby minimizing the following expression over r (recall Theorem 1).( d ( r ) + l ( r )) (1 − α ) − ρ · l ( r ) + 1 r min Π α E h e K − K i + Note that in the above expression, the delay term min Π α E h e K − K i + also dependson the sampling rate r via the probability of change p r = 1 − (1 − p ) (1 /r ) . The delaydue to coarse sampling l ( r )(1 − α ) − ρ · l ( r ) can be found analytically (see Appendix –I). We can approximate the delay min Π α E h e K − K i + by the asymptotic (as α → | ln( α ) | NI ( f ,f )+ | ln(1 − p r ) | where I ( f , f ) is the Kullback–Leibler (KL) divergencebetween the pdfs f and f (see [Tartakovsky and Veeravalli 2005]). But, obtainingthe network–delay (i.e., d ( r )(1 − α )) analytically is hard, and hence an analyticalcharacterisation of r ∗ appears intractable. Hence, we have resorted to numericalevaluation.The distribution of sensor observations are taken to be N (0 ,
1) and N (1 , ,before and after the change respectively for all the 10 nodes. We choose the prob- As usual, N ( a, v ) denotes a normal distribution with mean a and variance v ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. ·
60 80 100 120 140 160 180 200 220 0.01 0.015 0.02 0.025 0.03 0.035 M ea n D e t ec t i on D e l ay ( s l o t s ) Sampling rate, r (samples per slot)NODM: SimulationNODM: ApproximationNADM: Simulation
Fig. 15. Mean detection delay for N = 10 nodes is plotted against the sampling rate r for both NODM and
NADM (defined in Eqn. 24). For
NODM , an approximate analysis is also plotted.This was obtained with the prior probability ρ = 0 , p = 0 . α = 0 . , σ = 0 . N (0 ,
1) and N (1 , ability of occurrence of change in a slot to be p = 0 . Π α E h e K − K i + and d ( r ) are obtained from simu-lation for α = 0 .
01 and σ = 0 . r in Figure 15. Note that both NODM andNADM are threshold based, and we obtain the corresponding thresholds for a tar-get P FA = 0 .
01 by simulation. These thresholds are then used to obtain the meandetection delay by simulation. In Figure 15, we also plot the approximate meandetection delay which is obtained through the expression for l ( r ) and the approx-imation, min Π α E h e K − K i + ≈ | ln( α ) | NI ( f ,f )+ | ln(1 − p r ) | . We study this approximationas this provides an (approximate) explicit expression for the mean decision delay.The delay in the FJQ–GPS does not have a closed form expression. Hence, we stillneed simulation for the delay due to queueing network. It is to be noted that at k = 0, the size of all the queues is set to 0. The mean detection delay due to theprocedure defined in Eqn. 24 is also plotted in Figure 15.As would have been expected, we see from Figure 15 that the NADM procedurehas a better mean detection delay performance than the
NODM procedure. Notethat σ/N = 0 . σ/N = 0 . / < . < / r increases to 1/28 (the maximum allowed sampling rate), thequeueing delay increases rapidly. This is evident from Figure 15. Also, we see fromFigure 15 that operating at a sampling rate around 1 / ≈ . NODM and
NADM . At the optimal sampling rate the mean detection delayof
NODM is 90 slots and that of
NADM is 73 slots.
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY.
48 50 52 54 56 58 60 62 0 5 10 15 20 25 30 35 40 M ea n D ec i s i on D e l ay ( s l o t s ) Number of Sensors, N 0 10 20 30 40 50 60 0 5 10 15 20 25 30 35 40 D e l ay ( s l o t s ) Number of Sensors, NCoarse sampling delay Decision maker delay
Fig. 16. Mean decision delay of
NODM procedure for N × r = 1 / N . The plot is obtained with ρ = 0 , p = 0 . , α = 0 .
01 and with the sensorobservations being N (0 ,
1) and N (1 , − α ) l ( r ) − ρl ( r ), and the decisionmaker delay, r min Π α E h e K − K i + are shown on the right. Now let us consider fixing N × r . This is the number of observations the fusioncenter receives per slot in a network with N nodes sampling at a rate r (samples perslot). It is also a measure of the energy spent by the network per slot. Since it hasbeen assumed that the observations are conditionally independent and identicallydistributed across the sensors and over time, it is natural to ask how beneficial itis to have more nodes sampling at a lower rate, when compared to fewer nodessampling at a higher rate with the number of observations per slot being the same.With p = 0 . α = 0 .
01, and σ = 0 . f ∼ N (0 ,
1) and f ∼ N (1 , N r = 1 / N r = 1 /
100 (the case oflightly loaded network,
N r ≪ σ ).Figure 16 shows the plot of mean decision delay, l ( r )(1 − α − ρ )+ r min Π α E h e K − K i + versus the number of sensors when N r = 1 /
3. As N increases, the sampling rate r = 1 / (3 N ) decreases and hence the coarse sampling delay l ( r )(1 − α ) increases; thiscan be seem to be approximately linear by analysis of the expression for l ( r ) given inAppendix – I. Also, as N increases, the decision maker gets more samples at the de-cision instants and hence the delay due to the decision maker r min Π α E h e K − K i + decreases (this is evident from the right side of Figure 16). Figure 16 shows thatin the region where N is large (i.e., N >
20) or N is very small (i.e., N < N increases, the mean decision delay increases. This is because in this region as N increases, the decrease in the delay due to decision maker is smaller compared tothe increase in the delay due to coarse sampling. However, in the region where N is moderate (i.e., for 5 N < N increases, the decrease in the delay due todecision maker is large compared to the increase in the delay due to coarse sam-pling. Hence in this region, the mean decision delay decreases with N . Therefore,we infer that when N × r = , deploying 20 nodes sampling at 1 /
60 samples perslot is optimal, when there is no network delay.
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. ·
60 80 100 120 140 160 180 200 220 0 5 10 15 20 25 30 35 40 M ea n D e t ec t i on D e l ay ( s l o t s ) Number of Sensors, NNODM: SimulationNADM: Simulation
Fig. 17. Mean detection delay for N × r = 1 / N . Thiswas obtained with ρ = 0 , p = 0 . , α = 0 . σ = 0 . N (0 ,
1) and N (1 , Figure 17 shows the mean detection delay (i.e., the network delay plus the decisiondelay shown in Figure 16) versus the number of nodes N for a fixed N × r = 1 / N increases, the sampling rate r = 1 / (3 N ) decreases.For large N (and equivalently small r ), in the case of NODM with the Shiryaevprocedure, the network delay, d ( r ) ≈ Nσ as it requires N (independent) successes,each with probability σ , in the random access network to transport a batch of N samples (also, since the sampling rate r is small, one would expect that a batchis delivered before a new batch is generated) and the decision maker requires justone batch of N samples to stop (after the change occurs). Hence, for large N , thedetection delay is ≈ l ( r )(1 − α ) + d ( r )(1 − α ) ≈ l ( r )(1 − α ) + Nσ (1 − α ). It is to benoted that for large N , to achieve a false alarm probability of α , the decision makerrequires N α < N samples (the mean of the log–likelihood ratio, LLR of receivedsamples, after change, is the KL divergence between pdfs f and f , given by I ( f , f ) >
0. Hence, the posterior probability, which is a function of LLR, increaseswith the the number of received samples. Thus, to cross a threshold of γ ( α ), weneed N α samples). Thus, for large N , in the NADM procedure, the detection delayis approximately l ( r )(1 − α ) + N α σ (1 − α ), where N α /σ is the mean network–delayto transport N α samples. Thus, for large N , the difference in the mean detectiondelay between NODM and
NADM procedures is approximately − ασ ( N − N α ). Notethat N α depends only on α and hence the quantity − ασ ( N − N α ) increases with N .This behaviour is in agreement with Figure 17. Also, as N × r = 1 /
3, we expectthe network delay to be very large (as 1/3 is close to σ = 0 . N × r = 0 .
01 (seeFigure 18). Note that having N = 10 sensors is optimal for the NADM procedure.The
NODM procedure makes the decision only when it receives a batch of N samplescorresponding to a sampling instant, whereas NADM procedure makes the decision
ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY.
800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 0 5 10 15 20 25 30 35 40 M ea n D e t ec t i on D e l ay ( s l o t s ) Number of Sensors, NNODM: SimulationNADM: Simulation
Fig. 18. Mean detection delay for N × r = 0 .
01 is plotted against the the number of nodes N . Thiswas obtained with ρ = 0 , p = 0 . , α = 0 .
01 and with the sensor observations being N (0 , N (1 , at every time slot irrespective of whether it receives a sample in that time slot ornot. Thus, the Bayesian update that NADM does at every time slot makes it stopearlier than
NODM .
7. CONCLUSIONS
In this work we have considered the problem of minimizing the mean detectiondelay in an event detection on a small extent ad hoc wireless sensor network. Weprovide two ways of processing samples in the fusion center: i)
Network Oblivious ( NODM ) processing, and ii)
Network Aware ( NADM ) processing. We show thatin the
NODM processing, under periodic sampling, the detection delay decouplesinto decision and network delays. An important implication of this is that anoptimal sequential change detection algorithm can be used in the decision deviceindependently of the random access network. We also formulate and solve thechange detection problem in the
NADM setting in which case the optimal decisionmaker needs to use the network state in its optimal stopping rule. Also, we studythe network delay involved in this problem and show that it is important to operateat a particular sampling rate to achieve the minimum detection delay.
Appendix – I
Proof: (Theorem 1)min Π α E h(cid:16) e U − T (cid:17) I { e T > T } i = min Π α E (cid:20) ( e U − e T + e T − Kr + Kr − T ) I { e T > T } (cid:21) = min Π α (cid:26) E h ( e U − e T ) I { e T > T } i + E (cid:20)(cid:18) Kr − T (cid:19) I { e T > T } (cid:21) + 1 r E h(cid:16) e K − K (cid:17) I { e T > T } i(cid:27) (25) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Note that in Eqn. 25, the first term is the queueing delay, the second term is thecoarse sampling delay and the third term is the decision delay (all delays being inslots). Consider the first term, E h ( e U − e T ) I { e T > T } i = E h ( U e K − t e K ) I { e T > T } i = X j ≥ ,b ≥ ,x ≥ P (cid:16) T = j, e K = b, D b = x (cid:17) x · I { br > j } = X j ≥ ,b ≥ ,x ≥ P (cid:16) T = j, e K = b (cid:17) P ( D b = x ) x · I { br > j } where we have used the facts that (i) the decision process is based on only whatthe packets carry and not on their arrival time etc, and (ii) the condition thatsampling is done periodically at a known rate r . Assuming the queueing system tobe stationary, the above can be written as E h ( e U − e T ) I { e T > T } i = X x ≥ P ( D = x ) x X j,b P (cid:16) T = j, e K = b (cid:17) I { br > j } = E [ D ] P (cid:16) e T > T (cid:17) . Note that E [ D ] is a function of the sampling rate r , and does not depend on thedetection policy.Consider the second term of Eqn. 25, E (cid:20)(cid:18) Kr − T (cid:19) I { e T ≥ T } (cid:21) = E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K } (cid:21) = E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K,S =1 } (cid:21) + E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K,S =0 } (cid:21) For S = 1, we have T = 0 and K = 0. Hence, E (cid:20)(cid:18) Kr − T (cid:19) I { e T ≥ T } (cid:21) = 0 + E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K } (cid:21) where E [ · ] denote the expectation and P ( · ) the probability law, when the initialstate is S = 0. Now, E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K } (cid:21) = ∞ X b =1 ∞ X e b = b b/r X t =( b − /r +1 P (cid:16) T = t, K = b, e K = e b (cid:17) · (cid:18) br − t (cid:19) = ∞ X b =1 ∞ X e b = b P (cid:16) K = b, e K = e b (cid:17) · b/r X t =( b − /r +1 P (cid:16) T = t | K = b, e K = e b (cid:17) · (cid:18) br − t (cid:19) (26) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. We note that e K is independent of T given K . Hence, E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K } (cid:21) = ∞ X b =1 ∞ X e b = b P (cid:16) K = b, e K = e b (cid:17) · h /r − X y =0 y · P (cid:18) T = br − y | K = b (cid:19) i We have P ( T = t | K = b ) = ( (1 − ρ )(1 − p ) t − p (1 − ρ )(1 − p r ) b − p r , for t s.t. b = ⌈ t · r ⌉ , otherwise.Hence, for 0 ≤ y ≤ /r − P (cid:18) T = br − y | K = b (cid:19) = (1 − p ) b/r − y − p (1 − p r ) b − p r But, (1 − p r ) = (1 − p ) /r . Hence, P (cid:18) T = br − y | K = b (cid:19) = (1 − p ) b/r − y − p (1 − p r ) b − p r = (1 − p ) /r − y − p − (1 − p ) /r It can be shown that /r − X y =0 y · (1 − p ) /r − y − p − (1 − p ) /r = 1 r − (cid:18) p − rp r (1 − p r ) (cid:19) =: l ( r )Therefore, Eqn. 26 can be written as E (cid:20)(cid:18) Kr − T (cid:19) I { e K ≥ K } (cid:21) = l ( r ) · P (cid:16) e K ≥ K (cid:17) = l ( r ) · (cid:16) P (cid:16) e K ≥ K (cid:17) − ρ (cid:17) = l ( r ) · (cid:16) − P (cid:16) e K < K (cid:17) − ρ (cid:17) Finally, we havemin Π α E h ( e U − T ) I { e T ≥ T } i = min Π α (cid:26) d ( r ) (cid:16) − P (cid:16) e T < T (cid:17)(cid:17) + l ( r ) P (cid:16) e T ≥ T (cid:17) + 1 r E h ( e K − K ) + i(cid:27) = min Π α (cid:26) ( d ( r ) + l ( r )) (cid:16) − P (cid:16) e T < T (cid:17)(cid:17) − ρ · l ( r ) + 1 r E h ( e K − K ) + i(cid:27) Note that, in the above equation, the first term ( d ( r ) + l ( r )) (cid:16) − P (cid:16) e T < T (cid:17)(cid:17) is ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · minimum when P (cid:16) e T < T (cid:17) = α . It follows thatmin Π α E h ( e U − T ) I { e T ≥ T } i > ( d ( r ) + l ( r )) (1 − α ) − ρ · l ( r ) + 1 r min Π α E h ( e K − K ) + i Also, since the optimal policy for the problem min Π α E h ( e K − K ) + i achieves (cid:16) − P (cid:16) e T < T (cid:17)(cid:17) = α , we also have( d ( r ) + l ( r )) (1 − α ) − ρ · l ( r ) + 1 r min Π α E h ( e K − K ) + i > min Π α E h ( e U − T ) I { e T ≥ T } i It follows thatmin Π α E h ( e U − T ) I { e T ≥ T } i = ( d ( r ) + l ( r )) (1 − α ) − ρ · l ( r ) + 1 r min Π α E h ( e K − K ) + i We need 1 − α > ρ or α < − ρ . If α > − ρ , the optimal stopping is at t = 0.This will yield the desired probability of false alarm and E h ( e U − T ) I { e T ≥ T } i = 0. Appendix – II
Proof: (Theorem 2) The necessity of
N r < σ is clear. The sufficiency proof goes asfollows. Consider the FJQ-GPS system with every queue always containing a singledummy packet that is served at low priority. Let us call this the saturated FJQ-GPSsystem. When a queue becomes empty, the low priority dummy packet contendsfor service. If it receives service, then it immediately reappears and continuesto contend for service. If, while a dummy packet is in service, a regular packetarrives, then the service of the dummy packet is preempted and the regular packetstarts contending. It follows that the service rate applied to every queue (i.e., thosewith regular packets or those with dummy packets) is always σ/N . Now, consider avirtual service process of rate σ . In each slot, a service occurs with probability σ andthe service is applied to any one of the queues with equal probability. Equivalentlyeach queue is served by an independent Bernoulli process of rate σ/N . Consideringonly the services to the regular packets at each queue, we have a GI/M/ GI refers to a General distribution with Independent arrivals , M refers toa Markovian service process and ). Hence, the system hasproper stationary delay, iff r < σ/N . Also, it can be seen that the delays in theabove described system (with dummy packets when a queue is empty) upper boundthose in the original FJQ-GPS system. Hence, the result follows.
Appendix – III
Distribution of state noise N Let q = [ λ, b, δ, w , r ]. Note that P (cid:0) M k = m Q k = q , Θ k = θ (cid:1) = P (cid:0) M k = m Q k = q (cid:1) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. and is given by P (cid:0) M k = 0 Q k = q (cid:1) = (cid:26) φ N ( q ) = 01 − σ if φ N ( q ) > P (cid:0) M k = m Q k = q (cid:1) = (cid:26) φ N ( q ) = 0 σφ N ( q ) if φ L ( m ) ( q ) > , m = 1 , , , · · · , N. where φ N ( q ) and φ L ( m ) ( q ) are obtained from Eqns. 9 and 8.The distribution function, P (cid:0) O k = o Q k = q , Θ k = θ (cid:1) = P (cid:0) O k = o Q k = q , Θ k = θ (cid:1) is given by P (cid:0) O k = o Q k = q , Θ k = 0 (cid:1) = − p if o = 0 p if o = 1 , . P (cid:0) O k = o Q k = q , Θ k = 1 (cid:1) = (cid:26) o = 00 otherwise . Appendix – IV
Proof of Lemma–1
Let q = [ λ, b, δ, w , r ]. From Eqn. 17, Π k := P (cid:0) T k I k − , Q k = q , Y k (cid:1) = P (cid:0) T k − δ I k − , Q k = q , Y k (cid:1) + P (cid:0) k − δ < T k I k − , Q k = q , Y k (cid:1) = P (cid:0) T k − δ I k − , Q k = q , Y k (cid:1) + P (cid:0) T > k − δ I k − , Q k = q , Y k (cid:1) · P (cid:0) T k T > k − δ, I k − , Q k = q , Y k (cid:1) , = Ψ k + (1 − Ψ k ) · P (cid:0) T k T > k − δ, I k − , Q k = q , Y k (cid:1) , = Ψ k + (1 − Ψ k ) · P ( k − δ < T k ) P (cid:0) I k − , Q k = q , Y k k − δ < T k (cid:1) P ( T > k − δ ) P (cid:0) I k − , Q k = q , Y k T > k − δ (cid:1) = Ψ k + (1 − Ψ k ) · P ( k − δ < T k ) P ( T > k − δ ) (27)= Ψ k + (1 − Ψ k ) (cid:16) − (1 − p ) δ (cid:17) (28) ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Eqn. 27 is justified as follows. Note that P (cid:0) I k − , Q k = q , Y k k − δ < T k (cid:1) = P (cid:16) Q [0: k − , Q k = q , X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } , u [0: k − k − δ < T k (cid:17) = P (cid:0) Q [0: k − , Q k = q k − δ < T k (cid:1) · P (cid:16) X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } k − δ < T k, Q [0: k − , Q k = q (cid:17) · P (cid:16) u [0: k − k − δ < T k, Q [0: k − , Q k = q , X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } (cid:17) = P (cid:0) Q [0: k − , Q k = q (cid:1) · P (cid:16) X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } k − δ < T, Q [0: k − , Q k = q (cid:17) · P (cid:16) u [0: k − Q [0: k − , Q k = q , X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } (cid:17) = P (cid:0) Q [0: k − , Q k = q T > k − δ (cid:1) · P (cid:16) X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } T > k − δ, Q [0: k − , Q k = q (cid:17) · P (cid:16) u [0: k − T > k − δ, Q [0: k − , Q k = q , X [1: B k − , { X ( i ) B k : R ( i ) k = 1 } (cid:17) = P (cid:0) I k − , Q k = q , Y k T > k − δ (cid:1) . We use the following facts in the above justification: i) the evolution of the queueingsystem Q k is independent of the change point T , ii) whenever T > k − δ , thedistribution of any sample X ( i ) h , h B k is f , and iii) the control u k = ˜ µ ( I k ). Appendix – V
Recursive computation of Π k At time k , based on the index of the node that successfully transmits a packet M k , the set of all sample paths Ω can be partitioned based on the following events, E ,k := n ω : M k ( ω ) = 0 or M k ( ω ) = j > , R ( j ) k ( ω ) = 1 o E ,k := ( ω : M k ( ω ) = j > , R ( j ) k ( ω ) = 0 , N X i =1 R ( i ) k ( ω ) < N − ) E ,k := ( ω : M k ( ω ) = j > , R ( j ) k ( ω ) = 0 , N X i =1 R ( i ) k ( ω ) = N − ) , i.e., Ω = E ,k ∪ E ,k ∪ E ,k . We note that the above events can also be described byusing Q k and Q k +1 in the following manner E ,k = { ω : W k +1 ( ω ) = W k ( ω ) , R k +1 ( ω ) = R k ( ω ) } [ { ω : W k +1 ( ω ) = W k ( ω ) + e j , R k +1 ( ω ) = R k ( ω ) }E ,k = { ω : W k +1 ( ω ) = W k ( ω ) , R k +1 ( ω ) = R k ( ω ) + e j }E ,k = ( ω : N X i =1 R ( i ) k ( ω ) = N − , ∀ i, W ( i ) k +1 ( ω ) = ( W ( i ) k ( ω ) − + , R ( i ) k +1 ( ω ) = { W ( i ) k > } ) . Here, the events E ,k and E ,k represent the case B k +1 = B k , and the event E ,k represents the case B k +1 = B k + 1 (i.e., only if the event E ,k occurs then the ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. batch index is incremented). We are interested in obtaining Π k +1 from [ Q k , Π k ]and Z k +1 . We show that at time k + 1, the statistic Ψ k +1 (after having observed Z k +1 ) can be computed in a recursive manner using Ψ k and Q k . Using Lemma 1(using Eqn. 19) we compute Π k +1 from Ψ k +1 .Ψ k +1 = P (cid:16) e Θ k +1 = 1 | I k +1 (cid:17) = X c =1 P (cid:16) e Θ k +1 = 1 , E c,k | I k +1 (cid:17) = X c =1 P (cid:16) e Θ k +1 = 1 | E c,k , I k +1 (cid:17) E c,k ( ∵ E c,k is I k +1 measurable) • Case M k = 0 or M k = j > , R ( j ) k = 1:Π k +1 = P (Θ k +1 = 1 | E ,k , I k +1 )= P (Θ k +1 = 1 | E ,k , I k , Q k +1 = q ′ )= P (Θ k +1 = 1 | E ,k , I k ) · f Q k +1 | Θ k +1 , E ,k , I k ( q ′ | , E ,k , I k ) f Q k +1 E ,k , I k ( q ′ |E ,k , I k ) (by Bayes rule)= P (Θ k +1 = 1 | E ,k , I k ) ( Q k +1 is independent of Θ k +1 )= P (Θ k = 0 , Θ k +1 = 1 | I k ) + P (Θ k = 1 , Θ k +1 = 1 | I k )= (1 − Π k ) p + Π k • Case M k = j > , R ( j ) k = 0 , P Ni =1 R ( i ) k < N −
1: In this case, the sample X ( j ) B k is successfully transmitted and is passed on to the decision maker. The decisionmaker receives just this sample, and computes Π k +1 . We compute Ψ k +1 fromΨ k and then we use Lemma 1 (using Eqn. 19) to compute Π k +1 from Ψ k +1 .Ψ k +1 = P (cid:16) e Θ k +1 = 1 | E ,k , I k +1 (cid:17) = P (cid:16) e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) = P (cid:16) e Θ k = 0 , e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) + P (cid:16) e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) Since, we consider the case when the fusion center received a sample at time k + 1 and B k +1 = B k , ∆ k +1 = ∆ k + 1 and hence, the state e Θ k +1 = Θ k +1 − ∆ k +1 =Θ k − ∆ k = e Θ k . Thus, in this case, Ψ k +1 can be written as ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Ψ k +1 = P (cid:16)e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) ( a ) = P (cid:16)e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k (cid:17) · P (cid:16) Q k +1 = q ′ | e Θ k = 1 , e Θ k +1 = 1 , E ,k , I k (cid:17) P ( Q k +1 = q ′ |E ,k , I k ) · f Y k +1 |E ,k , I k , Q k +1 ( y |E ,k , I k , q ′ ) · f Y k +1 | e Θ k , e Θ k +1 , E ,k , I k , Q k +1 ( y | , , E ,k , q ′ , I k ) ( b ) = P (cid:16)e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k (cid:17) · P ( Q k +1 = q ′ |E ,k , I k ) · f Y k +1 | e Θ k ( y | P ( Q k +1 = q ′ |E ,k , I k ) · f Y k +1 |E ,k , I k , Q k +1 ( y |E ,k , I k , q ′ ) ( c ) = P (cid:16)e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k (cid:17) · f ( y ) P (cid:16)e Θ k = 0 | E ,k , I k , Q k +1 (cid:17) · f Y k +1 | e Θ k ( y |
0) + P (cid:16)e Θ k = 1 | E ,k , I k , Q k +1 (cid:17) · f Y k +1 | e Θ k ( y | ( d ) = Ψ k f ( y )(1 − Ψ k ) f ( y ) + Ψ k f ( y ) We explain the steps ( a ) , ( b ) , ( c ) , ( d ) below.(a) By Bayes rule, for events A, B, C, D, E, F , we have P ( AB | CDEF ) = P ( AB | CD ) P ( E | ABCD ) P ( F | ABCDE ) P ( E | CD ) P ( F | CDE )(b) Q k +1 is independent of e Θ k , e Θ k +1 . Also, given e Θ k , Y k +1 is independent of e Θ k +1 , E ,k , I k , Q k +1 (c) For any events A, B , and a continuous random variable Y , the conditionaldensity function f Y | A ( y | A ) = P ( B | A ) f Y | AB ( y | AB )+ P ( B c | A ) f Y | AB c ( y | AB c ).Also, given e Θ k , Y k +1 is independent of E ,k , I k , Q k +1 (d) E ,k is [ I k , Q k +1 ] measurable, and hence, given [ I k , Q k +1 ], e Θ k is independentof E ,k . • Case M k = j > , R ( j ) k = 0 , P Ni =1 R ( i ) k = N −
1: In this case, at time k + 1, thedecision maker receives the last sample of batch B k , X ( j ) B k (that is successfullytransmitted during slot k ) and the samples of batch B k + 1, if any, that arequeued in the sequencer buffer. We compute Ψ k +1 from Ψ k and then we useLemma 1 (using Eqn. 19) to compute Π k +1 from Ψ k +1 . In this case, the decisionmaker receives n := P Ni =1 { W ( i ) k > } samples of batch B k + 1. Also, note that n is I k measurable.Ψ k +1 = P (cid:16) e Θ k +1 = 1 | E ,k , I k +1 (cid:17) = P (cid:16) e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) = P (cid:16) e Θ k = 0 , e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) + P (cid:16) e Θ k = 1 , e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) Since, we consider the case B k +1 = B k + 1, ∆ k +1 = ∆ k + 1 − /r and hence, thestate e Θ k +1 = Θ k +1 − ∆ k +1 = Θ k − ∆ k +1 /r . ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. Let y = [ y , y , · · · , y n ]. Consider P (cid:16)e Θ k = e θ, e Θ k +1 = 1 | E ,k , I k , [ Q k +1 , Y k +1 ] = [ q ′ , y ] (cid:17) ( a ) = P (cid:16)e Θ k = e θ, e Θ k +1 = 1 | E ,k , I k (cid:17) · P (cid:16) Q k +1 = q ′ | e Θ k = e θ, e Θ k +1 = 1 , E ,k , I k (cid:17) P ( Q k +1 = q ′ |E ,k , I k ) · f Y k +1 |E ,k , I k , Q k +1 ( y |E ,k , I k , q ′ ) · f Y k +1 | e Θ k , e Θ k +1 , E ,k , I k , Q k +1 ( y | e θ, , E ,k , q ′ , I k ) ( b ) = P (cid:16)e Θ k = e θ, e Θ k +1 = 1 | E ,k , I k (cid:17) · P ( Q k +1 = q ′ |E ,k , I k ) · f e θ ( y ) Q ni =1 f ( y i ) P ( Q k +1 = q ′ |E ,k , I k ) · f Y k +1 |E ,k , I k , Q k +1 ( y |E ,k , I k , q ′ ) ( c ) = P (cid:16)e Θ k = e θ | E ,k , I k (cid:17) · P (cid:16)e Θ k +1 = 1 | e Θ k = e θ, E ,k , I k (cid:17) · f e θ ( y ) Q ni =1 f ( y i ) P e θ ′ =0 P e θ ′′ =0 P (cid:16)e Θ k = e θ ′ , e Θ k +1 = e θ ′′ , | E ,k , I k , Q k +1 (cid:17) · f Y k +1 | e Θ k , e Θ k +1 E ,k , I k , Q k +1 ( y | e θ ′ , e θ ′′ , E ,k , I k , q ′ ) . We explain the steps ( a ) , ( b ) , ( c ) below.(a) By Bayes rule, for events A, B, C, D, E, F , we have P ( AB | CDEF ) = P ( AB | CD ) P ( E | ABCD ) P ( F | ABCDE ) P ( E | CD ) P ( F | CDE )(b) Q k +1 is independent of e Θ k , e Θ k +1 . Also, given e Θ k , Y k +1 , is independent of e Θ k +1 , E ,k , I k , Q k +1 , and given e Θ k +1 , Y k +1 ,i is independent of e Θ k , E ,k , I k , Q k +1 .It is to be noted that given the state of nature, the sensor measurements Y k +1 , , Y k +1 , , · · · , Y k +1 ,n are conditionally independent.(c) For any events A, B , and a continuous random variable Y , the conditionaldensity function f Y | A ( y | A ) = P ( B | A ) f Y | AB ( y | AB )+ P ( B c | A ) f Y | AB c ( y | AB c ).Also, given e Θ k , Y k +1 is independent of E ,k , I k , Q k +1 It is to be noted that the event E ,k is [ I k , Q k +1 ] measurable, and hence, given[ I k , Q k +1 ], e Θ k is independent of E ,k . Thus, in this case, Ψ k +1 = (1 − Ψ k ) p r f ( y ) Q ni =1 f ( y i ) + Ψ k f ( y ) Q ni =1 f ( y i )(1 − Ψ k )(1 − p r ) f ( y ) Q ni =1 f ( y i ) + (1 − Ψ k ) p r f ( y ) Q ni =1 f ( y i ) + Ψ k f ( y ) Q ni =1 f ( y i ) . Thus, using Lemma 1 (using Eqn. 19), we haveΠ k +1 = Ψ k +1 + (1 − Ψ k +1 )(1 − (1 − p ) ∆ k +1 )=: φ Ψ (Ψ k , Z k +1 ) + (1 − φ Ψ (Ψ k , Z k +1 )) (1 − (1 − p ) ∆ k +1 )= φ Ψ (cid:18) Π k − (1 − (1 − p ) ∆ k )(1 − p ) ∆ k , Z k +1 (cid:19) + (cid:18) − φ Ψ (cid:18) Π k − (1 − (1 − p ) ∆ k )(1 − p ) ∆ k , Z k +1 (cid:19)(cid:19) (1 − (1 − p ) ∆ k +1 )=: φ Π ([ Q k , Π k ] , Z k +1 ) . Appendix – VI
Structure of τ ∗ We use the following Lemma to show that J ∗ ( q , π ) is concave in π . ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. · Lemma 2 If f : [0 , → R is concave, then the function h : [0 , → R defined by h ( y ) = E φ ( x ) (cid:20) f (cid:18) y · φ ( x ) + (1 − y ) p r · φ ( x ) y · φ ( x ) + (1 − y ) p r · φ ( x ) + (1 − y )(1 − p r ) · φ ( x ) (cid:19)(cid:21) is concave for each x , where φ ( x ) = y · φ ( x )+(1 − y ) p r · φ ( x )+(1 − y )(1 − p r ) · φ ( x ) , < p r < , and φ ( x ) , φ ( x ) , and φ ( x ) are pdfs on X . Proof.
See Appendix – I of [Premkumar and Kumar 2008].Note that in the finite H –horizon (truncated version of Eqn. 21), we note from valueiteration that the cost–to–go function, for a given q , J HH ([ q , π ]) = 1 − π is concavein π . Hence, by Lemma 2, we see that for any given q , the cost–to–go functions J HH − ([ q , π ]) , J HH − ([ q , π ]) , · · · , J H ([ q , π ]) are concave in π . Hence for 0 ≤ λ ≤ J ∗ ([ q , π ]) = lim H →∞ J H ([ q , π ]) J ∗ ([ q , λπ + (1 − λ ) π ]) = lim H →∞ J H (cid:16) [ q , λπ + (1 − λ ) π ] (cid:17) ≥ lim H →∞ λJ H ([ q , π ]) + lim H →∞ (1 − λ ) J H ([ q , π ])= λJ ∗ ([ q , π ]) + (1 − λ ) J ∗ ([ q , π ])It follows that for any given q , J ∗ ([ q , π ]) is concave in π .Define the map ξ : Q× [0 , → R + as ξ ([ q , π ]) := 1 − π and the map κ : Q× [0 , → R + , as κ ([ q , π ]) := c · π + A J ∗ ([ q , π ]) = c · π + E (cid:20) J ∗ ([ Q k +1 , φ Π ( ν k , Z k +1 )]) ν k = [ q , π ] (cid:21) .Note that ξ ([ q , κ ([ q , c , ξ ([ q , κ ([ q , E (cid:20) J ∗ ([ Q k +1 , φ Π ( ν k , Z k +1 )]) ν k = [ q , (cid:21) (2) = E (cid:20) J ∗ ([ φ Q ( Q k , M k ) , φ Π ( ν k , Z k +1 )]) ν k = [ q , (cid:21) = N X m =0 E (cid:20) J ∗ ([ φ Q ( q , m ) , φ Π ( ν k , Z k +1 )]) M k = m, ν k = [ q , (cid:21) P (cid:18) M k = m ν k = [ q , (cid:19) (4) N X m =0 J ∗ (cid:18)(cid:20) φ Q ( q , m ) , E (cid:20) φ Π ( ν k , Z k +1 ) M k = m, ν k = [ q , (cid:21)(cid:21)(cid:19) P (cid:18) M k = m ν k = [ q , (cid:19) = N X m =0 J ∗ ([ φ Q ( q , m ) , p ) P (cid:18) M k = m ν k = [ q , (cid:19) (6) N X m =0 (1 − p ) · P (cid:18) M k = m ν k = [ q , (cid:19) = 1 − p < Q k in step 2, the Jensen’sinequality (as for any given q , J ∗ ( q , π ) is concave in π ) in step 4, and the inequality J ∗ ( q , π ) − π in step 6. ACM Transactions on Sensor Networks, Vol. V, No. N, Month 20YY. Note that κ ([ q , − ξ ([ q , > κ ([ q , − ξ ([ q , <
0. Also, for a fixed q ,the function κ ([ q , π ]) − ξ ([ q , π ]) is concave in π . Hence, by the intermediate valuetheorem , for a fixed q , there exists γ ( q ) ∈ [0 ,
1] such that κ ([ q , γ ]) = ξ ([ q , γ ]). This γ is unique as κ ([ q , π ]) = ξ ([ q , π ]) for at most two values of π . If in the interval[0 , π for which κ ([ q , π ]) = ξ ([ q , π ]), then thesigns of κ ([ q , − ξ ([ q , κ ([ q , − ξ ([ q , τ ∗ = inf { k : Π k > γ ( Q k ) } where the threshold γ ( q ) is given by c · γ ( q ) + A J ∗ ([ q , γ ( q )]) = 1 − γ ( q ). REFERENCES
Aldosari, S. A. and Moura, J. M. F.
Proceedings of ICASSP . II:277–280.
Baccelli, F. and Makowski, A.
StochasticAnalysis of Computer and Communication Systems , H. Takagi, Ed. North–Holland, 57–131.
Bertsekas, D. P.
Dynamic Programming and Optimal Control , Second ed. Vol. I. AthenaScientific.
Bertsekas, D. P.
Dynamic Programming and Optimal Control , Second ed. Vol. II.Athena Scientific.
Honeywell Inc . http://hpsweb.honeywell.com/Cultures/en-US/Products/Wireless/SecondGenerationWireless/default.htm . ISA . . Kumar, A. , Manjunath, D. , and Kuri, J. Communication Networking: An AnalyticalApproach . Morgan-Kaufmann (an imprint of Elsevier), San Francisco.
Kumar, A. , Manjunath, D. , and Kuri, J. Wireless Networking . Morgan-Kaufmann (animprint of Elsevier), San Francisco.
Niu, R. and Varshney, P. K.
EURASIP Journal on Wireless Communications and Networking 4,
Premkumar, K. and Kumar, A.
Proc. IEEE Infocom . AZ, USA.
Premkumar, K. , Kumar, A. , and Kuri, J. Proc. th Annual Allerton Conference onCommunication, Control, and Computing . IL, USA.
Rajasegarar, S. , Leckie, C. , and Palaniswami, M. IEEE Wireless Communications 15,
Shiryaev, A. N.
Optimal Stopping Rules . Springer, New York.
Singh, C. K. , Kumar, A. , and Ameer, P. M. Wireless Networks 14,
Tartakovsky, A. G. and Veeravalli, V. V.
SIAM Theory of Probability and its Applications 49,
3, 458–497.
Tenny, R. R. and Sandell, N. R.
IEEE Transactionson Aerospace and Electronic Systems 17 , 501–510.
Veeravalli, V. V.
IEEE Transactions on Infor-mation theory 47,4 (May), 1657–1665.Received March 2009; revised December 2009 and June 2010; accepted Month Year