Mayur Thakur

Diversity maximization under matroid constraints

Aggregator websites typically present documents in the form of representative clusters. In order for users to get a broader perspective, it is important to deliver a diversified set of representative documents in those clusters. One approach to diversification is to maximize the average dissimilarity among documents. Another way to capture diversity is to avoid showing several documents from the same category (e.g. from the same news channel). We combine the above two diversification concepts by modeling the latter approach as a (partition) matroid constraint, and study diversity maximization problems under matroid constraints. We present the first constant-factor approximation algorithm for this problem, using a new technique. Our local search 0.5-approximation algorithm is also the first constant-factor approximation for the max-dispersion problem under matroid constraints. Our combinatorial proof technique for maximizing diversity under matroid constraints uses the existence of a family of Latin squares which may also be of independent interest. In order to apply these diversity maximization algorithms in the context of aggregator websites and as a preprocessing step for our diversity maximization tool, we develop greedy clustering algorithms that maximize weighted coverage of a predefined set of topics. Our algorithms are based on computing a set of cluster centers, where clusters are formed around them. We show the better performance of our algorithms for diversity and coverage maximization by running experiments on real (Twitter) and synthetic data in the context of real-time search over micro-posts. Finally we perform a user study validating our algorithms and diversity metrics.

Preprocessing DNS Log Data for Effective Data Mining

The Domain Name Service (DNS) provides a critical function in directing Internet traffic. Defending DNS servers from bandwidth attacks is assisted by the ability to effectively mine DNS log data for statistical patterns. Processing DNS log data can be classified as a data-intensive problem, and as such presents challenges unique to this class of problem. When problems occur in capturing log data, or when the DNS server experiences an outage (scheduled or unscheduled), the normal pattern of traffic for that server becomes clouded. Simple linear interpolation of the holes in the data does not preserve features such as peaks in traffic (which can occur during an attack, making them of particular interest). We demonstrate a method for estimating values for missing portions of time sensitive DNS log data. This method would be suitable for use with a variety of datasets containing time series values where certain portions are missing.

One-way permutations and self-witnessing languages

A desirable property of one-way functions is that they be total, one-to-one, and onto--in other words, that they be permutations. We prove that one-way permutations exist exactly if P ≠ UP ∩ coUP. This provides the first characterization of the existence of one-way permutations based on a complexity-class separation and shows that their existence is equivalent to a number of previously studied complexity-theoretic hypotheses. We study permutations in the context of witness functions of nondeterministic Turing machines. A language is in PermUP if, relative to some unambiguous, nondeterministic, polynomial-time Turing machine accepting the language, the function mapping each string to its unique witness is a permutation of the members of the language. We show that, under standard complexity-theoretic assumptions, PermUP is a strict subset of UP. We study SelfNP, the set of all languages such that, relative to some nondeterministic, polynomial-time Turing machine that accepts the language, the set of all witnesses of strings in the language is identical to the language itself. We show that SAT ∈ SelfNP, and, under standard complexity-theoretic assumptions, SelfNP ≠ NP.

A Game-Theoretic Framework for Bandwidth Attacks and Statistical Defenses

We introduce a game-theoretic framework for reasoning about bandwidth attacks, a common form of distributed denial of service (DDoS) attacks. In particular, our traffic injection game models the attacker as a rational but limited-resource entity who uses limited knowledge of traffic patterns to launch IP spoofing based bandwidth attacks on a server. We model the defender as a coarse-grained, relative volume based statistical filter. We analyze the effectiveness of the defender against the attacker by analyzing the payoffs of various strategies in the traffic injection game. Furthermore, we analyze how these payoffs change in the presence of random noise. Our results show that there is potential for using statistical methods for creating defense mechanisms that can detect a DDoS attack and that even when an attacker has a priori knowledge of the expected traffic volume for the dimension and divisions employed in the attack, the attack traffic can still be exposed to the defender.

Quantum and classical complexity classes: separations, collapses, and closure properties

We study the complexity of quantum complexity classes such as EQP, BQP, and NQP (quantum analogs of P, BPP, and NP, respectively) using classical complexity classes such as ZPP, WPP, and C=P. The contributions of this paper are threefold. First, via oracle constructions, we show that no relativizable proof technique can improve the best known classical upper bound for BQP (BQP⊆AWPP [Journal of Computer and System Sciences 59(2) (1999) 240]) to BQP⊆WPP and the best known classical lower bound for EQP (P⊆EQP) to ZPP⊆EQP. Second, we prove that there are oracles A and B such that, relative to A, coRP is immune to NQP and relative to B, BQP is immune to pC=P. Extending a result of de Graaf and Valiant [Technical Report quant-ph/0211179, Quantum Physics (2002)], we construct a relativized world where EQP is immune to MODpkP. Third, motivated by the fact that counting classes (e.g., LWPP, AWPP, etc.) are the best known classical upper bounds on quantum complexity classes, we study properties of these counting classes. We prove that WPP is closed under polynomial-time truth-table reductions, while we construct an oracle relative to which WPP is not closed under polynomial-time Turing reductions. The latter result implies that proving the equality of the similar appearing classes LWPP and WPP would require nonrelativizable proof techniques. We also prove that both AWPP and APP are closed under ≤TUP reductions. We use closure properties of WPP and AWPP to prove interesting consequences, in terms of the complexity of the polynomial-hierarchy, of the following hypotheses: NQP⊆BQP and EQP=NQP.

Complexity of linear connectivity problems in directed hypergraphs

We introduce a notion of linear hyperconnection (formally denoted L-hyperpath) between nodes in a directed hypergraph and relate this notion to existing notions of hyperpaths in directed hypergraphs. We observe that many interesting questions in problem domains such as secret transfer protocols, routing in packet filtered networks, and propositional satisfiability are basically questions about existence of L-hyperpaths or about cyclomatic number of directed hypergraphs w.r.t. L-hypercycles (the minimum number of hyperedges that need to be deleted to make a directed hypergraph free of L-hypercycles). We prove that the L-hyperpath existence problem, the cyclomatic number problem, the minimum cyclomatic set problem, and the minimal cyclomatic set problem are each complete for a different level (respectively, NP, Σ p 2 , Π p 2 , and DP) of the polynomial hierarchy.

One-Way Permutations and Self-Witnessing Languages

A desirable property of one-way functions is that they be total, one-to-one, and onto—in other words, that they be permutations. We prove that one-way permutations exist exactly if P ≠ UP ∩ coUP. This provides the first characterization of the existence of one-way permutations based on a complexity-class separation and shows that their existence is equivalent to a number of previously studied complexity-theoretic hypotheses.

Recursive Decomposition of Progress Graphs

Search of a state transition system is traditionally how deadlock detection for concurrent programs has been accomplished. This paper examines an approach to deadlock detection that uses geometric semantics involving the topological notion of dihomotopy to partition the state-space into components; after that the reduced state-space is exhaustively searched. Prior work partitioned the state-space inductively. In this paper we show that a recursive technique provides greater reduction of the size of the state transition system and therefore more efficient deadlock detection. If the preprocessing can be done efficiently, then for large problems we expect to see more efficient deadlock detection and eventually more efficient verification of some temporal properties.

Errata for the paper Predecessor existence problems for finite discrete dynamical systems [Theoret. Comput. Sci. 386 (1-2) (2007) 3-37]

We point out some minor corrections to the proof of Theorem 4.1 in the above paper. As presented, the proof of Theorem 4.1 uses a reduction from a special version of the Planar 3SAT problem (called RP3SAT) in which each clause has exactly three literals, each variable appears in at most three clauses and the factor graph corresponding to the instance is planar. It is incorrect to use a reduction from RP3SAT to prove the hardness of the predecessor existence problems for SDSs and SyDSs on grids since the RP3SAT problem is efficiently solvable even without the planarity restriction, as shown by Tovey [2]. However, the result of Theorem 4.1 can be established by a reduction from a restricted version of Planar 3SAT (henceforth referred to as Pl-B3SAT) in which each clause has two or three literals and each variable appears in at most three clauses. There is a local replacement-based ultra efficient (decision, parsimonious) reduction from 3SAT to Pl-B3SAT [1]. Thus, Pl-B3SAT is NP-complete, #Pl-B3SAT is #P-complete, Unique-Pl-B3SAT is DP -complete and Ambiguous-Pl-B3SAT is NP-complete. In the proof of Theorem 4.1, when the reduction is carried out from Pl-B3SAT, the constructions of the underlying grids for both SDSs and SyDSs remain the same. The only change is that the local transition functions for each node corresponding to a two literal clause should be the 4-simple-threshold function. (For each node corresponding to a three literal clause, the local transition function remains the 3-simple-threshold function, as in the paper.) Because of this change, some minor modifications are also needed for a few sentences in the discussion of the construction. These modifications are indicated below. (The page and line numbers used in the following discussion correspond to the corrected page proofs for the paper.)

Lower Bounds and the Hardness of Counting Properties

Rice’s Theorem states that all nontrivial language properties of recursively enumerable sets are undecidable. Borchert and Stephan [BS00] started the search for complexity-theoretic analogs of Rice’s Theorem, and proved that every nontrivial counting property of boolean circuits is UP-hard. Hemaspaandra and Rothe [HR00] improved the UP-hardness lower bound to UP O (1)-hardness. The present paper raises the lower bound for nontrivial counting properties from UP O (l)-hardness to FewPhardness, i.e., from constant-ambiguity nondeterminism to polynomial-ambiguity nondeterminism. Furthermore, we prove that this lower bound is rather tight with respect to relativizable techniques, i.e., no relativizable technique can raise this lower bound to FewP-≤ 1-tt p -hardness. We also prove a Rice-style theorem for NP, namely that every nontrivial language property of NP sets is NP-hard.