Algebraic Techniques

Wafi AlBalawi and Yan Liu Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV wafi,yanliu@csee.wvu.edu

1 Introduction

In this chapter we describe some basic randomization techniques with an underlying algebraic flavor. We begin by describing Freivalds’ technique for the verification of identities involving matrices, polynomials, and integers. We describe how this generalizes to the Schwartz-Zippel technique for identities involving multivariate polynomials, and we illustrate this technique by applying it to the problem of detecting the existence of perfect matchings in graphs.

Most of these techniques and their applications involve a fingerprinting mechanism, which can be described as follows. Consider the problem of deciding the equality of two elements x and y drawn from a large universe U. Under any ”reasonable” model of computation, testing the equality of x and y then has a deterministic complexity of at least log |U|. An alternative approach is to pick a random mapping from U into a significantly smaller universe V in such a way that there is a good chance that x and y are identical if and only if their images are identical. The images of x and y are their fingerprints, and their equality can be verified in log |V | time by comparing the fingerprints.

2 Fingerprinting and Freivalds’ Technique

We illustrate fingerprinting by describing a technique for verifying matrix multiplication. The fastest known algorithm for matrix multiplication runs in time O(n^{2.376}), which improves significantly on the obvious O(n^{3}) time algorithm but has the disadvantage of being extremely complicated. Suppose we are given an implementation of this algorithm and would like to verify its correctness. Since program verification is a difficult task, a reasonable goal might be to verify the correctness of the output produced on specific executions of the algorithm.

Given n × n matrices A, B, we can calculate matrix C = AB in O(n^{2.376}). The problem: if you are given A, B, and C over the field F, we would like to verify if indeed AB = C. We would like to use the fact that we do not have to compute C. The following technique, known as Freivalds’ technique 2.1, provides an elegant solution which takes O(n^{2}) time with bounded error probability.

1: Choose a random vector r⃗ ∈ {0, 1}^{n}; each component of r is chosen independently and uniformly at random

from 0 and 1, the additive and multiplicative identities of the field F. 2: Compute x = Br, y = Ax = ABr, z = Cr in O(n^{2}) time. 3: If AB = C then y = z 4: We show that Pr(y ≠ z|AB ≠ C) ≥ \frac{1}{2}, algorithem errs only if AB ≠ C BUT y = z

Algorithm 2.1: Freivalds’ Algorithm Theorem: 2.1 Let A, B and C be n × n matrices over F such that AB ≠ C. Then for r chosen uniformly at random from {0, 1}^{n}:

P r[ABr = Cr|AB ≠ C] ≤ \frac{1}{2}

\underline{Proof}:

Let D = AB − C, D ≠ Z where Z is the 0-matrix. Permute D so that the first row has non-zero entries and all non-zero occur before zeros. Let d be the first row in D, the first entry in Dr is d^{T} r. P r[d^{T} r] ≠ 0 is a lower bound on Pr[Dr]=[0].

Now the inner product, d^{T} r = 0 if and only if r_{1} = ^{\frac{− ∑_{i=2}^{k} d_{i}r_{i}}{d_{1}}} . By the principle of deferred decision, the right hand side of the above expression has a predetermined value v, which is 0 or 1 with uniform probability. Then Pr[r_{1} = v] ≤ \frac{1}{2}.

□

3 Verifying Polynomial Identities

In this section we show that Freivalds’ technique applies to the verification of identities involving polynomials. Two polynomials P (x) and Q(x) are said to be equal if they have the same coefficients for corresponding powers of x. Verifying identities of integers, or, in general, strings over any fixed alphabet, is a special case since we can represent any string of length n as a polynomial of degree n. This is achieved by treating the k^{th} element in the string as the coefficient of the k^{th} power of a symbolic variable.

3.1 Verifing Polynomial Identities (Univariate)

We first consider the polynomial product verification problem: given polynomials P_{1}(x), P_{2}(x), P_{3}(x) ∈ F[X] where F is a field, verify that P_{1}(x) × P_{2}(x) = P_{3}(x). Assume that the polynomials P_{1}(x) and P_{2}(x) are of degree at most n; then P_{3}(x) cannot have degree exceeding 2n. Polynomials of degree n can be multiplied in O(n log n) time using Fast Fourier Transforms, whereas the evaluation of a polynomial at a fixed point requires O(n) time.

Let S⊆ F be a set of size at least 2n + 1 pick r ∈ S uniformly at random. if P_{1}(r) × P_{2}(r) = P_{3}(r), then with high probability P_{1}(x) × P_{2}(x) = P_{3}(x) if P_{1}(r) × P_{2}(r) ≠ P_{3}(r), then we know P_{1}(x) × P_{2}(x) ≠ P_{3}(x) we err only if we say YES where it is not.

Theorem: 3.1 Let Q(x) be a polynomial of degree d over field F. Let S ⊆ F be a finite subset of F. Let r ∈ S be chosen uniformly at random, then

P r[Q(r) = 0 | Q(x) ≠ 0] ≤ \frac{d}{|S|}

\underline{Proof}: The degree of Q(x) is d, so it can be 0, at most, d points. Any two non-zero degree d polynomials can touch at, at most d points. □

3.2 Schwartz-Zippel Theorem

Theorem: 3.2 Let Q(x_{1}, ..., x_{n}) ∈ F[x_{1}, ..., x_{n}] be a multivariate of total degree d. Fix any finite set S ⊆ F, and let r_{1}, ..., r_{n} be chosen independently and uniformly at random form S. Then

P r[Q(r_{1}, ..., r_{n}) = 0 | Q(x_{1}, ..., x_{n}) ≠ 0] ≤ \frac{d}{|S|}

\underline{Proof}: Proof by induction: The Base Case: Q(x_{1}) of degree d. (As before). Assume that the induction hypothesis is true for a multivariate polynomial with n − 1 variable. Consider Q(x_{1}, .., x_{n}) = ∑_{i=0}^{k} X_{1}^{i}Q_{i}(x_{2}, ..., x_{n}), Consider Q(x_{(}2), ..., x_{n} ≠ 0) by choice of k. P r[Q_{k}(r_{2}, ..., r_{n}) = 0 | Q_{k}(x_{2}, ..., x_{n}) ≠ 0] ≤ \frac{d−k}{|S|} by the hypothesis. Now Assume that Q_{k}(r_{2}, r_{3}, ..., r_{n}) ≠ 0. Consider q(x_{1}) = Q(x_{1}, r_{2}, r_{3}, ..., r_{n}) = ∑_{i=0}^{k} X_{1}^{i}Q_{i}(r_{2}, ..., r_{n}), q(x_{1}) has degree k and it is NOT identically 0.

The base case implies that:

P r[q(r_{1} = 0) | q(x_{1} ≠ 0) ≤ \frac{k}{S} ]

Now we have these two facts:

1. P r[Q_{k}(r_{2}, r_{3}, ..., r_{n}) = 0] ≤ \frac{(d−k)}{S}.

2. P r[Q_{k}(r_{1}, r_{2}, ..., r_{n}) = 0 | Q_{k}(r_{2}, ..., r_{n}) ≠ 0] ≤ \frac{(k)}{S}.

Then Pr[Q_{k}(r_{1}, r_{2}, ..., r_{n}) = 0 | Q(x_{1}, x_{2}, ..., x_{n}) ≠ 0)] ≤ \frac{(d−k)}{S}.

Invoking the result in Exercise 3.1, we find that the probability that Q(r_{1}, r_{2}, ..., r_{n}) = 0 is no more than the sum of these two probabilities, which is \frac{d}{S} . □

Exercise 3.1: Show that for any two events E_{1} and E_{2},

P r[E_{1}] ≤ P r[E_{1}|Ē_{2}] + P r[E_{2}]

Solution: look at the lecture notes ”A Probabilistic Identity” which posted on the class website.

4 Perfect Matchings in Graphs

Consider a bipartite graph G(U, V, E) with the independent sets of vertices U = u_{1}, ..., u_{n} and V = v_{1}, ..., v_{n}. A matching is a collection of edges M ⊆ E such that each vertex occurs at most once in M. A perfect matching is a matching of size n. Each perfect matching M in G can be viewed as a permutation from U into V. More precisely, the perfect matchings in G can be put into a one-to-one correspondence with the permutations in S_{n}, where the matching corresponding to a permutation π ∈ S_{n} is given by the pairs (u_{i}, v_{π}(i)), for 1 ≤ i ≤ n.

Edmonds’ Theorem

Theorem: 4.1 Let A be the n × n matrix obtained from G(U, V, E) as follows:

A_{ij} = x_{ij} (u_{i}, v_{j}) ∈ E
= 0 (u_{i}, v_{j}) ∉ E

Define the multivariate polynomial Q(x_{1}, x_{2}, ..., x_{n}) as being equal to det(A). Then, G has a perfect matching if and only if Q ≠ 0.

\underline{Proof}: The determinant of A is given by

det(A) = ∑_{π∈S_{n}} sgn(π)A_{1,π(1)}, A_{2,π(2)}, ...A_{n,π(n)}.

Since each indeterminate x_{ij} occurs at most once in A, there can be no cancellation of the terms in the summation. Therefore the determinant is not identically zero if and only if there is a permutation π for which the corresponding term in the summation is non-zero. The latter happens if and only if each of the entries A_{i}, π_{(}i), for 1 ≤ i ≤ n, is non-zero. This is equivalent to having a perfect matching in G. □

References

[1] Rajeev Motawani, and Prabhakar Raghavan. Randomized Algorithms. Cambridge University Press, 1995.