Faster algorithm for computing the edit distance between SLP-compressed strings Paweł Gawrychowski Max-Planck-Institut für Informatik
October 24, 2012
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
1 / 18
We consider the question of computing the edit distance between two strings.
Edit distance Given two strings a and b, what is the minimum number of insert/delete/replace operations required to transform a into b?
Observation Computing the edit distance between a and b is equivalent to finding the length of longest common subsequence of a and b.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
2 / 18
We consider the question of computing the edit distance between two strings.
Edit distance Given two strings a and b, what is the minimum number of insert/delete/replace operations required to transform a into b?
Observation Computing the edit distance between a and b is equivalent to finding the length of longest common subsequence of a and b. a = ATGCCGAC b = CAGACTAGA
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
2 / 18
We consider the question of computing the edit distance between two strings.
Edit distance Given two strings a and b, what is the minimum number of insert/delete/replace operations required to transform a into b?
Observation Computing the edit distance between a and b is equivalent to finding the length of longest common subsequence of a and b. a = AT GCCGAC b = CAGACTAGA
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
2 / 18
The question is: how quickly can be compute this number?
Use dynamic programming! For each i and j compute the longest common subsequence of a[1..i] and b[1..j]. Which results in a very simple quadratic time algorithm. More precisely, the running time is O(|a||b|).
Is there anything faster? Can be improved by either using bit parallelism, or assuming that some parameter (edit distance, number of matches, . . . ) is small.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
3 / 18
A natural direction is to consider the case when the input strings a and b are highly compressible. The question is: what method of compression should we assume?
Straight-line programs, or grammar compression A context-free grammar in Chomsky normal form with exactly one production for each nonterminal, hence generating exactly one string.
Fibonacci words F0 = 1 F1 = 0 F0 = 1
F2 = 01
F1 = 0
F3 = 010
Fn+2 = Fn+1 Fn for all n ≥ 0 F4 = 01001 F5 = 01001010 F6 = 0100101001001 F7 = 010010100100101001010 Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
4 / 18
A natural direction is to consider the case when the input strings a and b are highly compressible. The question is: what method of compression should we assume?
Straight-line programs, or grammar compression A context-free grammar in Chomsky normal form with exactly one production for each nonterminal, hence generating exactly one string.
Fibonacci words F0 = 1 F1 = 0 F0 = 1
F2 = 01
F1 = 0
F3 = 010
Fn+2 = Fn+1 Fn for all n ≥ 0 F4 = 01001 F5 = 01001010 F6 = 0100101001001 F7 = 010010100100101001010 Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
4 / 18
A natural direction is to consider the case when the input strings a and b are highly compressible. The question is: what method of compression should we assume?
Straight-line programs, or grammar compression A context-free grammar in Chomsky normal form with exactly one production for each nonterminal, hence generating exactly one string.
Fibonacci words F0 = 1 F1 = 0 F0 = 1
F2 = 01
F1 = 0
F3 = 010
Fn+2 = Fn+1 Fn for all n ≥ 0 F4 = 01001 F5 = 01001010 F6 = 0100101001001 F7 = 010010100100101001010 Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
4 / 18
Why this compression method? 1
as powerful as all Lempel-Ziv schemes (i.e., LZ77 can be converted into SLP with small size increase),
2
easy to work with!
Problem We are given two strings a and b described by two SLPs. n is the total number of rules in both SLPs N is the total length of both words How quickly can we compute LCS(a, b)?
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
5 / 18
Why this compression method? 1
as powerful as all Lempel-Ziv schemes (i.e., LZ77 can be converted into SLP with small size increase),
2
easy to work with!
Problem We are given two strings a and b described by two SLPs. n is the total number of rules in both SLPs N is the total length of both words How quickly can we compute LCS(a, b)?
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
5 / 18
Whenever we are working with compressed data, the goal is to achieve running time depending just on the size of the compressed representation. Unfortunately, in our case this is not possible.
Lifshits and Lohrey MFCS 2006 Checking if a is a subsequence of b is N P-hard.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
6 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Tiskin Journal of Mathematical Sciences 158(5), 759–769 (2009) O(nN 1.5 ) is possible!
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Tiskin Journal of Mathematical Sciences 158(5), 759–769 (2009) O(nN 1.5 ) is possible!
Hermelin, Landau, Landau and Weimann STACS 2009 O(n1.4 N 1.2 ) is possible!
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Tiskin SODA 2010 Well, actually O(nN log N) is possible!
Hermelin, Landau, Landau and Weimann STACS 2009 O(n1.4 N 1.2 ) is possible!
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Tiskin SODA 2010 Well, actually O(nN log N) is possible!
Hermelin, Landau, Landau and Weimann STACS 2009 (full) ...finally, O(nN log Nn ) is possible.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
Lifshits CPM 2007 ...but maybe O(nN) running time would be possible?
Tiskin SODA 2010 Well, actually O(nN log N) is possible!
Hermelin, Landau, Landau and Weimann STACS 2009 (full) ...finally, O(nN log Nn ) is possible.
This paper O(nN
q
log Nn ) time algorithm.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
7 / 18
The aforementioned papers actually solve a more general weighted version of the problem in the same complexity, for any rational scoring function. The same is true for the improvement, but today we will focus just on the unweighted case.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
8 / 18
High-level idea Same as in the previous solutions: look at the alignment dag...
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
9 / 18
High-level idea Same as in the previous solutions: look at the alignment dag...
A
B
C
B
A
A
B A C B B A
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
9 / 18
... and notice that if the strings are highly repetitive, many fragments of the dag look somehow similar, and we can avoid duplicating work.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
10 / 18
... and notice that if the strings are highly repetitive, many fragments of the dag look somehow similar, and we can avoid duplicating work.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
10 / 18
... and notice that if the strings are highly repetitive, many fragments of the dag look somehow similar, and we can avoid duplicating work.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
10 / 18
... and notice that if the strings are highly repetitive, many fragments of the dag look somehow similar, and we can avoid duplicating work. X1
X2
X3
X4
X5
X6 x
X10 X20 X30 X40 X50 X60 x
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
10 / 18
To make life simple, we want to partition the whole dag into blocks corresponding to pairs of nonterminals.
Theorem Given a SLP and a parameter x, we can (quickly) construct a new SLP of roughly the same size and with the following two properties: 1
each nonterminal describes a string of length at most x,
2
the original string can be represented as a concatenation of roughly Nx new nonterminals.
x will be chosen later.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
11 / 18
To make life simple, we want to partition the whole dag into blocks corresponding to pairs of nonterminals.
Theorem Given a SLP and a parameter x, we can (quickly) construct a new SLP of roughly the same size and with the following two properties: 1
each nonterminal describes a string of length at most x,
2
the original string can be represented as a concatenation of roughly Nx new nonterminals.
x will be chosen later.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
11 / 18
To make life simple, we want to partition the whole dag into blocks corresponding to pairs of nonterminals.
Theorem Given a SLP and a parameter x, we can (quickly) construct a new SLP of roughly the same size and with the following two properties: 1
each nonterminal describes a string of length at most x,
2
the original string can be represented as a concatenation of roughly Nx new nonterminals.
x will be chosen later.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
11 / 18
We want to compute only the values on the boundaries between blocks. Consider the situation around a single block.
We need to take a closer look at how the values on the right and bottom boundary (outputs) depend on the values on the left and top boundary (inputs). Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
12 / 18
We want to compute only the values on the boundaries between blocks. Consider the situation around a single block. I0
I1
I2
I3
I4
I5
I6 O12
I−1
O11
I−2
O10
I−3
O9
I−4
O8
I−5 I−6 O0
O7 O1
O2
O3
O4
O5
O6
We need to take a closer look at how the values on the right and bottom boundary (outputs) depend on the values on the left and top boundary (inputs). Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
12 / 18
We want to compute only the values on the boundaries between blocks. Consider the situation around a single block. I0
I1
I2
I3
I4
I5
I6 O12
I−1
O11
I−2
O10
I−3
O9
I−4
O8
I−5 I−6 O0
O7 O1
O2
O3
O4
O5
O6
We need to take a closer look at how the values on the right and bottom boundary (outputs) depend on the values on the left and top boundary (inputs). Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
12 / 18
Let Hα,β (i, j) be the highest scoring path from the i-th input to the j-th output of the block corresponding to words α and β. Then the matrix Hα,β has a very simple structure: Hα,β (i, j) = j − i − P Σ (i, j)
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
13 / 18
Let Hα,β (i, j) be the highest scoring path from the i-th input to the j-th output of the block corresponding to words α and β. Then the matrix Hα,β has a very simple structure: Hα,β (i, j) = j − i − P Σ (i, j) 1 1 1 1 1 1 1
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
13 / 18
Let Hα,β (i, j) be the highest scoring path from the i-th input to the j-th output of the block corresponding to words α and β. Then the matrix Hα,β has a very simple structure: Hα,β (i, j) = j − i − P Σ (i, j) 1 1 1 1 (i, j)
1
1 1
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
13 / 18
Hence each Hα,β can be represented in a succinct form by simply storing the nonzeroes of the permutation matrix. The question is how to compute this representation efficiently?
Theorem (Tiskin SODA 2010) Given the representation of Hα0 ,β and Hα00 ,β , we can compute the representation of Hα0 α00 ,β in O(x log x) time, where |α0 |, |α00 |, |β| ≤ x. Note that Hα0 α00 ,β is simply the max-product of Hα0 ,β and Hα00 ,β .
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
14 / 18
Hence each Hα,β can be represented in a succinct form by simply storing the nonzeroes of the permutation matrix. The question is how to compute this representation efficiently?
Theorem (Tiskin SODA 2010) Given the representation of Hα0 ,β and Hα00 ,β , we can compute the representation of Hα0 α00 ,β in O(x log x) time, where |α0 |, |α00 |, |β| ≤ x. Note that Hα0 α00 ,β is simply the max-product of Hα0 ,β and Hα00 ,β .
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
14 / 18
Theorem Given the representation of Hα,β and the vector u storing the input values, we can compute the vector v storing the output values in O(x) time, where |α|, |β| ≤ x. Note that v is simply the max-product of Hα,β and u. This assumes the word RAM model with w ∈ Ω(log n).
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
15 / 18
Theorem Given the representation of Hα,β and the vector u storing the input values, we can compute the vector v storing the output values in O(x) time, where |α|, |β| ≤ x. Note that v is simply the max-product of Hα,β and u. This assumes the word RAM model with w ∈ Ω(log n).
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
15 / 18
Fast matrix-vector multiplication We reduce the question to a very simple data structure problem, which is to maintain an array of length x under two operations: 1
decreasing all values in some prefix of the array by one,
2
computing the maximum in the whole array.
This can be reduced further to an even simpler problem, which is to maintain a partition of [1, x] in to a number of disjoint segments under the following two operations: 1
locating the segment a given element belongs to,
2
merging two adjacent segments.
This is known as the interval-union-find problem, and a (not very complicated) amortized constant time solution is known.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
16 / 18
Fast matrix-vector multiplication We reduce the question to a very simple data structure problem, which is to maintain an array of length x under two operations: 1
decreasing all values in some prefix of the array by one,
2
computing the maximum in the whole array.
This can be reduced further to an even simpler problem, which is to maintain a partition of [1, x] in to a number of disjoint segments under the following two operations: 1
locating the segment a given element belongs to,
2
merging two adjacent segments.
This is known as the interval-union-find problem, and a (not very complicated) amortized constant time solution is known.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
16 / 18
Fast matrix-vector multiplication We reduce the question to a very simple data structure problem, which is to maintain an array of length x under two operations: 1
decreasing all values in some prefix of the array by one,
2
computing the maximum in the whole array.
This can be reduced further to an even simpler problem, which is to maintain a partition of [1, x] in to a number of disjoint segments under the following two operations: 1
locating the segment a given element belongs to,
2
merging two adjacent segments.
This is known as the interval-union-find problem, and a (not very complicated) amortized constant time solution is known.
Paweł Gawrychowski
Faster edit distance between SLPs
October 24, 2012
16 / 18
Now we only have to combine those two theorems and compute the total running time, which turns out to be proportional to: n2 x log x + choosing x = √ f
log f
, where f =
N n
N2 x
gives us log x ≤ log f and hence we
get the following bound on the running time: p p N2 p n f log f + log f = nN log f = nN f 2
Paweł Gawrychowski
Faster edit distance between SLPs
r log
N n
October 24, 2012
17 / 18
Now we only have to combine those two theorems and compute the total running time, which turns out to be proportional to: n2 x log x + choosing x = √ f
log f
, where f =
N n
N2 x
gives us log x ≤ log f and hence we
get the following bound on the running time: p p N2 p n f log f + log f = nN log f = nN f 2
Paweł Gawrychowski
Faster edit distance between SLPs
r log
N n
October 24, 2012
17 / 18
Now we only have to combine those two theorems and compute the total running time, which turns out to be proportional to: n2 x log x + choosing x = √ f
log f
, where f =
N n
N2 x
gives us log x ≤ log f and hence we
get the following bound on the running time: p p N2 p n f log f + log f = nN log f = nN f 2
Paweł Gawrychowski
Faster edit distance between SLPs
r log
N n
October 24, 2012
17 / 18
1 2
Can we avoid bit manipulation? q Can we get rid of the annoying log Nn factor?
Questions?
Paweł Gawrychowski
,
Faster edit distance between SLPs
October 24, 2012
18 / 18
