在做一道題時碰到的問題，記錄下來，備忘
題目：Substring with Concatenation of All Words
簡單就是說，給一個字符串s，再給一個由字符串words（每個字符串的長度一樣）組成的數組。遍歷s，如果其中的一個子字符串sub_s是可以由words中的所有字符串組成。那么就記錄該sub_s的首字符的index。再把所有的index返回成一個數組。
- 最開始的想法很直接，即把words中的所有元素進行一次排列
Array#permutation，然后將排列好的每一個數組join成一個字符串，再用String#index方法找到索引，再返回
def find_substring1(s, words)
array =[]
words.permutation.map(&:join).each do |ele|
array << s.index(ele)
end
array.compact.sort
end
這樣寫的好處是簡潔，充分體現了ruby的優勢，但缺點是一旦words中的元素有很多，比如10個以上，那么使用permutation就會生成10!這么多的排列，不僅執行速度上會變得很慢，還很容易溢出。所以這個方法是不可取的。
- 接著嘗試了另一種方法
def find_substring2(s, words)
array = []
m = words.first.size
return [] if s.size < words.size
j = 0
until j > s.size - words.size * m
words_dup = words.dup
j.step(s.size-1,m) do |i|
substr = s[i,m]
if words_dup.include? substr
index = words_dup.index(substr)
words_dup.delete_at(index)
if words_dup.empty?
array << j
break
end
else
break
end
end
j += 1
end
array
end
從s[0](j=0)開始，先判斷s[0,m](這里假設words中每個字符串的長度是m)是否在words_dup數組中，如果在，則從words_dup數組中刪除該字符串，然后再判斷s[m,m]是否在words_dup數組中，如果不在，則重新開始，從s[1](j+=1)開始重新判斷。如果words_dup為空了，說明s中存在一個子字符串是由words中所有的字符串組成的，此時將j添加到array中，最后返回array。
這樣可以滿足大部分情況，但是當s或者words特別大的時候，比如：
s = "ab" * 5000
words = ["ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba","ab","ba"]
此時該方法的性能會變得很差。
- 上面方法的瓶頸應該是在include?、index、delete_at這三個方法上。所以想到，是否可以用hash來將字符串的判斷變成整數的運算提高性能。
def find_substring3(s, words)
array = []
h = Hash.new(0)
words.each {|ele| h[ele] += 1}
m = words.first.size
return [] if s.size < words.size
j = 0
until j > s.size - words.size * m
hash = h.dup
j.step(s.size-1,m) do |i|
substr = s[i,m]
if hash[substr] > 0
hash[substr] -= 1
if hash.all? {|k,v| v == 0}
array << j
break
end
else
break
end
end
j += 1
end
array
end
整個判斷邏輯是一樣的，不同的是，在最開始的時候，將words數組轉化為了一個hash，其中key是words中的字符串元素，value是這些字符串元素在words中的重復次數。這樣hash的長度肯定是小于等于words的。省去了一些空間。同時將判斷變成了hash[substr] > 0(對應上面方法的include?)、刪除元素變成了hash[substr] -= 1(對應上面方法的Index,delete_at)。運行一下看看效果。
結果卻變得更慢了！
- 那是什么原因呢？看代碼發現有一行是hash.all? {|k,v| v == 0}，此處是判斷，當words中所有的字符串的次數都為0里，說明s中存在一個子字符串是由words中所有的字符串組成的。分析可以知道，每次循環，都會去調用all?這個方法，而這個方法是比較耗時的，因為是需要遍歷hash中的每一對鍵值。所以再次修改
def find_substring4(s, words)
array = []
h = Hash.new(0)
words.each {|ele| h[ele] += 1}
m = words.first.size
return [] if s.size < words.size
j = 0
until j > s.size - words.size * m
hash = h.dup
j.step(s.size-1,m) do |i|
substr = s[i,m]
if hash[substr] > 0
hash[substr] -= 1
#這里
unless hash.any? {|k,v| v > 0}
array << j
break
end
else
break
end
end
j += 1
end
array
end
將all?改為了any?,這樣就不需要遍歷整個hash了。
OK。可以用benchmark看一下結果(因為permutation方法在這種輸入的情況下已經溢出了，所以就不執行了)
require 'benchmark'
Benchmark.bm(10) do |t|
t.report("find_substring2") { find_substring2(s, words) }
t.report("find_substring3") { find_substring3(s, words) }
t.report("find_substring4") { find_substring4(s, words) }
end
結果如下：
user system total real
find_substring2 2.329000 0.000000 2.329000 ( 2.327370)
find_substring3 3.312000 0.000000 3.312000 ( 3.376481)
find_substring4 0.609000 0.000000 0.609000 ( 0.620764)
提升很明顯！
- 再試著優化一下
def find_substring5(s, words)
array = []
h = Hash.new(0)
words.each {|ele| h[ele] += 1}
m = words.first.size
return [] if s.size < words.size
j = 0
until j > s.size - words.size * m
hash = h.dup
j.step(s.size-1,m) do |i|
substr = s[i,m]
if hash[substr] > 0
hash[substr] -= 1
#這里
hash.delete(substr) if hash[substr] == 0
if hash.empty?
array << j
break
end
else
break
end
end
j += 1
end
array
end
將any?改成有empty?，并且在之前增加了一步操作，如果hash[substr]的重復次數為0，那么就在hash中刪除掉該元素。這樣在循環的過程中，hash的元素會變小，因此要比any? 操作的數據小一些。
驗證一下：
require 'benchmark'
Benchmark.bm(10) do |t|
t.report("find_substring4") { find_substring4(s, words) }
t.report("find_substring5") { find_substring5(s, words) }
end
結果如下：
user system total real
find_substring4 0.610000 0.016000 0.626000 ( 0.624748)
find_substring5 0.531000 0.000000 0.531000 ( 0.526358)
提升不是很明顯，但還是有提升的。
試著總結一下（不一定正確，但可以往這個方向去試著優化）
- 如果可以用整數操作，盡量用整數進行操作，比字符串的操作要快
- 一些方法比如permutation、combination用起來很方便，但卻是犧牲了空間和時間的
- 另一些方法比如all?需要遍歷才能最終判斷的，盡量用any?來取代。而empty?的開銷還要小一些。
- 一般來說，在大數據的情況下，Hash的表現要高于Array
