Question

我有一个数字标识符列表，后跟一个或多个字符串。我想为每个标识符计算也存在于另一个标识符的关联字符串的最长组合。然后还有与每个标识符关联的字符串数。

例如

1   AAA    BBBA    ACA    CCD    ABADA
2   AAA    ACA     CCD
3   AAB    BBAC    DDAD
4   AAA    ACA     DDAD   CCD
5   AAA    ACA     DDAD   CCD

会导致：

ID  Longest Combo  Number of strings  
1         3               5
2         3               3
3         1               3
4         4               4
5         4               4

为了清楚起见，说明结果：

1 - AAA, ACA and CCD present in 2 so longest combo is 3.
2 - AAA, ACA and CCD present in 1 so longest combo is 3.
3 - DDAD present in 4 and 5 so longest combo is 1.
4 - AAA, ACA, DDAD and CCD present in 5 so longest combo is 4.
5 - AAA, ACA, DDAD and CCD present in 4 so longest combo is 4.

通常情况下，我可以尝试自己尝试破解某些东西，但与此同时却碰壁了-不知道从哪里开始。到此为止的分析是在awk中进行的，因此bash是理想的选择，但可能这是一项更适合R的工作？

我试图根据问题Frequency of each unique combination in data frame格式化数据，但没有成功。

任何帮助将不胜感激。

大约有3000个标识符，每个标识符包含1-15个字符串。

Answer 1

这是awk解决方案的原型。请注意，由于对称性，您只需要i<j的（i，j）项。

$ awk 'NR==FNR {for(i=2;i<=NF;i++) a[$1,$i]; size=$1; next} 
               {for(i=$1+1;i<=size;i++) 
                  {for(j=2;j<=NF;j++) 
                     if((i,$j) in a) {c[$1,i]++; list[$1,i]=list[$1,i] FS $j}
                   if(max[$1]<c[$1,i]) {max[$1]=c[$1,i]; maxM[$1]=i}} 
                   if(maxM[$1]) print $1,maxM[$1],"->",max[$1],list[$1,maxM[$1]]}' file{,}

1 2 -> 3  AAA ACA CCD
2 4 -> 3  AAA ACA CCD
3 4 -> 1  DDAD
4 5 -> 4  AAA ACA DDAD CCD

可以简化一些，我猜可以从列表大小中计算出匹配长度。

说明双遍算法，将每个记录ID的所有元素存储在查找表中；还设置大小。在第二遍中，将当前行与索引较高的所有行进行比较。找到交集大小并记录最大，对应的行和匹配的字段。

Answer 2

不知道R，不喜欢awk。一个bash解决方案怎么样？ XD

已编辑以包含评论。进行了一些小的更改，发现了一个可能的重要警告。请注意，您有一个AAA 和一个AAAA，由于AAAA与*AAA*匹配，当前代码可能会给您提供错误的匹配。如果是这样，请换行（但仍要注释），它将仍然有效，并会处理这种可能性。

$: cat proc
#! /bin/env bash

# predeclare these as associative arrays - string-based dictionary lookups
declare -A cnt=() set=() lookup=() combos=() hitlst=() hitcnt=() hitid=()

while read -a lst # this declares lst as a normal array and loads each line in fields
do id=${lst[0]}                            # field 0 assigned as the id
   lst=( ${lst[@]:1} )                     # lst assigned itself MINUS the first field
   cnt[$id]=${#lst[@]}                     # count for this id assigned # of  remaining elements in lst
   set[$id]="${lst[@]}"                    # set for id assigned the space-delimited elements of lst
   for k in ${lst[@]}                      # this will iterate k as each item in lst
   do  lookup["$k"]="${lookup["$k"]} $id"  # additively assigns, space delimited, this id as having this string
   done
done<dataset                               # when done reading the file, each key has a list of which id's it's on

# now we analyze our accumulated data
for id1 in ${!cnt[@]}                      # for every id (cnt has how many strings for each, ${!cnt[@]} is all the ids)
do  for k in ${set["$id1"]}                # iterate k over each string id1 had
    do  for id2 in ${lookup["$k"]}         # iterate id2 over each id that had this string
        do  [[ "$id1" == "$id2" ]] && continue # skip when both id pointers are looking at the same record
            case "${set["$id2"]}" in       # look at the whole set of strings assigned to id2 as a space-delimited string
            ### ==>> CAVEAT: might needed to be "$k "*|*" $k "*|*" $k") so that AAA doesn't match AA
            ### "$k "*|*" $k "*|*" $k") hitlst["$id1 $id2"]="${hitlst["$id1 $id2"]} $k";
            *$k*) hitlst["$id1 $id2"]="${hitlst["$id1 $id2"]} $k"; # when the current key from $id1 is anywhere in it
                  (( hitcnt["$id1 $id2"]++ )) ;;                   # increment the matches per pair of ids
            esac
            if (( ${hitcnt["$id1 $id2"]}0 > ${hitcnt["$id1"]}0 ))  # if matches/pair > stored max for id1
            then hitcnt["$id1"]=${hitcnt["$id1 $id2"]}             # upgrade to the new max
                 hitid["$id1"]="$id2"                              # and remember to report which rec while we're at it
            fi
       done
    done
done

printf "%s\t%s\t%s\t%s\n" "ID" "Longest Combo" "#strings" "[List of strings] / [matches] / matching ID"
for id in ${!cnt[@]}   # again, for every id
do  id2=${hitid[$id]}  # and again, this is the rec that had the most matches in case you wanted it (not requested)
    # print the fields with some formatting to align them
    printf "%s\t%s\t%s\t%s\n" "$id" "${hitcnt[$id]}" "${cnt[$id]}" "[${set[$id]}] / [${hitlst["$id $id2"]# }] / $id2"
done

$: proc
 ID Longest Combo  #strings [List of strings] / [matches] / matching ID
  1             3         5 [AAA BBBA ACA CCD ABADA] / [AAA ACA CCD] / 2
  2             3         3 [AAA ACA CCD] / [AAA ACA CCD] / 1
  3             1         3 [AAB BBAC DDAD] / [DDAD] / 4
  4             4         4 [AAA ACA DDAD CCD] / [AAA ACA DDAD CCD] / 5
  5             4         4 [AAA ACA DDAD CCD] / [AAA ACA DDAD CCD] / 4

我添加了一些字段并调整了输出格式，但是您明白了。只需3k记录，时间就不会太长。

计算琴弦组合的频率

2 个答案: