q - 通过枚举进行数据规范化无速度增益

时间:2017-10-25 17:18:40

标签: kdb q-lang

q for mortals chapter on data normalisation中,即消除列表中重复的任务,它建议使用枚举来查找列表中的不同值,因为它遍历整数的速度快于遍历变量长度的符号

u:`g`ibm`intl`msft / unique list of tickers
v:1000000?u / list with duplicate tickers
k:u?v / positions in u
\t:10 distinct v / performing distinct on symbols 10 times and timing 
\t:10 distinct k / performing distinct on positions 10 times and timing 

我发现distinct vdistinct k快得多,这与承诺的内容不一致。

感谢您的帮助。

1 个答案:

答案 0 :(得分:2)

枚举通常用于保存到磁盘的数据以帮助压缩等 在那里,您将看到更大的性能提升。

KDB+ 3.5 2017.04.06 Copyright (C) 1993-2017 Kx Systems

Welcome to kdb+ 32bit edition
For support please see http://groups.google.com/d/forum/personal-kdbplus
Tutorials can be found at http://code.kx.com/wiki/Tutorials
To exit, type \\
To remove this startup msg, edit q.q
u:`g`ibm`intl`msft / unique list of tickers
v:1000000?u / list with duplicate tickers
q)k:`u$v //enumerate v against u
q)k
`u$`g`g`intl`ibm`intl`ibm`intl`msft`intl`ibm`g`msft`ibm`intl`intl`ibm`g`ibm`i..
q)save `:k
`:k
q)save `:u
`:u
q)save `:v
`:v
q)\\

KDB+ 3.5 2017.04.06 Copyright (C) 1993-2017 Kx Systems

Welcome to kdb+ 32bit edition
For support please see http://groups.google.com/d/forum/personal-kdbplus
Tutorials can be found at http://code.kx.com/wiki/Tutorials
To exit, type \\
To remove this startup msg, edit q.q
q)u:get `:u
q)\ts:10 distinct get `:v
462 8388848
q)\ts:10 distinct get `:k
37 4194544
q)

但你确实提出了一个有趣的问题,关于为什么在一个符号列表(在mem中)明显更快的一个整体列表。