如何减少这个数据库的大小?

时间:2011-04-03 15:20:00

标签: javascript jquery database compression

我有一个庞大的信息数据库,我需要减少它的大小,因为它非常大而且非常费力。什么是最好的方法来“压缩”然后使用javascript扩展它?

有什么工具可以帮我吗? (Dean Edward的包装不够,而且没有gzip)

我正在寻找能够利用所有允许的javascript变量并寻找模式(有许多模式)的东西,类似于zip压缩的工作方式。

以下是我的数据库结构的示例(最终结果必须采用此格式):

var crops =  {
Jan1: {
    Apple: [1995, 1996, 1998, 2003, 2007, 2008, 2009],
    Banana: [1995, 1998, 2001, 2002, 2003, 2007, 2008, 2009],
    Cantaloupe: [2000, 2003, 2004, 2005, 2006, 2007],
    Date: [1995, 1998, 2001, 2002, 2003, 2004, 2007],
    Elephant: [1995, 2002, 2004, 2007, 2009],
    Fig: [1996, 2003, 2004, 2005, 2006, 2007],
    Grape: [1999, 2000, 2001, 2004, 2006, 2008],
    Honeydew: [2001, 2005, 2008],
    Ita: [1997, 2000, 2001, 2002, 2004, 2005],
    Jasmine: [1997, 1998, 2002, 2004, 2005, 2007, 2008],
    Kiwi: [1997, 1998, 2003],
    Lemon: [1995, 1997, 2001, 2003, 2004, 2007, 2009],
    Melon: [1995, 1997, 2000, 2003, 2005],
    Nectarine: [1996, 1998, 2000, 2002, 2007],
    Orange: [1996, 1998, 2000, 2003, 2007],
    Pineapple: [1996, 1997, 1999, 2000, 2002, 2003, 2005, 2007, 2008, 2009],
    Quince: [1995, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2007, 2008],
    Raspberry: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2007],
    Strawberry: [2002, 2003, 2006, 2008],
    Tangerine: [1995, 1997, 1998, 1999, 2006, 2008],
    Ugli: [1997, 2001, 2002, 2006, 2009],
    Vanilla: [1998, 1999, 2001, 2002, 2004, 2005],
    Watermelon: [1997, 1998, 2001, 2002, 2003, 2005],
    Xigua: [1996, 2001, 2002, 2003, 2004, 2005, 2006],
    Yali: [1997, 1998, 1999, 2000, 2001, 2002, 2005, 2006, 2009],
    Zucchini: [2001, 2002]
},
Jan2: {
    Banana: [2005, 2009],
    Cantaloupe: [2006],
    Grape: [1997, 2005, 2006, 2007],
    Honeydew: [1999],
    Ita: [2001, 2002],
    Jasmine: [1999],
    Kiwi: [2008, 2009],
    Nectarine: [2001],
    Orange: [1997, 2008],
    Quince: [2005],
    Raspberry: [2002],
    Strawberry: [2004],
    Vanilla: [1995, 1999],
    Watermelon: [1997, 2002],
    Xigua: [2006, 2007],
    Yali: [2001, 2005],
    Zucchini: [2004]
},
Jan3: {
    Apple: [1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2009],
    Banana: [1997, 1998, 2001, 2006],
    Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2006, 2008],
    Date: [1995, 1998, 1999, 2000, 2001, 2005, 2007, 2008, 2009],
    Elephant: [1996, 1997, 1998, 2000, 2002, 2003, 2004, 2005, 2006, 2007],
    Fig: [1998, 1999, 2000, 2002, 2003, 2009],
    Grape: [1997, 1998, 2002, 2005, 2006, 2007],
    Honeydew: [1995, 1996, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008],
    Ita: [1995, 1996, 2000, 2001, 2004, 2005, 2008, 2009],
    Jasmine: [1995, 1999, 2002, 2007, 2008],
    Kiwi: [1996, 2003, 2006, 2008, 2009],
    Lemon: [1995, 1996, 1999, 2002, 2006, 2007, 2009],
    Melon: [1997, 1999, 2002, 2005, 2008],
    Nectarine: [1998, 2000, 2002, 2004, 2005, 2008],
    Orange: [1995, 1997, 2000, 2002, 2005, 2007, 2009],
    Pineapple: [1998, 2001, 2002, 2004, 2005, 2007, 2009],
    Quince: [1995, 1997, 2000, 2002, 2004, 2005, 2006, 2007, 2009],
    Raspberry: [1995, 1996, 1997, 1998, 2002, 2004, 2007, 2008],
    Strawberry: [1996, 1997, 1999, 2000, 2006],
    Tangerine: [1999, 2001, 2004, 2007, 2008],
    Ugli: [1995, 1997, 1998, 1999, 2000, 2006, 2007, 2009],
    Vanilla: [1996, 1997, 2003, 2006, 2008],
    Watermelon: [1995, 1997, 1999, 2002, 2003, 2005],
    Xigua: [1996, 1997, 1999, 2001, 2004, 2006, 2008, 2009],
    Yali: [1996, 1998, 2001, 2003, 2004, 2005, 2007, 2008],
    Zucchini: [1997, 2003, 2006, 2007, 2009]
},
Jan4: {
    Apple: [2000, 2002, 2003],
    Cantaloupe: [2009],
    Elephant: [2003],
    Fig: [2001, 2008],
    Honeydew: [2001],
    Pineapple: [2004],
    Quince: [1999],
    Ugli: [2009],
    Vanilla: [2005],
    Yali: [2007]
},
Feb1: {
    Apple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2008, 2009],
    Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
    Elephant: [1996, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Grape: [1995, 1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Honeydew: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Ita: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Melon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Nectarine: [1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Orange: [1995, 1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008],
    Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2009],
    Tangerine: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
    Ugli: [1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Vanilla: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009],
    Watermelon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2006, 2007, 2009],
    Xigua: [1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008, 2009],
    Zucchini: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009]
},
Feb2: {
    Apple: [1995, 1996, 1998, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2008, 2009],
    Cantaloupe: [1995, 1996, 1997, 1998, 1999, 2001, 2004, 2005, 2006, 2007],
    Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Elephant: [1996, 1997, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Fig: [1995, 1996, 1997, 1998, 1999, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Grape: [1995, 1996, 1998, 1999, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Honeydew: [1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Ita: [1995, 1997, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008],
    Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2006, 2007, 2009],
    Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008],
    Melon: [1995, 1996, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Nectarine: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Orange: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Quince: [1995, 1996, 1997, 1999, 2000, 2001, 2002, 2003, 2006, 2007, 2008, 2009],
    Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Tangerine: [1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2008, 2009],
    Ugli: [1995, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2008, 2009],
    Vanilla: [1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2008],
    Watermelon: [1997, 1999, 2000, 2001, 2002, 2003, 2004, 2007, 2009],
    Xigua: [1995, 1996, 1997, 1999, 2000, 2001, 2002, 2005, 2006, 2007, 2008, 2009],
    Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2007, 2008, 2009],
    Zucchini: [1995, 1997, 1998, 1999, 2001, 2003, 2006, 2007, 2008, 2009]
},
Feb3: {
    Apple: [1996, 1997, 2003, 2004],
    Banana: [1998],
    Cantaloupe: [1996, 1997, 1999, 2000],
    Date: [1996, 1997, 2006],
    Elephant: [2004],
    Fig: [1995, 1996, 2001, 2006, 2008],
    Grape: [2003],
    Honeydew: [2004],
    Ita: [1997, 2009],
    Jasmine: [1999, 2004, 2006, 2008],
    Kiwi: [1999, 2002, 2003, 2006],
    Lemon: [1995, 1997, 2004],
    Melon: [2008],
    Nectarine: [1995, 1998, 2005],
    Orange: [2003, 2009],
    Pineapple: [1995, 2001],
    Quince: [1995, 1996, 2000, 2004, 2005, 2006],
    Raspberry: [1996, 1999, 2000, 2003, 2009],
    Strawberry: [1995, 1998, 2001, 2004, 2008],
    Tangerine: [1996, 1998, 2002],
    Ugli: [2008],
    Vanilla: [1996, 2007, 2009],
    Watermelon: [2006, 2008],
    Xigua: [1998, 1999, 2002, 2005, 2009],
    Yali: [1996, 2000, 2001, 2002, 2004],
    Zucchini: [1998, 2000, 2003, 2007]
},
Feb4: {
    Apple: [1995, 1996, 1997, 1998, 2000, 2002, 2003, 2006, 2009],
    Banana: [1995, 1996, 1997, 2000, 2002, 2004, 2005, 2007, 2008, 2009],
    Cantaloupe: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008],
    Date: [1995, 1997, 1998, 2000, 2001, 2002, 2004, 2005, 2008, 2009],
    Elephant: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2006, 2007, 2008, 2009],
    Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2007, 2008, 2009],
    Grape: [1995, 1996, 1997, 1998, 2003, 2005, 2006, 2007, 2008],
    Honeydew: [1995, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Ita: [1995, 1997, 1999, 2000, 2001, 2006, 2007, 2008, 2009],
    Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009],
    Kiwi: [1995, 1996, 1997, 1999, 2000, 2002, 2003, 2004, 2005, 2007, 2008],
    Lemon: [1995, 1996, 1998, 1999, 2000, 2001, 2003, 2005, 2006, 2007, 2008, 2009],
    Melon: [1995, 1996, 1998, 2000, 2002, 2003, 2005, 2006, 2007, 2008],
    Nectarine: [1996, 1997, 1998, 2000, 2003, 2004, 2006, 2007, 2008, 2009],
    Orange: [1996, 1998, 1999, 2000, 2001, 2002, 2003, 2006, 2008],
    Pineapple: [1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009],
    Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2002, 2004, 2005, 2007, 2009],
    Raspberry: [1997, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
    Strawberry: [1995, 1996, 1997, 1998, 2000, 2001, 2004, 2007, 2008],
    Tangerine: [1997, 1998, 1999, 2000, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
    Ugli: [1995, 1996, 1997, 1998, 1999, 2001, 2003, 2007, 2008, 2009],
    Vanilla: [1997, 1998, 1999, 2000, 2001, 2002, 2003, 2005, 2006, 2007, 2008, 2009],
    Watermelon: [1995, 1996, 1997, 1998, 1999, 2001, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Xigua: [1996, 1997, 1998, 2000, 2003, 2005, 2007, 2008, 2009],
    Yali: [1995, 1997, 1998, 2005, 2007, 2008],
    Zucchini: [1995, 1996, 1997, 2000, 2003, 2005, 2006, 2007, 2008, 2009]
},
Mar1: {
    Apple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Banana: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Cantaloupe: [1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Date: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Elephant: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Fig: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Grape: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Honeydew: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Ita: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Jasmine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Kiwi: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Lemon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Melon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Nectarine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2004, 2005, 2006, 2007, 2008, 2009],
    Orange: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Pineapple: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Quince: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Raspberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008],
    Strawberry: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Tangerine: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Ugli: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Vanilla: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Watermelon: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Xigua: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Yali: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009],
    Zucchini: [1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009]
}
/* Much much more */
}

5 个答案:

答案 0 :(得分:1)

您需要压缩数据。你有12个月,(比如说)20个水果,以及可能有下限的一系列年份。存储数据如下:

{[1,[1,[1,2,4]],[2,[4,6,7]]],[2,[1,[4,5,6]]], ... }

每个数组元素都是:

[month, [fruit, [year1, year2, year3,...]]]

如果你需要以长篇形式处理它,你可以通过参考轻松转换它。

1 = Apple,2 =香蕉,......

1 = 1月,2 = 2月,...

1 = 1995年,2 = 1996年,......

答案 1 :(得分:0)

您可以将水果,日和年的每个组合存储为一个位。 如果你让Apple为1,Banana bit 2等等你每天得到26位。您可以在32位整数中存储26位。 所以每年需要365个整数(闰年除外366个)或1460个字节。

这将是您的数据存储。然后你可以在javascript中解压缩个别(或所有)天。

答案 2 :(得分:0)

如果程序是通过线路传输太多数据,那么这将有所帮助:从年份中减去标准值并删除任何不必要的空格。在这里,我从数字中减去了2000。

// Before.
Jan1: {
    Apple: [1995, 1996, 1998, 2003, 2007, 2008, 2009],

// After.
Jan1:{Apple:[-5,-4,-2,3,7,8,9],

您可以使用inflate / gzip压缩数据。大多数浏览器都支持此功能。设置取决于您的Web服务器。另外:我不确定这是否受到AJAX所依赖的对象XmlHttpRequest的支持。

答案 3 :(得分:0)

Alexei White在online JS compression上发布了博客文章(以及概念验证)(如果在客户端缓存,则会使JS足迹变小)。然而,他的解决方案仍然很慢,使用尖端的Chrome。我尝试了你的样本数据,它锁定浏览器约5秒钟。

答案 4 :(得分:0)

为什么必须使用Javascript?难道你不能把它放在真实数据库的服务器上,然后通过服务或OData公开数据,以便页面可以查询它吗?