我正在构建一个脱机词典应用程序。具有数据库文件150,000行。我已经使用Sqlite,但是我需要保护数据库的安全,因此我使用SqlCipher库进行加密。加密后我遇到的问题是加密后无法压缩读取数据和SqlCipher文件的速度。数据库文件大小大大增加。
Sqlite(9Mb)-> SqlCipher(93MB)。
我也尝试使用Realm数据库。阅读速度快,安全性好。但是,数据库文件的大小也显着增加。
Sqlite(9Mb)->领域(50MB)。
有什么办法可以减少数据库的大小? android有另一个数据库可以克服上述缺点
(安全性,速度,数据库大小)?
答案 0 :(得分:2)
您也许可以考虑实施自己的加密/解密,然后仅对实际敏感数据进行部分加密。
例如,下面的演示代码使用了具有280000个定义(尽管定义重复)的基本字典(单词/定义)。不加密时占用20.9mB,加密时占用36.6mB。
mydbenc加密版本
再一次,实际上存储的数据(例如,较长的单词定义)可能会产生很大的影响,因为单词和定义的数量相同,但定义明显更长(在14个单词之一的情况下)定义(每次重复20000次),然后加密数据库的大小增加4mB,加密数据库的大小也增加约4mB。)(演示使用较大的DB)
因此,对于您的150,000行,数据库的加密大小约为20Mb。
大小也会受到加密方法的影响。通常,加密方法越弱,开销越少,但是安全性较低。
为克服搜索问题,示例应用程序在启动时将其解密为临时表中的数据。这确实需要不到一分钟的时间,这可能是无法接受的。
该示例代码由两个数据库助手组成,一个用于比较的未加密版本,另一个是加密版本。两者都使用具有3列 id (未加密),单词和定义的同一表,后两者在加密版本中进行了加密
数据库,表和名称列通过名为DBConstants的类中的常量按如下方式定义:-
public class DBConstants {
public static int FILTEROPTION_ANYWHERE = 0;
public static int FILTEROPTION_MUSTMATCH = 1;
public static int FILTEROPTION_STARTSWITH = 2;
public static int FILTEROPTION_ENDSWITH = 4;
public static final String DBName = "mydb";
public static final int DBVERSION = 1;
public static final String DECRYPTEXTENSION = "_decrypt";
public static class MainTable {
public static final String TBLNAME = "main";
public static final String COL_ID = BaseColumns._ID;
public static final String COl_WORD = "_word";
public static final String COL_DEFINITION = "_definition";
public static final String CRT_SQL = "CREATE TABLE IF NOT EXISTS " + TBLNAME +
"(" +
COL_ID + " INTEGER PRIMARY KEY," +
COl_WORD + " TEXT," +
COL_DEFINITION + " TEXT" +
")";
}
public static class DecrtyptedMainTable {
public static final String TBLNAME = MainTable.TBLNAME + DECRYPTEXTENSION;
public static final String CRT_SQL = "CREATE TEMP TABLE IF NOT EXISTS " + TBLNAME +
"(" +
MainTable.COL_ID + " INTEGER PRIMARY KEY," +
MainTable.COl_WORD + " TEXT, " +
MainTable.COL_DEFINITION + " TEXT " +
")";
public static final String CRTIDX_SQL = "CREATE INDEX IF NOT EXISTS " +
TBLNAME + "_index " +
" ON " + TBLNAME +
"(" + MainTable.COl_WORD + ")";
}
}
Word 类用于允许按以下方式提取 Word 对象:-
public class Word {
private long id;
private String word;
private String definition;
public Word() {
this.id = -1L;
}
public Word(String word, String definition) {
this(-1L,word,definition);
}
public Word(Long id, String word, String definition) {
this.id = id;
this.word = word;
this.definition = definition;
}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
public String getWord() {
return word;
}
public void setWord(String word) {
this.word = word;
}
public String getDefinition() {
return definition;
}
public void setDefinition(String definition) {
this.definition = definition;
}
}
用于非加密数据库的数据库帮助器 DBHelperStandard.java (仅出于比较目的而存在)是:-
public class DBHelperStandard extends SQLiteOpenHelper {
SQLiteDatabase db;
public DBHelperStandard(Context context) {
super(context, DBConstants.DBName, null, DBConstants.DBVERSION);
db = this.getWritableDatabase();
}
@Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(DBConstants.MainTable.CRT_SQL);
}
@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
public long insertWord(String word, String definition) {
ContentValues cv = new ContentValues();
cv.put(DBConstants.MainTable.COl_WORD,word);
cv.put(DBConstants.MainTable.COL_DEFINITION,definition);
return db.insert(DBConstants.MainTable.TBLNAME,null,cv);
}
public long insertWord(Word word) {
return insertWord(word.getWord(),word.getDefinition());
}
public int deleteWord(long id) {
String whereclause = DBConstants.MainTable.COL_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
return db.delete(DBConstants.MainTable.TBLNAME,whereclause,whereargs);
}
public int deleteWord(Word word) {
return deleteWord(word.getId());
}
public int updateWord(long id, String word, String defintion) {
ContentValues cv = new ContentValues();
if (word != null && word.length() > 0) {
cv.put(DBConstants.MainTable.COl_WORD,word);
}
if (defintion != null && defintion.length() > 0) {
cv.put(DBConstants.MainTable.COL_DEFINITION,defintion);
}
if (cv.size() < 1) return 0;
String whereclause = DBConstants.MainTable.COL_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
return db.update(DBConstants.MainTable.TBLNAME,cv,whereclause,whereargs);
}
public int updateWord(Word word) {
return updateWord(word.getId(),word.getWord(),word.getDefinition());
}
public List<Word> getWords(String wordfilter, int filterOption, Integer limit) {
ArrayList<Word> rv = new ArrayList<>();
String whereclause = DBConstants.MainTable.COl_WORD + " LIKE ?";
StringBuilder sb = new StringBuilder();
switch (filterOption) {
case 0:
sb.append("%").append(wordfilter).append("%");
break;
case 1:
sb.append(wordfilter);
break;
case 2:
sb.append(wordfilter).append("%");
break;
case 4:
sb.append("%").append(wordfilter);
}
String[] whereargs = new String[]{sb.toString()};
if (wordfilter == null) {
whereclause = null;
whereargs = null;
}
String limitclause = null;
if (limit != null) {
limitclause = String.valueOf(limit);
}
Cursor csr = db.query(
DBConstants.MainTable.TBLNAME,
null,
whereclause,
whereargs,
null,
null,
DBConstants.MainTable.COl_WORD,
limitclause
);
while (csr.moveToNext()) {
rv.add(new Word(
csr.getLong(csr.getColumnIndex(DBConstants.MainTable.COL_ID)),
csr.getString(csr.getColumnIndex(DBConstants.MainTable.COl_WORD)),
csr.getString(csr.getColumnIndex(DBConstants.MainTable.COL_DEFINITION))
));
}
return rv;
}
}
加密数据库的数据库帮助器 DBHelperEncrypted.java 是:-
public class DBHelperEncrypted extends SQLiteOpenHelper {
private String secretKey;
private String ivpParemeter;
SQLiteDatabase db;
public DBHelperEncrypted(Context context, String secretKey, String ivpParamter) {
super(context, DBConstants.DBName + DBConstants.DECRYPTEXTENSION, null, DBConstants.DBVERSION);
this.secretKey = secretKey;
this.ivpParemeter = ivpParamter;
db = this.getWritableDatabase();
}
@Override
public void onCreate(SQLiteDatabase db) { db.execSQL(DBConstants.MainTable.CRT_SQL); }
@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
public long insertWord(String word, String definition) {
ContentValues cv = new ContentValues();
cv.put(DBConstants.MainTable.COl_WORD,EncryptDecrypt.encrypt(word,secretKey,ivpParemeter));
cv.put(DBConstants.MainTable.COL_DEFINITION,EncryptDecrypt.encrypt(definition,secretKey,ivpParemeter));
return db.insert(DBConstants.MainTable.TBLNAME,null,cv);
}
public long insertWord(Word word) {
return insertWord(word.getWord(),word.getDefinition());
}
public int deleteWord(long id) {
String whereclause = DBConstants.MainTable.COL_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
return db.delete(DBConstants.MainTable.TBLNAME,whereclause,whereargs);
}
public int deleteWord(Word word) {
return deleteWord(word.getId());
}
public int updateWord(long id, String word, String defintion) {
ContentValues cv = new ContentValues();
if (word != null && word.length() > 0) {
cv.put(DBConstants.MainTable.COl_WORD,EncryptDecrypt.encrypt(word,secretKey,ivpParemeter));
}
if (defintion != null && defintion.length() > 0) {
cv.put(DBConstants.MainTable.COL_DEFINITION,EncryptDecrypt.encrypt(defintion,secretKey,ivpParemeter));
}
if (cv.size() < 1) return 0;
String whereclause = DBConstants.MainTable.COL_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
return db.update(DBConstants.MainTable.TBLNAME,cv,whereclause,whereargs);
}
public int updateWord(Word word) {
return updateWord(word.getId(),word.getWord(),word.getDefinition());
}
public List<Word> getWords(String wordfilter, int filterOption, Integer limit) {
ArrayList<Word> rv = new ArrayList<>();
String whereclause = DBConstants.MainTable.COl_WORD + " LIKE ?";
StringBuilder sb = new StringBuilder();
switch (filterOption) {
case 0:
sb.append("%").append(wordfilter).append("%");
break;
case 1:
sb.append(wordfilter);
break;
case 2:
sb.append(wordfilter).append("%");
break;
case 4:
sb.append("%").append(wordfilter);
}
String[] whereargs = new String[]{sb.toString()};
String limitclause = null;
if (limit != null) {
limitclause = String.valueOf(limit);
}
Cursor csr = db.query(
DBConstants.DecrtyptedMainTable.TBLNAME,
null,
whereclause,
whereargs,
null,
null,
DBConstants.MainTable.COl_WORD,
limitclause
);
while (csr.moveToNext()) {
rv.add(
new Word(
csr.getLong(csr.getColumnIndex(DBConstants.MainTable.COL_ID)),
csr.getString(csr.getColumnIndex(DBConstants.MainTable.COl_WORD)),
csr.getString(csr.getColumnIndex(DBConstants.MainTable.COL_DEFINITION))
)
);
}
return rv;
}
public void buildDecrypted(boolean create_index) {
db.execSQL(DBConstants.DecrtyptedMainTable.CRT_SQL);
Cursor csr = db.query(DBConstants.MainTable.TBLNAME,null,null,null,null,null,null);
ContentValues cv = new ContentValues();
while (csr.moveToNext()) {
cv.clear();
cv.put(DBConstants.MainTable.COL_ID,csr.getLong(csr.getColumnIndex(DBConstants.MainTable.COL_ID)));
cv.put(DBConstants.MainTable.COl_WORD,
EncryptDecrypt.decrypt(csr.getString(csr.getColumnIndex(DBConstants.MainTable.COl_WORD)),secretKey,ivpParemeter));
cv.put(DBConstants.MainTable.COL_DEFINITION,
EncryptDecrypt.decrypt(csr.getString(csr.getColumnIndex(DBConstants.MainTable.COL_DEFINITION)),secretKey,ivpParemeter));
db.insert(DBConstants.DecrtyptedMainTable.TBLNAME,null,cv);
}
csr.close();
if (create_index) {
db.execSQL(DBConstants.DecrtyptedMainTable.CRTIDX_SQL);
}
}
}
主要区别在于(可能不被使用,因为看起来好像数据库已经出厂了)插入和更新方法对数据进行加密,并且还包含方法 buildDecrypted 从加密表中创建一个临时表。用于搜索和提取数据的临时表。
加密和解密由EncryotDecrypt类按照 EncryptDecrypt.java 处理:-
public class EncryptDecrypt {
public static Cipher cipher;
/**
* Encryption, irrespective of the USER type, noting that this should
* only be used in conjunction with an EncryptDecrypt instance created
* using the 2nd/extended constructor
*
* @param toEncrypt The string to be encrypted
* @return The encrypted data as a string
*/
public static String encrypt(String toEncrypt, String secretKey, String ivParameterSpec) {
byte[] encrypted;
try {
if (cipher == null) {
cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
}
if (secretKey.length() < 16) {
secretKey = (secretKey + " ").substring(0,16);
}
SecretKeySpec secretKeySpec = new SecretKeySpec(secretKey.getBytes(),"AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE,secretKeySpec,new IvParameterSpec(ivParameterSpec.getBytes()));
encrypted = cipher.doFinal(toEncrypt.getBytes());
} catch (Exception e) {
e.printStackTrace();
return null;
}
return Base64.encodeToString(encrypted,Base64.DEFAULT);
}
/**
* Decrypt an encrypted string
* @param toDecrypt The encrypted string to be decrypted
* @return The decrypted string
*/
public static String decrypt(String toDecrypt, String secretKey, String ivParameterSpec) {
byte[] decrypted;
try {
if (cipher == null) {
cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
}
if (secretKey.length() < 16) {
secretKey = (secretKey + " ").substring(0,16);
}
SecretKeySpec secretKeySpec = new SecretKeySpec(secretKey.getBytes(),"AES/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE,secretKeySpec,new IvParameterSpec(ivParameterSpec.getBytes()));
decrypted = cipher.doFinal(Base64.decode(toDecrypt,Base64.DEFAULT));
} catch (Exception e) {
e.printStackTrace();
return null;
}
return new String(decrypted);
}
}
最后,将本演示放在一起,是 MainActivity.java :-
public class MainActivity extends AppCompatActivity {
public static final String SK = "mysecretkey";
public static final String SALT = "124567890ABCDEFG";
DBHelperEncrypted mDBE;
DBHelperStandard mDBS;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mDBE = new DBHelperEncrypted(this,SK,SALT);
mDBS = new DBHelperStandard(this);
//Stage 1 - Build the demo databases
ArrayList<Word> wordsanddefinitions = new ArrayList<>();
for (int i=0; i < 20000; i ++) {
wordsanddefinitions.add(new Word("Apple","Something that falls on peoples heads that causes them to discover gravity."));
wordsanddefinitions.add(new Word("Bachelor","An unmarried man."));
wordsanddefinitions.add(new Word("Bachelor","A person who has been awarded a bachelor's degree."));
wordsanddefinitions.add(new Word("Bachelor","A fur seal, especially a young male, kept from the breeding grounds by the older males."));
wordsanddefinitions.add(new Word("Cat","A small domesticated carnivore, Felis domestica or F. catus, bred in a number of varieties."));
wordsanddefinitions.add(new Word("Dog","A domesticated canid, Canis familiaris, bred in many varieties."));
wordsanddefinitions.add(new Word("Eddy","A current at variance with the main current in a stream of liquid or gas, especially one having a rotary or whirling motion."));
wordsanddefinitions.add(new Word("Eddy","A small whirlpool."));
wordsanddefinitions.add(new Word("Eddy","Any similar current, as of air, dust, or fog."));
wordsanddefinitions.add(new Word("Eddy","A current or trend, as of opinion or events, running counter to the main current."));
wordsanddefinitions.add(new Word("Orange","A colour bewteen Red and Yellow."));
wordsanddefinitions.add(new Word("Orange","a globose, reddish-yellow, bitter or sweet, edible citrus fruit."));
wordsanddefinitions.add(new Word("Orange","any white-flowered, evergreen citrus trees of the genus Citrus, bearing this fruit, " +
"as C. aurantium (bitter orange, Seville orange, or sour orange) " +
"and C. sinensis (sweet orange), cultivated in warm countries."));
wordsanddefinitions.add(new Word("Orange","Any of several other citrus trees, as the trifoliate orange."));
}
Log.d("STAGE1","Starting to build the Standard (non-encrypted) DB with " + String.valueOf(wordsanddefinitions.size()) + " definitions");
mDBS.getWritableDatabase().beginTransaction();
for (Word w: wordsanddefinitions ) {
mDBS.insertWord(w);
}
mDBS.getWritableDatabase().setTransactionSuccessful();
mDBS.getWritableDatabase().endTransaction();
Log.d("STAGE2","Starting to build the Encrypted DB with " + String.valueOf(wordsanddefinitions.size()) + " definitions");
mDBE.getWritableDatabase().beginTransaction();
for (Word w: wordsanddefinitions) {
mDBE.insertWord(w);
}
// Decrypt the encrypted table as a TEMPORARY table
Log.d("STAGE 3","Bulding the temporary unencrypted table");
mDBE.buildDecrypted(true); // Build with index on word column
mDBE.getWritableDatabase().setTransactionSuccessful();
mDBE.getWritableDatabase().endTransaction();
// Database now usable
Log.d("STAGE4","Extracting data (all words that include ap in the word) from the Standard DB");
List<Word> extracted_s = mDBS.getWords("ap",DBConstants.FILTEROPTION_ANYWHERE,10);
for (Word w: extracted_s) {
Log.d("WORD_STANDARD",w.getWord() + " " + w.getDefinition());
}
Log.d("STAGE5","Extracting data (all words that include ap in the word) from the Encrypted DB");
List<Word> extracted_e = mDBE.getWords("ap",DBConstants.FILTEROPTION_ANYWHERE,10);
for (Word w: extracted_e) {
Log.d("WORD_ENCRYPTED",w.getWord() + " " + w.getDefinition());
}
Log.d("STAGE5","Extracting demo data from standard and from encrypted without decryption");
Cursor csr = mDBE.getWritableDatabase().query(DBConstants.MainTable.TBLNAME,null,null,null,null,null,null,"10");
DatabaseUtils.dumpCursor(csr);
csr = mDBS.getWritableDatabase().query(DBConstants.MainTable.TBLNAME,null,null,null,null,null,null,"10");
DatabaseUtils.dumpCursor(csr);
mDBS.close();
mDBE.close();
}
}
此:-
(第2阶段)对加密的数据库使用insert方法(用于加密数据)。
(第3阶段)构建临时的解密表以及索引(可以使用mDBE.buildDecrypted(false)跳过索引; (它似乎没有太多imapct,尤其是它是在插入内容之后构建的))。
(阶段4)使用 getWords 方法(已过滤)从未加密的数据库中提取一些数据,并将提取的数据写入日志。
(第5阶段)使用 getWords 方法(已过滤)从加密的数据库(从解密的临时表)中提取一些数据,并将提取的数据写入日志。
从加密数据库的持久表(即加密数据,而不是解密数据)中提取前10行(为简便起见,显示了2行)。
从未加密的数据库中提取前10行(通过berevity显示2行),并将Cursor转储到日志中。
6-05 13:51:36.932 D/STAGE1: Starting to build the Standard (non-encrypted) DB with 280000 definitions
06-05 13:51:59.274 D/STAGE2: Starting to build the Encrypted DB with 280000 definitions
06-05 13:52:45.327 D/STAGE 3: Bulding the temporary unencrypted table
06-05 13:52:45.350 W/CursorWindow: Window is full: requested allocation 111 bytes, free space 98 bytes, window size 2097152 bytes
.........
06-05 13:53:35.024 D/STAGE4: Extracting data (all words that include ap in the word) from the Standard DB
06-05 13:53:35.346 D/WORD_STANDARD: Apple Something that falls on peoples heads that causes them to discover gravity.
..........
06-05 13:53:35.346 D/STAGE5: Extracting data (all words that include ap in the word) from the Encrypted DB
06-05 13:53:35.346 D/WORD_ENCRYPTED: Apple Something that falls on peoples heads that causes them to discover gravity.
..........
06-05 13:53:35.347 D/STAGE5: Extracting demo data from standard and from encrypted without decryption
06-05 13:53:35.347 I/System.out: >>>>> Dumping cursor android.database.sqlite.SQLiteCursor@d05c965
06-05 13:53:35.347 I/System.out: 0 {
06-05 13:53:35.347 I/System.out: _id=1
06-05 13:53:35.347 I/System.out: _word=3mqQlZl55WNjeZhALFQU7w==
06-05 13:53:35.347 I/System.out: _definition=s9Waa2HLUS2fy8q1uC9/MEKogmImu6m9MIpi9wasD9D3Zom6+/u40DnFfP6zXOyI8IgnQOKcWfQ8
06-05 13:53:35.347 I/System.out: G3uJN9a/YHMoQdEQMDMEEdSE2kWyJrc=
06-05 13:53:35.347 I/System.out: }
06-05 13:53:35.347 I/System.out: 1 {
06-05 13:53:35.347 I/System.out: _id=2
06-05 13:53:35.347 I/System.out: _word=LtLlycoBd9fm3eYF9aoItg==
06-05 13:53:35.347 I/System.out: _definition=B1XJJm0eC8wPi3xGg4XgJtvIS3xL7bjixNhVAVq1UwQ=
06-05 13:53:35.347 I/System.out: }
06-05 13:53:35.348 I/System.out: >>>>> Dumping cursor android.database.sqlite.SQLiteCursor@7f1b63a
06-05 13:53:35.348 I/System.out: 0 {
06-05 13:53:35.348 I/System.out: _id=1
06-05 13:53:35.348 I/System.out: _word=Apple
06-05 13:53:35.348 I/System.out: _definition=Something that falls on peoples heads that causes them to discover gravity.
06-05 13:53:35.348 I/System.out: }
06-05 13:53:35.348 I/System.out: 1 {
06-05 13:53:35.348 I/System.out: _id=2
06-05 13:53:35.348 I/System.out: _word=Bachelor
06-05 13:53:35.348 I/System.out: _definition=An unmarried man.
06-05 13:53:35.348 I/System.out: }
06-05 13:53:35.349 I/System.out: <<<<<
答案 1 :(得分:0)
您尝试使用VACUUM吗?
VACUUM命令重建整个数据库。应用程序可能会这样做的原因有几个:
除非SQLite在较大的情况下以“ auto_vacuum = FULL”模式运行 大量数据从其留下的数据库文件中删除 空白空间或“空闲”数据库页面。这意味着数据库文件 可能比严格必要的要大。运行VACUUM以重建 数据库回收此空间并减小数据库的大小 文件。
频繁的插入,更新和删除会导致数据库 文件变得碎片化-单个表或索引的数据在其中 分散在数据库文件周围。运行VACUUM可确保每个 表和索引大部分连续地存储在数据库中 文件。在某些情况下,VACUUM还可以减少部分 在数据库中填充页面,减少了数据库文件的大小 进一步...
VACUUM命令通过复制以下内容来工作: 数据库到临时数据库文件,然后覆盖 原始文件和临时文件的内容。覆盖时 原始的,仅使用回滚日志或预写日志WAL文件 就像处理其他任何数据库事务一样。这意味着 在对数据库进行真空处理时,其大小是原始数据库的两倍 可用磁盘空间中需要数据库文件...
修改
还有另一件事不要忘记使用正确的数据类型,例如TEXT字段比INTEGER字段需要更多的空间。