Question

以下代码位于异步任务中，该任务在解析csv文件并将其添加到sqlite数据库之前通过Internet获取csv文件。

如此，我需要在添加任何新行之前检查它们在数据库中是否已存在。

我的问题是，使用重复的检查代码，每个csv文件需要2.5分钟但没有检查每个文件需要6秒。

有关如何通过重复检查加快速度的任何线索，请？

  public class GetData extends AsyncTask<String, String, String> {

    @Override
    protected String doInBackground(String... params) {

        //for each csv file
        for (int x = 0; x < 11; x++) {
            get_config("sen" + x + "data.csv");
        }

        return null;
    }



    private void get_config(String siteurl) {

    StringBuilder total = new StringBuilder();

        String mfolder = Environment.getExternalStorageDirectory().getPath() + "/HG";
        SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(mcontext);
        String basesiteurl = prefs.getString("site_url", "");


        if (!DetectConnection.checkInternetConnection(mcontext)) {
            write_alertlog("AlertService: Fetch csv data - No internet connection");
        }else{

            write_alertlog("AlertService: Fetch csv data - Internet connection success");

            String onlineupdateline = "";
            try {

              Random randomGenerator = new Random();
              int randomInt = randomGenerator.nextInt(100); //defeat android cache

              DefaultHttpClient httpclient = new DefaultHttpClient();
              HttpGet httppost = new HttpGet(basesiteurl + "/csv/" + siteurl + "?unused=" + randomInt);
              HttpParams httpParameters = new BasicHttpParams();
              HttpConnectionParams.setConnectionTimeout(httpParameters, 8000);
              HttpConnectionParams.setSoTimeout(httpParameters, 8000);
              HttpResponse response = httpclient.execute(httppost);
              HttpEntity ht = response.getEntity();

              BufferedHttpEntity buf = new BufferedHttpEntity(ht);
              InputStream is = buf.getContent();
              BufferedReader r = new BufferedReader(new InputStreamReader(is));


              //reformat csv filename into sensor name for db
              String sensor_name = siteurl.replace("data.csv", "");

                database.beginTransaction();
                String line;
                while ((line = r.readLine()) != null) {
                    //total.append(line + "\n");

                    if (line.contains("<!DOCTYPE")){
                        write_applog("AlertService: Fetch csv data - csv file empty or does not exist: " + siteurl);
                        Log.d("test", "csv file empty or does not exist: " + siteurl);
                        database.endTransaction();
                        is.close();
                        r.close();
                        return;
                    }

                    List<String> singlecsvline = new ArrayList<String>(Arrays.asList(line.split(",")));

                    //csv date format
                    SimpleDateFormat fmt = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy");
                    Date pi_date = null;
                    try {
                        pi_date = fmt.parse(singlecsvline.get(0).toString().trim());
                    } catch (ParseException e) {
                        e.printStackTrace();
                    }

                    //database date format
                    fmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
                    String db_dateString = fmt.format(pi_date);
                    Date d = null;
                    try {
                        d = fmt.parse(db_dateString);
                    } catch (ParseException e) {
                        e.printStackTrace();
                    }
                    long db_dateLong = d.getTime();

                    //check if not in db already and if not add it
                    //this duplicate check makes processing each csv file go from 6 secs to 2.5 minutes
                    sql = "SELECT * FROM sensordata WHERE sensor_date='" + db_dateLong + "' AND sensor_id='" + sensor_name + "'" ;
                    Cursor findrawdata2 = database.rawQuery(sql, null);
                    if (!findrawdata2.moveToFirst()) {
                        database.execSQL("INSERT INTO sensordata (sensor_id, sensor_date, sensor_value, alerted, setflag) VALUES ('" + sensor_name + "', '" + db_dateLong + "', '" + Double.parseDouble(singlecsvline.get(1).toString().trim()) + "', '0', '" + setflag + "');");
                    }else{
                        //Log.d("test", "Not Adding - exists");
                    }
                    findrawdata2.close();

                }//for each line in csv file
                database.setTransactionSuccessful();
                database.endTransaction();

                is.close();
                r.close();
                write_applog("AlertService: Fetch csv data - added sensor " + siteurl);
                Log.d("test", "finsihed adding csv sensor " + siteurl);


            } catch (ConnectTimeoutException e) {
                write_alertlog("AlertService: Fetch csv data - Internet connection request timed out >6000ms");
            } catch (MalformedURLException e) {
                write_alertlog("AlertService: Fetch csv data - Error - " + e.toString());
            } catch (IOException e) {
                write_alertlog("AlertService: Fetch csv data - Error - " + e.toString());
            }

        }//have internet connection


        return;
    }

Answer 1

如果您只想检查它是否存在，实际上不需要从数据库中加载一行;使用像EXISTS或helper function：

这样的东西

long count = DatabaseUtils.queryNumEntries(database, "sensordata",
                "sensor_date = " + db_dateLong + " AND sensor_id = ?",
                new String[]{ sensor_name });
if (count > 0) {
    ...

然而，最大的减速来自搜索数据库中的行。您需要索引搜索列以获得快速查找：

CREATE INDEX insert_name_here ON sensordata(sensor_date, sensor_id);

查询重复项时，Android Sqlite插入速度

1 个答案: