大型SQL数据集的大数据方法

时间:2016-08-02 14:22:30

标签: performance postgresql amazon-web-services

我有一个表格,其中包含来自SaaS产品(我工作的公司)的用户活动,此表格为public class btnRunBackup { private String s; public void checkStatus() { String s, path = null; Process p; try{//try1 //create a temp file named tempfilexxx.tmp File temp = File.createTempFile("tempfile", ".tmp"); //get file path path = temp.getAbsolutePath(); System.out.println("checkStatus: " + path); //write to tempfilexxx.tmp BufferedWriter bw = new BufferedWriter(new FileWriter(temp)); try{// try2 //set p = to the content of ls home p = Runtime.getRuntime().exec("ls /home | grep ariel"); BufferedReader br = new BufferedReader( new InputStreamReader(p.getInputStream())); //write content of p to tempfilexxx.tmp line by line while ((s = br.readLine()) != null) bw.write(s + "\n"); //close BufferedReader br.close(); }catch (Exception e){} //END OF try2 //close BufferedWriter bw.close(); /* Now that we ran the 'mt -f /dev/nst0 status command under home we will filter for one of the following strings (for testing we will use ls -la /home and filter for ariel) We will do this by calling the checkStatus method */ this.statusControl(path); }catch(IOException e){ e.printStackTrace(); }// END OF try1 }// END OF listDir //throws FileNotFoundException for bufferedReader if file not found public void statusControl(String param) throws FileNotFoundException, IOException{ /* On production code there will be 4 possible conditions: 1. ONLINE - ready to write (currently we will use ariel) 2. DR_OPEN - no tape available 3. /dev/nst0: Device or resource busy - resource bussy 4. If other than stated above give error 1000 */ System.out.println("statusControl: " + param); String ONLINE = "arielvz", OPEN = "DR_OPEN", BUSSY = "Device or resource busy", sCurrentLine; //Scan file line by line for one of the above options BufferedReader br = new BufferedReader(new FileReader(param)); while ((sCurrentLine = br.readLine()) != null){ //Tape is online and ready for writing if (sCurrentLine.contains(ONLINE)){ System.out.println("found ariel"); } //There is no tape in the tape drive else if (sCurrentLine.contains(OPEN)){ //lblRunBck should tell the user to put a tape in the drive System.out.println("No tap in tape drive"); } else if (sCurrentLine.contains(BUSSY)){ //lblRunBck should notify user that the resource is in use System.out.println("Device or resource bussy"); } else{ //Something unexpected happend System.out.println("Error 1001: Please notify Administrator"); } } }//END OF statusControl public String returnHandler(String param){ return param; } } ,并且确实有大约4700行。 我制作的每份报告大约需要2小时才能运行,是否有relational SQL的任何方法可以帮助我改进它的执行时间?目前,我们正在Big Data托管PostgreSQL 9.5。我正在考虑关于AWS的一些事情但不知道这是否是最佳或更好的方式。

表格结构如下:

Amazon Redshift

0 个答案:

没有答案