我想通过在Spring Boot应用程序代码中使用SQL查询仅从mongodb中获取以前的日期数据
在mongo数据库中,日期存储在ISOFormat中。我只需要按日期=前一天的日期来过滤记录即可。
仅获取前一天的数据后,我仅将该数据存储在新数据集中。
代码中使用的简单查询是:
SELECT b._id as _id,a.msisdn as msisdn,
a.classification as classification, a.event_date_actual as
event_date_actual,(IFNULL(a.up_vol_mb,0) + IFNULL(b.up_vol_mb,0)) as
up_vol_mb,(IFNULL(a.down_vol_mb,0) + IFNULL(b.down_vol_mb,0)) as
down_vol_mb,(IFNULL(a.total_vol_mb,0) + IFNULL(b.total_vol_mb,0)) as
total_vol_mb FROM SlaveSchemaTable a LEFT JOIN MasterSchemaTable b ON
(a.msisdn = b.msisdn) AND
(a.classification = b.classification) AND
(a.event_date_actual = b.event_date_actual);
下面是代码:
private boolean writeMongoDBMaster(Dataset<Row> slaveSchemaTable,
SparkSession sparkSession) {
if(slaveSchemaTable.limit(1).count() !=0) {
StringBuilder joinBuilder = new StringBuilder("SELECT b._id as _id,");
joinBuilder.append("a.msisdn as msisdn,");
joinBuilder.append("a.classification as classification,");
joinBuilder.append("a.event_date_actual as event_date_actual,");
joinBuilder.append("(IFNULL(a.up_vol_mb,0) + IFNULL(b.up_vol_mb,0)) as
up_vol_mb,");
joinBuilder.append("(IFNULL(a.down_vol_mb,0) +
IFNULL(b.down_vol_mb,0)) as down_vol_mb,");
joinBuilder.append("(IFNULL(a.total_vol_mb,0) +
IFNULL(b.total_vol_mb,0)) as total_vol_mb ");
joinBuilder.append("FROM SlaveSchemaTable a LEFT JOIN
MasterSchemaTable b ");
joinBuilder.append("ON (a.msisdn = b.msisdn) AND ");
joinBuilder.append("(a.classification = b.classification) AND ");
joinBuilder.append("(a.event_date_actual = b.event_date_actual)");
Dataset<Row> masterTable = sparkSession.sql(joinBuilder.toString());
MongoSpark.save(masterTable.write().mode("append"));**
上面是一个简单的查询,它在适当的连接条件之后获取数据。我只想获取上一个日期的数据。
例如:select ---------------其中event_date_actual =前一天的日期。 event_date_actual为ISO格式。