Question

I have a mobile application where users fill and submit a form (all are text fields) with few photographs. The mobile app is also supporting offline mode.

The users submit the forms in their devices (Android, iOS mobile phones & Apple tab) and the data will be stored in the local db (SQLite).

When the user comes online, locally stored data will be sent to SQL Server 2014 via WebAPIs.

So we are writing two different WebAPIs.

To receive the FORM DATA (all text fields) in json format
To receive only photographs related to a FORM

There is a relationship exists between FORM DATA table and Photograph table in mobile db (SQLite) using GUID.

Once the WebAPIs receive data, we are saving in two STAGING_FORMDATA and STAGING_PHOTOGRAPH tables in SQL Server 2014. And then a success message will be sent to the device - "Data submitted successfully". Based on this acknowledgement, the mobile app can delete the data in its local db tables(SQLite). We don't want the devices to wait till whole data is processed.

Mobile App doesn't have to worry about the data validations on the server side. It just needs to submit data to server, it's job is over.

DBSchema for the tables is here. SQLite should also have the same structure (except RecordUploadedDate column).

FORMDATA

ID - GUID - PrimaryKey

Text1 - nvarchar(100)

Text2 - nvarchar(100)

RecordSubmittedDate - DateTime2

RecordUploadedDate  - DateTime2

PHOTOGRAPH

ID - GUID - PrimaryKey

FORMDATAID - GUID - ForiegnKey

PhotoBytes - varbinary(max)

RecordUploadedDate  - DateTime2

Here is the structure for STG tables.

STG_FORMDATA (No keys, all are nullable fields, to ingest all records from the devices)

ID - GUID

Text1 - nvarchar(100)

Text2 - nvarchar(100)

RecordSubmittedDate - DateTime2

RecordUploadedDate  - DateTime2

STG_PHOTOGRAPH

ID - GUID

STG_FORMDATA

FORMDATAID - GUID

PhotoBytes - varbinary(max)

RecordUploadedDate  - DateTime2

I should read the data from STAGING tables every 5 mins and do some processing and then save, successfully validated data in main tables. Few scenarios could be possible here.

Mobile has lost connection (gone offline immediately etc.) after sending the data to WebAPI layer (SQL Server DB) so acknowledgement not recieved by mobile app. So the same data will be sent again to the WebAPI layer (SQL Server DB) when when the device comes online again. So duplicate entries will be added in STG tables.
During my processing, few records may fail some validations, so those records must be marked as failed.

I thought about adding few flag columns - IsRecordProcessed, HasErrors, DuplicateEntry in the STAGING tables.

But I don't like (just) adding flags and then process the records in the staging tables.

In a bigger context, Iam lacking some thing with this approach. I should have a better design in place to cater future changes as well. Do we have any DB design architecture references to accomplish such tasks ?

Can anyone guide me here please ?

How to process data from staging tables to main tables in SQL Server 2014?

0 个答案: