将数据帧数据附加到json文件而不删除以前的内容

时间:2018-06-03 10:14:29

标签: python json pandas

首先让我说我是新手,这段代码可能很难看。 我正在尝试将数据帧数据附加到json文件,而不会在每次后续运行时删除以前的json数据内容。

import json
import pandas as pd
import datetime

json_backup = 'temp.json'
df_store = pd.DataFrame(columns=["Time", "Average Rate"])

while True:
   #doing some data gathering (code not included here) at each loop
   df_store = df_store.append({
             "Time": datetime.datetime.now(),
            "Average Rate": average_rate
             }, ignore_index=True)
  df_store.to_json(json_backup)

backup = pd.read_json(json_backup)
print (backup)

所以这可以正常工作,所有新数据都添加到json,直到我重新启动脚本并删除json数据。 我应该如何继续保存这些数据,并且所有新数据都只是附加到json文件?

2 个答案:

答案 0 :(得分:0)

如果'temp.json'文件存在,我认为您应该在while循环开始之前阅读df_store 'temp.json'变量。

答案 1 :(得分:0)

@Tamis ,请尝试以下代码Just copy the code(Python3), paste, run

  

注意:我已将 average_rate 的初始值作为 datetime.now()的微秒部分,仅用于测试目的代码。您可以根据您的逻辑计算其值,因此请更改它。

     

您无需创建任何 temp.json 文件。它将自动创建。

     

只需复制 append_data_to_json-v2-py3.py 文件的内容,然后粘贴然后运行。

     

temp.json 的每次更新后按 Y / y 继续,否则按任何其他键从while循环中退出。

»append_data_to_json-v2-py3.py

import json
import pandas as pd
import datetime
import time # {Added}

json_backup = 'temp.json'
df_store = pd.DataFrame(columns=["Time", "Average Rate"])

average_rate = 50  # {Added}
count = 1          # {Added}

while True:
    # Doing some data gathering (code not included here) at each loop
    time_data = str(datetime.datetime.now())

    # Store micro seconds of datetime as average 
    # (For test only, use your own logic to calculate it)
    # In case of 2018-06-03 18:44:56.220778 => 220778
    average_rate = int((time_data.split()[1]).split('.')[1]) 

    try:
        df_store = pd.read_json(json_backup)

        df_store = df_store.append({
            "Time": time_data,
            "Average Rate": average_rate
        }, ignore_index=True)

        df_store.to_json(json_backup)
        print (df_store)
        print ("***********************************************")       
    except Exception as e:
        df_store = df_store.append({
            "Time": time_data,
            "Average Rate": average_rate
            }, ignore_index=True)

        df_store.to_json(json_backup)
        print(df_store)
        print("***********************************************")

    # time.sleep(30.0 - ((time.time() - starttime) % 30.0))  # {Commented}
    print(count, "temp.json updated")

    # If user presses Y/y then continue otherwise exit from loop
    choice = input("\n" + str(count) + " Do you want to continue the operation (Y/N): ")
    if choice == 'y' or choice == 'Y':
        count = count + 1
        continue
    else:
        break

»首次运行

(py3.6) H:\RishikeshAgrawani\Projects\Python3\Pandas>python append_data_to_json-v2-py3.py
                         Time Average Rate
0  2018-06-03 18:51:57.506959       506959
***********************************************
1 temp.json updated

1 Do you want to continue the operation (Y/N): y
                         Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
***********************************************
2 temp.json updated

2 Do you want to continue the operation (Y/N): y
                         Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
***********************************************
3 temp.json updated

3 Do you want to continue the operation (Y/N): y
                         Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
***********************************************
4 temp.json updated

4 Do you want to continue the operation (Y/N): y
                         Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
***********************************************
5 temp.json updated

5 Do you want to continue the operation (Y/N): n

»第二次运行:

(py3.6) H:\RishikeshAgrawani\Projects\Python3\Pandas>python append_data_to_json-v2-py3.py
                       Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
5  2018-06-03 18:53:52.484954        484954
***********************************************
1 temp.json updated

1 Do you want to continue the operation (Y/N): y
                       Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
5  2018-06-03 18:53:52.484954        484954
6  2018-06-03 18:53:54.733274        733274
***********************************************
2 temp.json updated

2 Do you want to continue the operation (Y/N): y
                       Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
5  2018-06-03 18:53:52.484954        484954
6  2018-06-03 18:53:54.733274        733274
7  2018-06-03 18:53:57.037358         37358
***********************************************
3 temp.json updated

3 Do you want to continue the operation (Y/N): y
                       Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
5  2018-06-03 18:53:52.484954        484954
6  2018-06-03 18:53:54.733274        733274
7  2018-06-03 18:53:57.037358         37358
8  2018-06-03 18:53:58.437644        437644
***********************************************
4 temp.json updated

4 Do you want to continue the operation (Y/N): y
                       Time  Average Rate
0  2018-06-03 18:51:57.506959        506959
1  2018-06-03 18:52:00.925554        925554
2  2018-06-03 18:52:02.325613        325613
3  2018-06-03 18:52:03.508673        508673
4  2018-06-03 18:52:07.772553        772553
5  2018-06-03 18:53:52.484954        484954
6  2018-06-03 18:53:54.733274        733274
7  2018-06-03 18:53:57.037358         37358
8  2018-06-03 18:53:58.437644        437644
9  2018-06-03 18:53:59.181472        181472
***********************************************
5 temp.json updated

5 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:52:02.325613        325613
3   2018-06-03 18:52:03.508673        508673
4   2018-06-03 18:52:07.772553        772553
5   2018-06-03 18:53:52.484954        484954
6   2018-06-03 18:53:54.733274        733274
7   2018-06-03 18:53:57.037358         37358
8   2018-06-03 18:53:58.437644        437644
9   2018-06-03 18:53:59.181472        181472
10  2018-06-03 18:53:59.805276        805276
***********************************************
6 temp.json updated

6 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:53:59.805276        805276
3   2018-06-03 18:52:02.325613        325613
4   2018-06-03 18:52:03.508673        508673
5   2018-06-03 18:52:07.772553        772553
6   2018-06-03 18:53:52.484954        484954
7   2018-06-03 18:53:54.733274        733274
8   2018-06-03 18:53:57.037358         37358
9   2018-06-03 18:53:58.437644        437644
10  2018-06-03 18:53:59.181472        181472
11  2018-06-03 18:54:00.436774        436774
***********************************************
7 temp.json updated

7 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:53:59.181472        181472
3   2018-06-03 18:54:00.436774        436774
4   2018-06-03 18:53:59.805276        805276
5   2018-06-03 18:52:02.325613        325613
6   2018-06-03 18:52:03.508673        508673
7   2018-06-03 18:52:07.772553        772553
8   2018-06-03 18:53:52.484954        484954
9   2018-06-03 18:53:54.733274        733274
10  2018-06-03 18:53:57.037358         37358
11  2018-06-03 18:53:58.437644        437644
12  2018-06-03 18:54:00.997659        997659
***********************************************
8 temp.json updated

8 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:53:57.037358         37358
3   2018-06-03 18:53:58.437644        437644
4   2018-06-03 18:54:00.997659        997659
5   2018-06-03 18:53:59.181472        181472
6   2018-06-03 18:54:00.436774        436774
7   2018-06-03 18:53:59.805276        805276
8   2018-06-03 18:52:02.325613        325613
9   2018-06-03 18:52:03.508673        508673
10  2018-06-03 18:52:07.772553        772553
11  2018-06-03 18:53:52.484954        484954
12  2018-06-03 18:53:54.733274        733274
13  2018-06-03 18:54:01.549498        549498
***********************************************
9 temp.json updated

9 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:52:07.772553        772553
3   2018-06-03 18:53:52.484954        484954
4   2018-06-03 18:53:54.733274        733274
5   2018-06-03 18:54:01.549498        549498
6   2018-06-03 18:53:57.037358         37358
7   2018-06-03 18:53:58.437644        437644
8   2018-06-03 18:54:00.997659        997659
9   2018-06-03 18:53:59.181472        181472
10  2018-06-03 18:54:00.436774        436774
11  2018-06-03 18:53:59.805276        805276
12  2018-06-03 18:52:02.325613        325613
13  2018-06-03 18:52:03.508673        508673
14  2018-06-03 18:54:02.061568         61568
***********************************************
10 temp.json updated

10 Do you want to continue the operation (Y/N): y
                        Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:54:00.436774        436774
3   2018-06-03 18:53:59.805276        805276
4   2018-06-03 18:52:02.325613        325613
5   2018-06-03 18:52:03.508673        508673
6   2018-06-03 18:54:02.061568         61568
7   2018-06-03 18:52:07.772553        772553
8   2018-06-03 18:53:52.484954        484954
9   2018-06-03 18:53:54.733274        733274
10  2018-06-03 18:54:01.549498        549498
11  2018-06-03 18:53:57.037358         37358
12  2018-06-03 18:53:58.437644        437644
13  2018-06-03 18:54:00.997659        997659
14  2018-06-03 18:53:59.181472        181472
15  2018-06-03 18:54:03.420695        420695
***********************************************
11 temp.json updated

11 Do you want to continue the operation (Y/N): n

»第三次运行

(py3.6) H:\RishikeshAgrawani\Projects\Python3\Pandas>python append_data_to_json-v2-py3.py
                          Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:54:01.549498        549498
3   2018-06-03 18:53:57.037358         37358
4   2018-06-03 18:53:58.437644        437644
5   2018-06-03 18:54:00.997659        997659
6   2018-06-03 18:53:59.181472        181472
7   2018-06-03 18:54:03.420695        420695
8   2018-06-03 18:54:00.436774        436774
9   2018-06-03 18:53:59.805276        805276
10  2018-06-03 18:52:02.325613        325613
11  2018-06-03 18:52:03.508673        508673
12  2018-06-03 18:54:02.061568         61568
13  2018-06-03 18:52:07.772553        772553
14  2018-06-03 18:53:52.484954        484954
15  2018-06-03 18:53:54.733274        733274
16  2018-06-03 18:55:39.415698        415698
***********************************************
1 temp.json updated

1 Do you want to continue the operation (Y/N): y
                          Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:52:02.325613        325613
3   2018-06-03 18:52:03.508673        508673
4   2018-06-03 18:54:02.061568         61568
5   2018-06-03 18:52:07.772553        772553
6   2018-06-03 18:53:52.484954        484954
7   2018-06-03 18:53:54.733274        733274
8   2018-06-03 18:55:39.415698        415698
9   2018-06-03 18:54:01.549498        549498
10  2018-06-03 18:53:57.037358         37358
11  2018-06-03 18:53:58.437644        437644
12  2018-06-03 18:54:00.997659        997659
13  2018-06-03 18:53:59.181472        181472
14  2018-06-03 18:54:03.420695        420695
15  2018-06-03 18:54:00.436774        436774
16  2018-06-03 18:53:59.805276        805276
17  2018-06-03 18:55:41.861641        861641
***********************************************
2 temp.json updated

2 Do you want to continue the operation (Y/N): y
                          Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:53:57.037358         37358
3   2018-06-03 18:53:58.437644        437644
4   2018-06-03 18:54:00.997659        997659
5   2018-06-03 18:53:59.181472        181472
6   2018-06-03 18:54:03.420695        420695
7   2018-06-03 18:54:00.436774        436774
8   2018-06-03 18:53:59.805276        805276
9   2018-06-03 18:55:41.861641        861641
10  2018-06-03 18:52:02.325613        325613
11  2018-06-03 18:52:03.508673        508673
12  2018-06-03 18:54:02.061568         61568
13  2018-06-03 18:52:07.772553        772553
14  2018-06-03 18:53:52.484954        484954
15  2018-06-03 18:53:54.733274        733274
16  2018-06-03 18:55:39.415698        415698
17  2018-06-03 18:54:01.549498        549498
18  2018-06-03 18:55:44.381318        381318
***********************************************
3 temp.json updated

3 Do you want to continue the operation (Y/N): y
                          Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:52:02.325613        325613
3   2018-06-03 18:52:03.508673        508673
4   2018-06-03 18:54:02.061568         61568
5   2018-06-03 18:52:07.772553        772553
6   2018-06-03 18:53:52.484954        484954
7   2018-06-03 18:53:54.733274        733274
8   2018-06-03 18:55:39.415698        415698
9   2018-06-03 18:54:01.549498        549498
10  2018-06-03 18:55:44.381318        381318
11  2018-06-03 18:53:57.037358         37358
12  2018-06-03 18:53:58.437644        437644
13  2018-06-03 18:54:00.997659        997659
14  2018-06-03 18:53:59.181472        181472
15  2018-06-03 18:54:03.420695        420695
16  2018-06-03 18:54:00.436774        436774
17  2018-06-03 18:53:59.805276        805276
18  2018-06-03 18:55:41.861641        861641
19  2018-06-03 18:55:45.181318        181318
***********************************************
4 temp.json updated

4 Do you want to continue the operation (Y/N): y
                          Time  Average Rate
0   2018-06-03 18:51:57.506959        506959
1   2018-06-03 18:52:00.925554        925554
2   2018-06-03 18:55:44.381318        381318
3   2018-06-03 18:53:57.037358         37358
4   2018-06-03 18:53:58.437644        437644
5   2018-06-03 18:54:00.997659        997659
6   2018-06-03 18:53:59.181472        181472
7   2018-06-03 18:54:03.420695        420695
8   2018-06-03 18:54:00.436774        436774
9   2018-06-03 18:53:59.805276        805276
10  2018-06-03 18:55:41.861641        861641
11  2018-06-03 18:55:45.181318        181318
12  2018-06-03 18:52:02.325613        325613
13  2018-06-03 18:52:03.508673        508673
14  2018-06-03 18:54:02.061568         61568
15  2018-06-03 18:52:07.772553        772553
16  2018-06-03 18:53:52.484954        484954
17  2018-06-03 18:53:54.733274        733274
18  2018-06-03 18:55:39.415698        415698
19  2018-06-03 18:54:01.549498        549498
20  2018-06-03 18:55:45.765547        765547
***********************************************
5 temp.json updated

5 Do you want to continue the operation (Y/N): n