如何防止我的Python网络抓取工具停止运行?

时间:2020-09-24 01:19:33

标签: python python-requests continuous-integration

嗨!我编写了一个快速(Python)程序,该程序每五分钟捕获一次攀登健身房的使用情况,以供以后分析。我希望它不间断运行,但是我注意到经过几个小时后,将会发生以下两种情况之一。

  1. 它将检测到键盘中断(我没有输入)并停止,或者
  2. 它将简单地停止写入.csv文件,而不会在shell中显示任何故障。

代码如下:

import os
os.chdir('~/Documents/Other/g1_capacity') #ensure program runs in correct directory if opened elsewhere

import requests
import time
from datetime import datetime
import numpy as np
import csv

def get_count():
    url = 'https://portal.rockgympro.com/portal/public/b01ab221559163c5e9a73e078fe565aa/occupancy?&iframeid=occupancyCounter&fId='
    text = requests.get(url).text
    line = ""

    for item in text.split("\n"):
        if "\'count\'" in item:
            line = (item.strip())
            
    count = int(line.split(":")[1][0:-1]) #really gross way to get count number for this specific source

    return count

while True: #run until manual stop

    with open('g1_occupancy.csv', mode='a') as occupancy:
        occupancy_writer = csv.writer(occupancy)
        occupancy_writer.writerow([datetime.now(), get_count()]) #append new line to .csv with timestamp and current count

    time.sleep(60 * 5) #wait five minutes before adding new line

我是Web爬网的新手(实际上,这是我第一次来),我想知道是否有人可以提出建议来帮助消除上述问题。非常感谢!

0 个答案:

没有答案
相关问题