Question

u尝试运行我的程序，但是每次运行中都会出现错误基本上，我的程序这样做： 1.从我的网站获取xml 2.运行所有网址 3.从我的网页获取数据（SKU，名称，标题，价格等） 4.通过以相同的sku比较价格来从另一个网站获得最低的价格

问题是我的xml中有超过7,000个网址，因此我的程序每次都会出现错误网络该怎么办？我该如何解决？

def parse_sitemap (url):
resp = requests.get(XXXX)
for u in urls:
    loc = u.find ('loc').string
    # not a sitemap requirement skip if not present
    out.append ([loc])
    return out
    def get_sku (u):
html = requests.get(u)
bsObj = BeautifulSoup(html.content,'xml')
sku = bsObj.find('span',attrs={'itemprop':'sku'}).get_text()
return sku
def get_price ( u):
    try:
        html = requests.get(u)
        bsObj = BeautifulSoup(html.content,'xml')
        price = bsObj.find('span',attrs={'itemprop':'price'}).get_text()
        price = str(price).replace(' ₪‎','')
        return price
    except:
        return 'no price'
    def get_zapPrice (makat):
try:
    check ='https://www.zap.co.il/search.aspx?keyword='+makat
    r = requests.get(check)
    html = requests.get(r.url)
    bsObj = BeautifulSoup(html.content,'html.parser')
    zapPrice = bsObj.select_one('div.StoresLines div.PriceNum').text.strip().replace(' ₪','')
    return zapPrice
except:
    return 'no zap product'

   def get_zapStoreName (makat):
try:
    check ='https://www.zap.co.il/search.aspx?keyword='+makat
    r = requests.get(check)
    html = requests.get(r.url)
    bsObj = BeautifulSoup(html.content,'html.parser')
    storeName = bsObj.select_one('div.StoresLines 
    div.BuyButtonsTxt').text.strip().replace('ב-','')
    return storeName
except:
    return 'no zap product'

for u in urls:
        ws1 [ 'A1' ] = u
        makat = get_sku(u)
        ws1 [ 'F1' ] = makat
        zapPrice = get_zapPrice(makat)
        ws1['I1'] = zapPrice
        storeName = get_zapStoreName(makat)
        ws1['J1'] = storeName
        ws1.insert_rows(1)
        ws1.append ([])
        print("writing product no." + str(i))
ws1['F1'] = 'makat'
ws1['I1'] = 'zap price'
ws1['J1'] = 'zap store'
wb.save ("sample.xlsx")
wb.close ()
print ('end')

我没有写我所有的代码-基本是在这里每个def它都以requests.get开始，获取我想要的并返回之后，我将其写入excel文件

我在获得1000个URL后检查的问题... 有什么问题吗？

我在我的python程序中收到连接错误

0 个答案: