应用错误收集

list.append（value）没有添加任何内容

时间：2017-06-30 05:30:54

标签： python list append

我写了一个简单的程序，在URL的末尾添加一个字符串查询（'）来检查mySQL错误。

这是我的计划的重要部分

vulnURLS = []

def checkforMySQLError(URL):
    global vulnURLS
    req = requests.get(URL, timeout=2)
    soup = BeautifulSoup(req.text, "lxml")
    soup_string = (str(soup))
    if "SQL syntax" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "mysql_fetch" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "mysql_num_rows" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "MySQL Error" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "MySQL_connect()" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "UNION SELECT" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    else:
        print ("\t [-] " + URL)
        print(vulnURLS)

所有打印线都运行良好。但我附加网址无济于事。无论我做什么，都没有加入。我做错了什么？

这是完整的源代码

import urllib.request
import urllib.error
import urllib
from urllib.request import urlopen
import socket
import threading
import multiprocessing
import itertools
import time
import ssl
import requests
from lxml import html
from bs4 import BeautifulSoup
import re
import sys
from functools import partial

vulnURLS = []
headers={'User-agent' : 'Mozilla/5.0'}

class MyException(Exception):
    pass

def checkforMySQLError(URL):
    global vulnURLS
    req = requests.get(URL, timeout=2)
    soup = BeautifulSoup(req.text, "lxml")
    soup_string = (str(soup))
    if "SQL syntax" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "mysql_fetch" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "mysql_num_rows" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "MySQL Error" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "MySQL_connect()" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    elif "UNION SELECT" in soup_string:
        vulnURLS.append(URL)
        print ("\t [+] " + URL)
    else:
        print ("\t [-] " + URL)
        print(vulnURLS)

def tryMethod(req):
    try:
        checkforMySQLError(req)
    except urllib.error.HTTPError as e:
        if e.code == 404:
            print("\t [-] Page not found.")
        if e.code == 400:
            print ("\t [+] " + req)
    except urllib.error.URLError as e:
        print("\t [-] URL Timed Out")
    except socket.timeout as e:
        print("\t [-] URL Timed Out")
        pass
    except socket.error as e:
        print("\t [-] Error in URL")
        pass

def mainMethod(URLList):
    URL = URLList.replace("\n", "")
    URL = URL.replace("https://","http://")
    URL = URL.replace("\s", "%20")
    URL = (URL + "'")
    tryMethod(URL)

def mp_handler(URLList):
    p = multiprocessing.Pool(10)
    p.imap(mainMethod, URLList)
    p.close()
    p.join()

if __name__=='__main__':
    URLList = open('sites.txt', 'r')
    mp_handler(URLList)

正如我所说，所有打印行都运行良好，所以我知道代码运行正常。但为什么我的名单中没有添加任何内容？我被困在这两个小时，我觉得很荒谬。

这里有一些结果编码来证明我并不总是得到其他情况（我在每次迭代后打印列表）

C:\Users\Brice\Desktop\b>py SiteHunter.py
         [-] Error in URL
         [-] http://m.jccal.org/Default.asp?ID=840'
[]
         [-] http://www.splitcoaststampers.com/gallery/showgallery.php?cat=3089'
[]
         [+] http://aaqct.org.ar/capacitacion/single.php?id=1'
         [-] http://wisc.jobs/public/links_summary_page.asp?catid=85'
[]
         [-] http://www.ceciliareynoso.com.ar/page.php?lang=es&album=7&id=28'
[]
         [-] http://www.snejbjergcykelmotion.dk/pages/webside.asp?articleGuid=47645'
[]
         [-] http://www.pcmusic.org/about/team.php?lang=en'
[]
         [-] http://www.thehealthsite.com/wp-content/themes/health2014/images/health-logo.png?v=20170617-V1'
[]
         [-] http://www.ipdia.com/index.php?page=our_products'
[]
         [-] http://cosmet.ac.jp/mt4i/mt4i.cgi?id=11&mode=individual&eid=1169'
[]
         [-] http://www.timeworx.co.nz/index.php?route=product/product&product_id=59'
[]
         [-] http://www.e-scow.org/content.aspx?page_id=22&club_id=180842&module_id=129141'
[]
         [-] http://www.andersonsbooks.com/searchResults.php?category_id=332&action=browse'
[]
         [-] http://www2.ub.edu/assessling/cgi/consultes/resposta.pl?consultes_id=743'
[]
         [-] http://www.butterpaper.com/vanilla/comments.php?DiscussionID=1095'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://www.zombierunner.com/cgi-sys/cgiwrap/zr/sc/order.cgi?storeid=*1241f8a65e9100618c5065&cert=gift'
[]
         [-] http://www.mcspaddendulcimers.com/SearchResults.asp?Cat=18'
[]
         [-] http://www.tumbletots.com/programmes.php?ID=4'
[]
         [-] http://ventas.ticketplus.com.pa/ordertickets.asp?p=4324'
[]
         [-] http://www.thebestpageintheuniverse.net/c.cgi?u=buzzfeed'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://ufw.org/_page.php?menu=about&inc=about_exe.html'
[]
         [-] http://643boxbreaks.com/browse.php?id=198'
[]
         [-] http://wiki.carleton.edu/pages/viewpage.action?pageId=8717123'
[]
         [-] http://champersrestaurant.com/index.php?categoryid=16'
[]
         [+] http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'
         [-] http://www.german-profec.com/cms/front_content.php?idcat=100'
[]
         [-] http://www.manuela-maurer.ch/default.aspx?navid=6'
[]
         [-] http://wms.wusd13.org/index.cfm?pID=4809'
[]
         [-] http://www.doctortyre.it/index.php?option=com_phocagallery&view=category&id=1&Itemid=190'
[]
         [-] http://www.coldplay.com/newsdetail.php?id=1531'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://www.nantucketlittleleague.com/getpdf.php?id=19'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [-] http://www.usw745.org/index.cfm?action=cat&categoryID=2631168f-fc4d-4b5f-a640-9df545603e29'
[]
         [-] http://support.authorize.net/authkb/index?page=content&id=A601'
[]
         [-] http://www.integro-usa.com/Store/default.asp?categoryID=10'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://www.humhotels.hr/hotels_details.asp?id=6'
[]
         [-] http://krolldesign.co.il/works.asp?catid=2&subcatid=7'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [-] http://www.qrz.com/cgi-bin/ikonboard.cgi?s=3f3438492d5effff;act=ST;f=7;t=41245'
[]
         [-] http://forum.slowtwitch.com/gforum.cgi?post=1612485;search_string=runtraining'
[]
         [-] http://www.elgressy.com/list.asp?categoryId=287&id=763'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://fireflylisting.com/listing.php?id=1060'
[]
         [+] http://si.chanwanich.com/technologydetail.php?id=8'
         [-] Error in URL
         [-] http://vestas.taleo.net/careersection/jobdetail.ftl?job=170001ZZ'
[]
         [-] http://www.biblioteca.juriquilla.unam.mx/gsdl/cgi-bin/linux/library.cgi?e=d-01000-00---off-0cgeo--00-1----0-10-0---0---0direct-10----4-------0-1l--11-es-50---20-about---00-1-1-00-00--4--0--0-0-11-10-0utfZz-8-00&a=d&c=cgeo&cl=CL4.4'
[]
         [-] Error in URL
         [-] http://www.icms.org.uk/workshop.php?id=299'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [-] http://owentrailers.com/products/products.asp?cat=darkrides'
[]
         [-] http://www.deaftravel.co.uk/directorydetails.php?id=26'
[]
         [-] http://www.willdan.com/newsDetail.aspx?NewsID=Willdan-Announces-New-Board-Appointments'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [-] http://www.lawtv.kr/bbs/board.php?tbl=notice&mode=VIEW&num=2&chr=&category=&findType=&findWord=&sort1=&sort2=&program_code=&page=1'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] Error in URL
         [-] http://www.thebarbiecollection.com/shop/OrderStatusGuestView?storeId=10154&langId=-1&catalogId=10551'
[]
         [-] http://www.enom.com/help/hostinghelp.aspx?hosthelp=9'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [-] http://cai.caionline.org/eweb/DynamicPage.aspx?site=CAI&WebCode=storeItemDetail&parentKey=0b859898-096b-4890-8293-c7256d54423b'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [-] http://www.haganenet.co.il/content.asp?pageID=6215'
[]
         [-] http://wiki.contrexx.com/en/index.php?title=New_Module'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://www.kartell.com/cms/legal/useterms.asp?tskay=52EE612B'
[]
         [-] Error in URL
         [-] http://www.sungkwang21.org/board/view.php?id=gallery&npage=1&mode=view&idx=1483&search_text='
[]
         [-] http://dsgr.org/cpage.php?pt=138'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [+] http://isindexing.com/isi/journaldetails.php?id=327'
         [-] http://www.sofame.com/modules.php?op=modload&name=News&file=article&sid=195&newlang=eng'
[]
         [-] http://www.eas.sk/mod/product/show.php?ID=59'
[]
         [-] http://www.jzsjxw.gov.cn/nskslist.asp?ztid=304'
["http://aaqct.org.ar/capacitacion/single.php?id=1'"]
         [-] http://www.unknowncheats.me/forum/downloads.php?do=file&id=16716'
[]
         [-] Error in URL
         [-] http://csn.augusoft.net/index.cfm?fuseaction=1010'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [+] http://www.bajantube.com/play-video.php?videoid=8'
         [-] http://www.inaya.com.kw/contactus.aspx?Pid=1019'
["http://isindexing.com/isi/journaldetails.php?id=327'"]
         [-] http://www.przemysl.ap.gov.pl/index.php?c=page&id=61'
[]
         [-] http://www.tmbvacuum.com/file-prodotti.php?id=31'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [-] http://www.ashanet.org/projects/project-view.php?p=636'
[]
         [-] http://uplnd.com/products/detail.php?product_id=170190768'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [-] http://www.presidency.ucsb.edu/ws/?pid=15349'
["http://aaqct.org.ar/capacitacion/single.php?id=1'", "http://www.bajantube.com/play-video.php?videoid=8'"]
         [-] http://www.sanin-chuo.co.jp/shashin/modules/news/photonews.php?topicid=203'
[]
         [-] http://aencore.com/pro/list.php?pid=130&f=129'
[]
         [-] http://www.quanjinglian.com/portal.php?mod=topic&topicid=65'
[]
         [-] http://secy.klk.com/navigation/categorylist.asp?MySessionID=119-431116457&CatID=14'
[]
         [-] http://sanctuarysimon.org/projects/project_info.php?projectID=100391'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [-] http://thetourismcompany.com/topic.asp?topicid=31'
[]
         [-] http://rusmusic.su/song.php?id=36'
["http://si.chanwanich.com/technologydetail.php?id=8'"]
         [-] http://percichester.com/Asset.asp?AssetID=71806&AKey=MCFLQYDL'
["http://www.nlcng.org/newsview.php?viewarticleactiveschool=85'"]
         [+] http://www.mazetrivia.com/viewmaze.php?id=77'
         [-] Error in URL
         [-] http://www.sheffieldphoenix.com/showbook.asp?bkid=637'
[]
         [-] http://www.lexingtonnc.net/modules/showdocument.aspx?documentid=1581'
[]
         [-] http://www.investor.nexteraenergy.com/phoenix.zhtml?c=88486&p=earningsRelease'
[]

编辑：我看到它正在添加到列表中，但之后立即清除。这是为什么？

1 个答案:

答案 0 :(得分：0)

您正在使用multiprocessing.Pool。这会创建进程，而不是线程。内存不在进程之间共享，就像在线程之间共享一样，因此每个工作者都有自己的vulnURLS列表副本。您需要重新设计程序以使用线程或让进程返回列表，然后在主进程中将它们连接起来。