用selenium打开浏览器

时间:2013-03-09 21:49:19

标签: python selenium tor

是否有可能使硒使用TOR浏览器?有没有人有他们可以复制粘贴的代码?

10 个答案:

答案 0 :(得分:10)

请勿使用TBB,只需在您使用的任何浏览器中设置正确的代理设置即可。例如,在FF中,像这样:

#set some privacy settings
ff_prof.set_preference( "places.history.enabled", False )
ff_prof.set_preference( "privacy.clearOnShutdown.offlineApps", True )
ff_prof.set_preference( "privacy.clearOnShutdown.passwords", True )
ff_prof.set_preference( "privacy.clearOnShutdown.siteSettings", True )
ff_prof.set_preference( "privacy.sanitize.sanitizeOnShutdown", True )
ff_prof.set_preference( "signon.rememberSignons", False )
ff_prof.set_preference( "network.cookie.lifetimePolicy", 2 )
ff_prof.set_preference( "network.dns.disablePrefetch", True )
ff_prof.set_preference( "network.http.sendRefererHeader", 0 )

#set socks proxy
ff_prof.set_preference( "network.proxy.type", 1 )
ff_prof.set_preference( "network.proxy.socks_version", 5 )
ff_prof.set_preference( "network.proxy.socks", '127.0.0.1' )
ff_prof.set_preference( "network.proxy.socks_port", 9050 )
ff_prof.set_preference( "network.proxy.socks_remote_dns", True )

#if you're really hardcore about your security
#js can be used to reveal your true i.p.
ff_prof.set_preference( "javascript.enabled", False )

#get a huge speed increase by not downloading images
ff_prof.set_preference( "permissions.default.image", 2 )

##
# programmatically start tor (in windows environment)
##
tor_path = "C:\\this\\is\\the\\location\\of\\" #tor.exe
torrc_path = "C:\\you\\need\\to\\create\\this\\file\\torrc"

DETACHED_PROCESS = 0x00000008
#calling as a detached_process means the program will not die with your python program - you will need to manually kill it
##
# somebody please let me know if there's a way to make this a child process that automatically dies (in windows)
##
tor_process = subprocess.Popen( '"' + tor_path+'tor.exe" --nt-service "-f" "' + torrc_path + '"', creationflags=DETACHED_PROCESS )

#attach to tor controller
## imports ##
# import stem.socket
# import stem.connection
# import stem.Signal
##
tor_controller = stem.socket.ControlPort( port=9051 )

control_password = 'password'
#in your torrc, you need to store the hashed version of 'password' which you can get with: subprocess.call( '"' + tor_path+'tor.exe" --hash-password %s' %control_password )

stem.connection.authenticate( tor_controller, password=control_password )

#check that everything is good with your tor_process by checking bootstrap status
tor_controller.send( 'GETINFO status/bootstrap-phase' )
response = worker.tor_controller.recv()
response = response.content()

#I will leave handling of response status to you

答案 1 :(得分:6)

是的,可以让selenium使用TOR浏览器。

我能够在Ubuntu和Mac OS X上都这样做。

必须要做两件事:

  1. 将二进制路径设置为Tor使用的firefox二进制文件。在Mac上,此路径通常为/Applications/TorBrowser.app/Contents/MacOS/firefox。在我的Ubuntu机器上,它是/usr/bin/tor-browser/Browser/firefox

  2. Tor浏览器通过Vidalia或Tor安装在127.0.0.1:9150使用SOCKS主机。从Finder启动Tor一次并将其打开,以便Vidalia运行。使用selenium启动的实例也将使用Vidalia启动的SOCKS主机。

  3. 这是完成这两件事的代码。我在Mac OS X Yosemite上运行它:

    import os
    from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
    from selenium import webdriver
    
    
    # path to the firefox binary inside the Tor package
    binary = '/Applications/TorBrowser.app/Contents/MacOS/firefox'
    if os.path.exists(binary) is False:
        raise ValueError("The binary path to Tor firefox does not exist.")
    firefox_binary = FirefoxBinary(binary)
    
    
    browser = None
    def get_browser(binary=None):
        global browser  
        # only one instance of a browser opens, remove global for multiple instances
        if not browser: 
            browser = webdriver.Firefox(firefox_binary=binary)
        return browser
    
    if __name__ == "__main__":
        browser = get_browser(binary=firefox_binary)
        urls = (
            ('tor browser check', 'https://check.torproject.org/'),
            ('ip checker', 'http://icanhazip.com')
        )
        for url_name, url in urls:
            print "getting", url_name, "at", url
            browser.get(url)
    

    在Ubuntu系统上,我能够通过selenium运行Tor浏览器。这台机器在端口9051运行tor和在端口8118使用tor的privoxy http代理。为了让Tor浏览器通过tor检查页面,我必须将http代理设置为privoxy。

    from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
    from selenium.webdriver.common.proxy import Proxy, ProxyType
    from selenium import webdriver
    browser = None
    
    proxy_address = "127.0.0.1:8118"
    proxy = Proxy({
        'proxyType': ProxyType.MANUAL,
        'httpProxy': proxy_address,
    })
    
    tor = '/usr/bin/tor-browser/Browser/firefox'
    firefox_binary = FirefoxBinary(tor)
    
    urls = (
        ('tor_browser_check', 'https://check.torproject.org/'),
        ('icanhazip', 'http://icanhazip.com'),
    )
    keys, _ = zip(*urls)
    urls_map = dict(urls)
    
    def get_browser(binary=None, proxy=None):
        global browser
        if not browser:
            browser = webdriver.Firefox(firefox_binary=binary, proxy=proxy)
        return browser
    
    if __name__ == "__main__":
        browser = get_browser(binary=firefox_binary, proxy=proxy)
        for resource in keys:
            browser.get(urls_map.get(resource))
    

答案 2 :(得分:5)

//只需检查您的托管浏览器的端口号,然后在//代码

中进行相应更改
from selenium import webdriver
profile=webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
browser=webdriver.Firefox(profile)
browser.get("http://yahoo.com")
browser.save_screenshot("screenshot.png")
browser.close()

答案 3 :(得分:2)

from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

#path to TOR binary
binary = FirefoxBinary(r'...\Tor Browser\Browser\firefox.exe')
#path to TOR profile
profile = FirefoxProfile(r'...\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')

driver = webdriver.Firefox(firefox_profile= profile, firefox_binary= binary)
driver.get("http://icanhazip.com")
driver.save_screenshot("screenshot.png")
driver.quit()

在Windows 10上使用Python 3.5.1

答案 4 :(得分:2)

要使用驱动的Selenium打开GeckoDriver浏览器,您需要:

  • 下载并安装TOR Browser

  • 下载最新的GeckoDriver v0.26.0并将其放入系统中。

  • 安装最新的 Mozilla Firefox v77.0.1 浏览器。

  • 您可以使用以下代码块打开启用 TOR 的浏览器:

    from selenium import webdriver
    from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
    import os
    
    torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
    profile = FirefoxProfile(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
    profile.set_preference('network.proxy.type', 1)
    profile.set_preference('network.proxy.socks', '127.0.0.1')
    profile.set_preference('network.proxy.socks_port', 9050)
    profile.set_preference("network.proxy.socks_remote_dns", False)
    profile.update_preferences()
    firefox_options = webdriver.FirefoxOptions()
    firefox_options.binary_location = r'C:\Program Files\Mozilla Firefox\firefox.exe'
    driver = webdriver.Firefox(firefox_profile= profile, options = firefox_options, executable_path=r'C:\WebDrivers\geckodriver.exe')
    driver.get("http://check.torproject.org")
    
  • 浏览器快照:

torFirefox_torproject


使用Firefox Nightly的替代方法

您也可以下载,安装和使用最新的 Firefox Nightly v79.0a1 浏览器。

  • 代码块:

    from selenium import webdriver
    from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
    import os
    
    torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
    profile = FirefoxProfile(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
    profile.set_preference('network.proxy.type', 1)
    profile.set_preference('network.proxy.socks', '127.0.0.1')
    profile.set_preference('network.proxy.socks_port', 9050)
    profile.set_preference("network.proxy.socks_remote_dns", False)
    profile.update_preferences()
    firefox_options = webdriver.FirefoxOptions()
    firefox_options.binary_location = r'C:\Program Files\Firefox Nightly\firefox.exe'
    driver = webdriver.Firefox(firefox_profile= profile, options = firefox_options, executable_path=r'C:\WebDrivers\geckodriver.exe')
    driver.get("http://check.torproject.org")
    
  • 浏览器快照:

torNightly_torproject


使用Chrome的替代方法

您也可以下载,安装和使用最新的 Chrome v84 浏览器。

  • 代码块:

    from selenium import webdriver
    import os
    
    torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
    PROXY = "socks5://localhost:9050" # IP:PORT or HOST:PORT
    options = webdriver.ChromeOptions()
    options.add_argument('--proxy-server=%s' % PROXY)
    driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
    driver.get("http://check.torproject.org")
    
  • 浏览器快照:

torChrome_torproject


参考文献

您可以在以下位置找到几个相关的详细讨论:

答案 5 :(得分:1)

很多答案都朝着正确的方向发展,但这正是对我有用的:

在Ubuntu上:

您需要使用apt命令或其他方法(而不是二进制版本)安装Tor。

安装指南:

https://linuxconfig.org/how-to-install-tor-browser-in-ubuntu-18-04-bionic-beaver-linux

在sample.py内,您可能需要:

  • 将Firefox的配置文件设置为torrc中的大多数情况下的/etc/tor/
  • 将二进制文件设置为Tor的Firefox二进制文件,因为Tor只是在Firefox之上构建的一系列配置。

您还需要geckodriver用硒自动执行Firefox:

注意:

  • “ network.proxy.socks_port” = 9150
  • 在torcrc ControlPort 9050内部,CookieAuthentication 1
  • 打开TorBrowser
  • sudo lsof -i -P -n | grep LISTEN tor网络的LISTEN端口必须与脚本中的相同
  • TorBrowser打开的同时运行python脚本

感谢user2426679 https://stackoverflow.com/a/21836296/3816638的设置。

from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.proxy import Proxy, ProxyType
from selenium.webdriver.firefox.options import Options
import subprocess
import os

profileTor = '/etc/tor/' #  torrc
binary = os.path.expanduser("~/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/firefox")

firefox_binary = FirefoxBinary(binary)
firefox_profile = FirefoxProfile(profileTor)


#set some privacy settings
firefox_profile.set_preference( "places.history.enabled", False )
firefox_profile.set_preference( "privacy.clearOnShutdown.offlineApps", True )
firefox_profile.set_preference( "privacy.clearOnShutdown.passwords", True )
firefox_profile.set_preference( "privacy.clearOnShutdown.siteSettings", True )   
firefox_profile.set_preference( "privacy.sanitize.sanitizeOnShutdown", True )
firefox_profile.set_preference( "signon.rememberSignons", False )
firefox_profile.set_preference( "network.cookie.lifetimePolicy", 2 )
firefox_profile.set_preference( "network.dns.disablePrefetch", True )
firefox_profile.set_preference( "network.http.sendRefererHeader", 0 )

#set socks proxy
firefox_profile.set_preference( "network.proxy.type", 1 )
firefox_profile.set_preference( "network.proxy.socks_version", 5 )
firefox_profile.set_preference( "network.proxy.socks", '127.0.0.1' )
firefox_profile.set_preference( "network.proxy.socks_port", 9150 )
firefox_profile.set_preference( "network.proxy.socks_remote_dns", True )

#if you're really hardcore about your security
#js can be used to reveal your true i.p.
firefox_profile.set_preference( "javascript.enabled", False )

#get a huge speed increase by not downloading images
firefox_profile.set_preference( "permissions.default.image", 2 )

options = Options()
options.set_headless(headless=False)

driver = webdriver.Firefox(firefox_profile=firefox_profile,firefox_options=options)

print(driver)

driver.get("https://check.torproject.org/")
driver.save_screenshot("screenshot.png")

答案 6 :(得分:0)

使用ruby,

profile = Selenium::WebDriver::Firefox::Profile.new
profile.proxy = Selenium::WebDriver::Proxy.new :socks => '127.0.0.1:9050' #port where TOR runs
browser = Watir::Browser.new :firefox, :profile => profile

要确认您使用的是Tor,请使用https://check.torproject.org/

答案 7 :(得分:0)

我调查了这一点,除非我弄错了,否则就不可能了。

无法做到这一点的原因是:

  • Tor浏览器基于Firefox代码。
  • Tor浏览器具有针对Firefox代码的特定补丁,以防止外部应用程序与Tor浏览器进行通信(包括阻止Components.Interfaces库)。
  • Selenium Firefox WebDriver通过Javascript库与浏览器进行通信,如上所述,这些库被Tor浏览器阻止。

据推测,无论是在您的盒子上还是在互联网上,Tor浏览器之外都没有人知道您的浏览情况。

您的替代方案是:

  • 通过Firefox而不是Tor浏览器使用Tor代理(请参阅问题评论中的链接)。
  • 使用Tor浏览器补丁重建Firefox源代码,不包括那些阻止与Tor浏览器进行外部通信的补丁。

我建议前者。

答案 8 :(得分:0)

As a newer alternative to Selenium, which only controls Firefox, have a look at Marionette. To use with the Tor Browser, enable marionette at startup via

Browser/firefox -marionette

(inside the bundle). Then, you can connect via

from marionette import Marionette
client = Marionette('localhost', port=2828);
client.start_session()

and load a new page for example via

url='http://mozilla.org'
client.navigate(url);

For more examples, there is a tutorial.


Older answer

The Tor project has a selenium test for its browser. It works like:

from selenium import webdriver
ffbinary = webdriver.firefox.firefox_binary.FirefoxBinary(firefox_path=os.environ['TBB_BIN'])
ffprofile = webdriver.firefox.firefox_profile.FirefoxProfile(profile_directory=os.environ['TBB_PROFILE'])
self.driver = webdriver.Firefox(firefox_binary=ffbinary, firefox_profile=ffprofile)
self.driver.implicitly_wait(30)
self.base_url = "about:tor"
self.verificationErrors = []
self.accept_next_alert = True
self.driver.get("http://check.torproject.org/")
self.assertEqual("Congratulations. This browser is configured to use Tor.", driver.find_element_by_css_selector("h1.on").text)

As you see, this uses the environment variables TBB_BIN and TBB_PROFILE for the browser bundle and profile. You might be able to hardcode these in your code.

答案 9 :(得分:0)

System.setProperty("webdriver.firefox.marionette", "D:\\Lib\\geckodriver.exe");
        String torPath = "C:\\Users\\HP\\Desktop\\Tor Browser\\Browser\\firefox.exe";
        String profilePath = "C:\\Users\\HP\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default";

        File torProfileDir = new File(profilePath);
        FirefoxBinary binary = new FirefoxBinary(new File(torPath));
        FirefoxProfile torProfile = new FirefoxProfile(torProfileDir);

        FirefoxOptions options = new FirefoxOptions();
        options.setBinary(binary);
        options.setProfile(torProfile);
        options.setCapability(FirefoxOptions.FIREFOX_OPTIONS,options);
        WebDriver driver = new FirefoxDriver(options);