Question

我需要通过WebDriver验证测试请求。不幸的是，没有简单的方法可以做到这一点，因为没有原生支持。看起来我应该能够使用HtmlUnit来获取请求，但我只能得到响应。有没有办法用HtmlUnit做到这一点，还是我需要设置像Browsermob Proxy这样的东西？我正在使用Java来做到这一点。

谢谢！

Answer 1

我在下面提供了一个使用HtmlUnit的例子：

final WebClient webClient = new WebClient(BrowserVersion.CHROME);
final HtmlPage loginPage = webClient.getPage("http://www.stackoverflow.com");
WebResponse response = loginPage.getWebResponse(); // the response loaded to create this page
WebRequest request = response.getWebRequest();    // the request used to load this page

Answer 2

如果我正确理解您的问题，您希望查看HTMLUnit所做的每个请求和响应。

如果您使用的是Windows下载Fiddler http://www.telerik.com/fiddler

将HTMLUnit的http代理设置设置为使用Fiddler作为代理。

        BrowserVersion bv = BrowserVersion.CHROME;
        bv.setUserAgent("Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36");
        webClient = new WebClient(bv, "127.0.0.1", 8888);

上述内容本身适用于任何不使用HTTPS的网站

如果您想捕获HTTPS流量，请在项目中创建以下类

import java.security.AccessController;
import java.security.InvalidAlgorithmParameterException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.PrivilegedAction;
import java.security.Security;
import java.security.cert.X509Certificate;

import javax.net.ssl.ManagerFactoryParameters;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactorySpi;
import javax.net.ssl.X509TrustManager;

public final class XTrustProvider extends java.security.Provider
{
    /**
     * 
     */
    private static final long   serialVersionUID    = 1L;
    private final static String NAME        = "XTrustJSSE";
    private final static String INFO        = "XTrust JSSE Provider (implements trust factory with truststore validation disabled)";
    private final static double VERSION = 1.0D;

    @SuppressWarnings({ "unchecked", "rawtypes" })
    public XTrustProvider()
    {
        super(NAME, VERSION, INFO);

        AccessController.doPrivileged(new PrivilegedAction()
        {
            public Object run()
            {
                put("TrustManagerFactory." + TrustManagerFactoryImpl.getAlgorithm(), TrustManagerFactoryImpl.class.getName());
                return null;
            }
        });
    }

    public static void install()
    {
        if (Security.getProvider(NAME) == null)
        {
            Security.insertProviderAt(new XTrustProvider(), 2);
            Security.setProperty("ssl.TrustManagerFactory.algorithm", TrustManagerFactoryImpl.getAlgorithm());
        }
    }

    public final static class TrustManagerFactoryImpl extends TrustManagerFactorySpi
    {
        public TrustManagerFactoryImpl()
        {
        }

        public static String getAlgorithm()
        {
            return "XTrust509";
        }

        protected void engineInit(KeyStore keystore) throws KeyStoreException
        {
        }

        protected void engineInit(ManagerFactoryParameters mgrparams) throws InvalidAlgorithmParameterException
        {
            throw new InvalidAlgorithmParameterException(XTrustProvider.NAME + " does not use ManagerFactoryParameters");
        }

        protected TrustManager[] engineGetTrustManagers()
        {
            return new TrustManager[]
            { new X509TrustManager()
            {
                public X509Certificate[] getAcceptedIssuers()
                {
                    return null;
                }

                public void checkClientTrusted(X509Certificate[] certs, String authType)
                {
                }

                public void checkServerTrusted(X509Certificate[] certs, String authType)
                {
                }
            } };
        }
    }
}

调用类的安装方法

XTrustProvider.install();

确保在HTMLUnit发出任何http请求之前调用上述方法。

现在，您将捕获HTMLUnit发出的所有请求，包括https请求。

如果您遇到任何问题评论，我会帮忙。

Answer 3

这样的事情怎么样：

HtmlUnitDriver driver = new HtmlUnitDriver() {
  public Set<WebRequest> requests = new HashSet<>();
  @Override
  protected WebClient modifyWebClient(WebClient originalClient) {
    return new WebClient() {
      @Override
      public WebResponse getPage(WebWindow window, WebRequest request) {
        requests.add(request);
        return super.getPage(window, request)
      }
      @Override
      public WebResponse loadWebResponse(WebRequest request) {
        requests.add(request);
        return super.loadWebResponse(request);
      }
      // If it's really necessary for your use case, you can also override the "download" method in a similar way, but note that this is an internal API
    }
  }
};
driver.open("http://www.example.com/");
Set<WebRequest> requests = (Set<WebRequest>) driver.getClass().getField("requests").get(driver);
for (WebRequest request : requests) {
  System.out.println(request.getUrl().toString()); // or whatever you want
}

当然，如果订单很重要，您可以使用List代替Set，但是您必须检查请求是否已经存在以避免重复。

使用WebDriver / HtmlUnit获取所有HTTP请求

3 个答案: