我正在尝试从此网站下载链接: http://prices.shufersal.co.il
当我尝试从网站上自己下载它工作正常, 但是,当我尝试从代码中下载链接时,它会说FileNotFoundException - 404状态代码。
我认为每次请求页面时都会提供不同的下载链接, 但我仍然无法从代码中下载它。
这是我的代码:
public void download() {
for (int currType = 1; currType <= ShufersalConstants.TYPES.size(); currType++) {
String link = BASE_URL + CAT_MODIFIER + currType + MODIFIER_SEPERATOR + STORE_MODIFIER + 0;
StringBuilder a = HttpUtils.sendGetRequest(link);
System.out.println(a.toString());
fetchLinks(a.toString());
}
}
@Override
public ArrayList<String> fetchLinks(String htmlPage) {
Logger.log("Fetching the links from the HTML page");
// Creating new list that will hold all the links for the .gz files
ArrayList<String> links = new ArrayList<String>();
// Getting the start position of the first download link
int startLinkPos = htmlPage.indexOf("http://pricesprodpublic.blob.core.windows.net");
// Getting the end position of the first download link
int endLinkPos = htmlPage.indexOf(" target=", startLinkPos);
// Running over the string until there are no more download links
while (startLinkPos != -1) {
links.add(htmlPage.substring(startLinkPos, endLinkPos - 1));
HttpUtils.getGZStream(htmlPage.substring(startLinkPos, endLinkPos - 1));
// Returning the list of links
return (links);
}
HttpUtils类对网址执行常规请求。
这是HttpUtils类中的方法:
/**
* The function gets GZ file from the link that the function gets
* @param link The link for the GZ file
* @return GZStream to use with
*/
public static GZIPInputStream getGZStream(String link) {
HttpURLConnection connection = null;
GZIPInputStream gzStream = null;
try {
// Create connection using the link we get as a parameter
URL url = new URL(link);
connection = (HttpURLConnection) url.openConnection();
// Get the response from the connection after sending the request
InputStream is = (InputStream) connection.getInputStream();
// Creating the GZ stream base on the InputStream
gzStream = new GZIPInputStream(is);
} catch (Exception e) {
Logger.logError("An ERROR occured while trying to get GZ stream");
e.printStackTrace();
}
// Returning the response
return (gzStream);
}
有人可以帮我写这样的东西吗? 我真的卡住了,需要下载那些文件..
感谢。