Question

当我开始scrap手y脚时，我正试图获得一辆汽车的价格。

scrapy shell https://www.rentalcars.com/SearchResults.do?country=Argentina&doYear=2018&doFiltering=true&exSuppliers=&fromLocChoose=true&filterTo=49&dropLocationName=Buenos+Aires&emptySearchResults=false&ftsType=C&longitude=-58.3816&ftsLocationSearch=51&dropFtsSearch=L&doDay=25&chinese-license=on&searchType=allareasgeosearch&filterFrom=0&puMonth=9&dropFtsInput=buenos+&dropCountry=Argentina&puDay=24&dropFtsLocationSearch=51&puHour=10&doMonthYear=9-2018&dropFtsEntry=18908&filterCoordinates=-34.44103514%2C-58.8644886&enabler=&puMonthYear=9-2018&distance=10&reducedCategory=small&ftsEntry=18908&city=Buenos+Aires&ordering=price&latitude=-34.6038&filterName=CarCategorisationSupplierFilter&dropCity=Buenos+Aires&dropFtsType=C&ftsAutocomplete=Buenos+Aires%2C+Argentina&driversAge=30&dropFtsAutocomplete=Buenos+Aires%2C+Argentina&dropFtsLocationName=Buenos+Aires&dropCountryCode=&countryCode=&doMinute=0&advSearch=&filterAdditionalInfo=&puYear=2018&puSameAsDo=on&locationName=Buenos+Aires&puMinute=0&ftsInput=buenos+&coordinates=-34.6038%2C-58.3816&dropLocation=-1&doHour=10&dropCoordinates=-34.6038%2C-58.3816&ftsLocationName=Buenos+Aires&ftsSearch=L&location=-1&doMonth=9

fetch('https://www.rentalcars.com/SearchResults.do?country=Argentina&doYear=2018&doFiltering=true&exSuppliers=&fromLocChoose=true&filterTo=49&dropLocationName=Buenos+Aires&emptySearchResults=false&ftsType=C&longitude=-58.3816&ftsLocationSearch=51&dropFtsSearch=L&doDay=25&chinese-license=on&searchType=allareasgeosearch&filterFrom=0&puMonth=9&dropFtsInput=buenos+&dropCountry=Argentina&puDay=24&dropFtsLocationSearch=51&puHour=10&doMonthYear=9-2018&dropFtsEntry=18908&filterCoordinates=-34.44103514%2C-58.8644886&enabler=&puMonthYear=9-2018&distance=10&reducedCategory=small&ftsEntry=18908&city=Buenos+Aires&ordering=price&latitude=-34.6038&filterName=CarCategorisationSupplierFilter&dropCity=Buenos+Aires&dropFtsType=C&ftsAutocomplete=Buenos+Aires%2C+Argentina&driversAge=30&dropFtsAutocomplete=Buenos+Aires%2C+Argentina&dropFtsLocationName=Buenos+Aires&dropCountryCode=&countryCode=&doMinute=0&advSearch=&filterAdditionalInfo=&puYear=2018&puSameAsDo=on&locationName=Buenos+Aires&puMinute=0&ftsInput=buenos+&coordinates=-34.6038%2C-58.3816&dropLocation=-1&doHour=10&dropCoordinates=-34.6038%2C-58.3816&ftsLocationName=Buenos+Aires&ftsSearch=L&location=-1&doMonth=9')

我试图获得价值

response.css('span[class="carResultRow_Price-now"]::text').extract()

结果与网站上显示的结果不同

['\n\t\t                                ARS1,584.37']

正确值为1,563.03

Answer 1

当然，此网站根据标头或其他客户端详细信息来处理请求。例如。计价货币基于您所在的位置。使用以下命令查看使用浏览器得到的响应：

// Create a temp Array to keep the data for the loop
int[] temp = new int[a1.length];

int position = 0;
for (int i = 0; i < a1.length; i++) {
    boolean unique = true;

    for (int j = 0; j < a2.length; j++) {
        if (a1[i] == a2[j]) {
            unique = false;
            break;
        }
    }

    if (unique == true) {
        temp[position] = a1[i];
        position++;
    }
}

// This part merely copies the temp array of the previous size into the proper sized smaller array
int[] result = new int[position];

for (int k = 0; k < result.length; k++) {
    result[k] = temp[k];
}

价格应该与Scrapy解析器的价格相同。

您可以尝试发出与浏览器请求尽可能接近的请求（使用相同的view(response)和User-Agent标头），以获得相同的价格值。

希望有帮助。

响应结果不同

1 个答案: