Question

我想用蒸汽市场中的某些项目填充数据库表，特别是目前来自CSGO的枪支。我似乎无法找到所有枪支名称，皮肤名称和皮肤质量的数据库或列表，这就是我想要的。

我想到的一种方法是到达我想要的项目列表，EG＆＃34; Shotguns＆＃34;，并将页面上的每个项目保存到数据库中，并浏览每个页面搜索。例如： http://steamcommunity.com/market/search?appid=730&q=shotgun#p1_default_desc http://steamcommunity.com/market/search?appid=730&q=shotgun#p2_default_desc 等..

首先，我不确定如何做到这一点，其次，我想知道是否会有更简单的方法。

我计划使用商品名称以后通过将名称替换为此来获取价格：http://steamcommunity.com/market/priceoverview/?currency=3&appid=730&market_hash_name=StatTrak%E2%84%A2%20P250%20%7C%20Steel%20Disruption%20%28Factory%20New%29

通过对每个项目运行检查，每小时更新一次价格。（可能至少有几千......）

Answer 1

您需要做的一般要点归结为：

确定您需要解析的网址。在您的情况下，您会注意到结果是通过ajax加载的。右键单击页面，单击“检查元素”并转到网络选项卡。您会看到实际的网址是：http://steamcommunity.com/market/search/render/?query=&start=<STARTVALUE>&count=<NUMBEROFRESULTS>&search_descriptions=0&sort_column=quantity&sort_dir=desc&appid=730&category_730_ItemSet%5B%5D=any&category_730_TournamentTeam%5B%5D=any&category_730_Weapon%5B%5D=any&category_730_Type%5B%5D=tag_CSGO_Type_Pistol&category_730_Type%5B%5D=tag_CSGO_Type_SMG&category_730_Type%5B%5D=tag_CSGO_Type_Rifle&category_730_Type%5B%5D=tag_CSGO_Type_SniperRifle&category_730_Type%5B%5D=tag_CSGO_Type_Shotgun&category_730_Type%5B%5D=tag_CSGO_Type_Machinegun&category_730_Type%5B%5D=tag_CSGO_Type_Knife
确定响应类型。在这种情况下，它是json，我们想要的数据在html-snippet
找到解析它所需的框架。您可以使用json_decode(...)解码json字符串。 This question将提供有关如何解析html的更多信息。
您现在可以将这些网址提供给加载网页的功能。您可以使用file_get_contents(...)或curl library。
输入您从响应中解析到数据库中的值。确保脚本运行太久时不会被杀死。 This question会为您提供更多相关信息。

您可以将以下内容用作框架。你必须自己计算html的结构，并查找你想要使用的html解析器和mysql库的教程。

<?php
  //Prevent this script from being killed. Please note that if this script never
  //ends, you'll have to kill it manually
  set_time_limit( 0 );

  //The api does not allow for more than 100 results at a time
  $start = 0;
  $count = 100;
  $maxresults = PHP_INT_MAX;
  $baseurl = "http://steamcommunity.com/market/search/render/?query=&start=$1&count=$2&search_descriptions=0&sort_column=quantity&sort_dir=desc&appid=730&category_730_ItemSet%5B%5D=any&category_730_TournamentTeam%5B%5D=any&category_730_Weapon%5B%5D=any&category_730_Type%5B%5D=tag_CSGO_Type_Pistol&category_730_Type%5B%5D=tag_CSGO_Type_SMG&category_730_Type%5B%5D=tag_CSGO_Type_Rifle&category_730_Type%5B%5D=tag_CSGO_Type_SniperRifle&category_730_Type%5B%5D=tag_CSGO_Type_Shotgun&category_730_Type%5B%5D=tag_CSGO_Type_Machinegun&category_730_Type%5B%5D=tag_CSGO_Type_Knife";

  while( $start < $maxresults ) {
    //Constructing the next url
    $url = str_replace( "$1", $start, $baseurl );
    $url = str_replace( "$2", $count, $url );

    //Doing the request
    $ch = curl_init();
    curl_setopt( $ch, CURLOPT_URL, $url );
    curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
    $result = json_decode( curl_exec( $ch ), TRUE );
    curl_close( $ch );

    //Doing things with the result
    //
    //First let's see if everything went according to plan
    if( $result == NULL || $result["success"] !== TRUE ) {
      echo "Something went horribly wrong. Please edit the script to take this error into account and rerun it.";
      exit( -1 );
    }

    //Bookkeeping for the next url we have to fetch
    $count = $result["pagesize"];
    $start += $count;
    $maxresults = $result["total_count"];

    //This is the html we have to parse
    $html = $result["results_html"];

    //Look up an example how to parse html, and how to get data from it
    //Look up how to make a database connection and how to insert data into
    //your database
  }

  echo "And we are done!";

从蒸汽市场制作商品列表

1 个答案: