我在c中构建了一个scraper，我尝试仅下载值列表中定义的类型。示例：curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL, conf->action->url);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeDataInFile);
//write in file
char filename[255];
strcpy(filename, conf->action->name);
file = fopen(filename, "w+");
//Get Type value
for (int i = 0; i < conf->optionSize; i++) {
if (strcmp(conf->option[i]->label, "type") == 0) {
char* contentType[255];
strcpy(contentType, conf->option[i]->value);
}
}
if (file != NULL) {
curl_easy_setopt(curl, CURLOPT_WRITEDATA, file);
errorCurl= curl_easy_perform(curl);
fclose(file);
if (errorCurl == CURLE_OK) {
char* contentType = NULL;
curl_easy_getinfo(curl, CURLINFO_CONTENT_TYPE, &contentType);
}
else println("Cannot download url");
我仅在箭头后面的值列表中下载定义的类型。
我尝试过此方法，但没有用，请问有人有主意吗？
-1 main cat
-1.1 sub-cat
-1.2 sub-cat ...and so on
-2 main cat
- 2.1 sub-cat
- 2.2 sub-cat...and so on