转到https://www.comparis.ch/carfinder/default时，“模型1”下拉菜单将处于非活动状态，直到您从“马克”下拉列表中选择了某些内容。 我的目标是通过浏览所有“ Marke”值的列表来抓取所有“ Modell”值。由于从“ Marke”下拉菜单中选择某些内容时URL不会更改，因此我正在努力实现这一目标。使用此代码，我可以轻松获得“ Marke”值的列表：
from bs4 import BeautifulSoup
newest_secondhand_cars = 'https://www.comparis.ch/carfinder/default'
newest_secondhand_cars = requests.get(newest_secondhand_cars)
newest_secondhand_cars = newest_secondhand_cars.content
soup = BeautifulSoup(newest_secondhand_cars, "lxml")
marken = [str(el).split('>')[1].split('<')[0] for el in list(soup.find('select', {'id': 'ddlMake'}).find_all('option'))][1:]
print(marken)
现在如何动态地插入“ Marke”值，然后抓取所有“ Modell”值？ 我认为这个答案非常接近，但我没有设法完成：scraping a response from a selected option in dropdown list。 这将是代码尝试使用“ Marke”值执行POST请求，然后从响应中删除“ Modell”值的代码（无效）：
from bs4 import BeautifulSoup
import requests
url = 'https://www.comparis.ch/carfinder/default'
with requests.Session() as session:
session.headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36'}
# parsing parameters
response = session.get(url)
soup = BeautifulSoup(response.content, 'lxml')
data = {
'MakeList': 'BMW',
#'ctl00$ctl00$txtSearchWord': '',
#'__EVENTTARGET': soup.find('input', {'name': '__EVENTTARGET'}).get('value', ''),
#'__EVENTARGUMENT': soup.find('input', {'name': '__EVENTARGUMENT'}).get('value', ''),
#'__LASTFOCUS': soup.find('input', {'name': '__LASTFOCUS'}).get('value', ''),
#'__VIEWSTATE': soup.find('input', {'name': '__VIEWSTATE'}).get('value', ''),
#'__VIEWSTATEGENERATOR': soup.find('input', {'name': '__VIEWSTATEGENERATOR'}).get('value', ''),
#'__EVENTVALIDATION': soup.find('input', {'name': '__EVENTVALIDATION'}).get('value', ''),
}
# parsing data
response = session.post(url, data=data)
soup = BeautifulSoup(response.content, 'lxml')
print(soup.find('select', {'id': 'ddlModelGroup'}))
答案 0 :(得分：1)
import requests
from bs4 import BeautifulSoup
from itertools import zip_longest
r = requests.get("https://www.comparis.ch/carfinder/default")
soup = BeautifulSoup(r.text, 'html.parser')
marke = []
tag = []
model = []
for item in soup.findAll("select", {'id': 'ddlMake'}):
for name in item.findAll("option")[1:]:
marke.append(name.text)
for item in soup.findAll("select", {'id': 'ddlMake'}):
for num in item.findAll("option")[1:]:
tag.append(num.get("value"))
for a, b in zip_longest(marke, tag):
print(f"{'*' * 10}Extracting: {a}{'*' * 10}\n")
r = requests.get(
f"https://www.comparis.ch/carfinder/searchservice/GetModelGroupsForMake?makeID={b}").json()
data = list(item['Value'].strip() for item in r)
print(f"{data}\n")
model.append(data)
输出样本：
**********Extracting: Aixam**********
['300', '325', '400', '500', '600', '721', '751', 'Crossline', 'MAC', 'Scouty']
等等。
或使用以下内容：
import requests
from bs4 import BeautifulSoup
from itertools import zip_longest
r = requests.get("https://www.comparis.ch/carfinder/default")
soup = BeautifulSoup(r.text, 'html.parser')
marke = []
tag = []
model = []
for item in soup.findAll("select", {'id': 'ddlMake'}):
for name in item.findAll("option")[1:]:
marke.append(name.text)
for item in soup.findAll("select", {'id': 'ddlMake'}):
for num in item.findAll("option")[1:]:
tag.append(num.get("value"))
for a, b in zip_longest(marke, tag):
print(f"Extracting: {a}")
r = requests.get(
f"https://www.comparis.ch/carfinder/searchservice/GetModelGroupsForMake?makeID={b}").json()
data = list(item['Value'] for item in r)
model.append(data)
for a, b in zip_longest(marke, model):
print("Marke: {:<20}, Model: {:<100}".format(
a, ", ".join(str(i).strip() for i in b)))
Marke: Aixam , Model: 300, 325, 400, 500, 600, 721, 751, Crossline, MAC,Scouty
Marke: Alfa Romeo , Model: Stelvio, 145, 146, 147, 155, 156, 159, 164, 166, 1750, 33, 4C, 6, 75, 8C, 90, Alfasud, Alfetta, Arna, Brera, Giulia, Giulietta, GT, GTV, Mito, Montreal, Spider, Zagato
Marke: American Motors , Model: Eagle
Marke: Artega , Model: GT
Marke: Aston Martin , Model: Cygnet, DB 11, DB 7, DB 9, DBS, Lagonda, Rapide, V12 Vanquish, V12 Vantage, V8, V8 Vantage, Virage
Marke: Audi , Model: 100, 200, 80, 90, A1, A2, A3, A4, A4 Allroad, A5, A6, A6 Allroad, A7, A8, Cabriolet, Coupé, e-Tron, Q2, Q3, Q5, Q7, Q8, Quattro, R8, RS Q3, RS3, RS4, RS5, RS6, S1, S3, S4, S5, S6, S7, S8, SQ5, TT, V8
Marke: Austin , Model: Healey, Maestro, Metro, Montego
Marke: Bentley , Model: Arnage, Azure, Bentayga, Brooklands, Continental, Flying Spur, Mulsanne, Turbo
Marke: Bertone , Model: Freeclimber
Marke: BMW , Model: 1 Series, 2 Series, 2002, 3 Series, 4 Series, 5 Series, 6 Series, 7 Series, 8 Series, i3, i8, M-Series, X-Series, Z-Series
Marke: Bmw-Alpina , Model: B10, B12, B3, B4, B5, B6, B7, B8, D10, D3, D4, D5,Roadster, XD3, XD4, Z8
Marke: Bugatti , Model: EB, Veyron
Marke: Buick , Model: Century, Electra, Park Avenue, Regal, Riviera, Skylark
Marke: Cadillac , Model: Allanté, ATS, BLS, CT, CTS, Deville, Eldorado, Escalade, Fleetwood, Seville, SRX, STS, XLR, XT
Marke: Caterham , Model: Seven, Super Seven
Marke: Chevrolet , Model: Alero, Astro, Aveo, Beretta, Blazer, Camaro, Caprice, Captiva, Cavalier, Celebrity, Citation, Corsica, Corvair, Corvette, Cruze, Epica, Evanda, HHR, Impala, Kalos, Lacetti, Malibu, Matiz, Nubira, Orlando, Spark, Tacuma, Tahoe, TrailBlazer, Trans-Sport, Trax, Volt
Marke: Chrysler , Model: 300C, 300M, Crossfire, Daytona, ES, Grand Voyager,GS, GTS, Le Baron, Neon, New Yorker, PT Cruiser, Saratoga, Sebring, Stratus, Valiant, Viper, Vision, Voyager
Marke: Citroen , Model: 2CV, Acadiane, AX, Berlingo, Berlingo Electric, BX, C 15, C 25, C 35, C1, C2, C3, C3 Picasso, C3 Pluriel, C4, C4 Aircross, C4 Cactus, C4 Grand Picasso, C4 Picasso, C5, C6, C8, C-Crosser, CX, C-Zero, DS3, DS4, DS5, E-Mehari, Evasion, GSA, Jumper, Jumpy, LNA, Nemo, Saxo, Spacetourer, Visa, Xantia, XM, Xsara, ZX
Marke: Cupra , Model: Ateca
Marke: Dacia , Model: Dokker, Duster, Lodgy, Logan, Sandero
Marke: Daewoo , Model: Espero, Evanda, Kalos, Korando, Lacetti, Lanos, Leganza, Matiz, Musso, Nexia, Nubira, Rexton, Tacuma
Marke: Daihatsu , Model: Applause, CAB, Charade, Charmant, Copen, Cuore, Feroza, Gran Move, Hijet, Materia, Move, Pick-up, Rocky, Sirion, Taft, Terios, Trevis, YRV
Marke: Daimler , Model: X300, X308, XJ 40, XJ Serie III, XJ Serie V
Marke: De Tomaso , Model: Guarà, Pantera
Marke: DFM , Model: Mini
Marke: Dfsk , Model: C-Serie, K-Serie, V-Serie
Marke: Dodge , Model: Avenger, Caliber, Challenger, Dart, Durango, Journey, Magnum, Nitro, Ram, Viper
Marke: Donkervoort , Model: D8, S8
Marke: Ds Automobiles , Model: DS3, DS4, DS5, DS7
Marke: Ferrari , Model: 308, 328, 348, 412, 456, 512, 575, 599, 612, 812, California, Dino, Enzo, F12, F355, F360, F40, F430, F458, F488, F50, F550, FF, GTC, Laferrari, Mondial, Portofino, Testarossa
Marke: Fiat , Model: 124, 126, 127, 130, 131, 500, 500L, 500X, 595, 850, Argenta, Barchetta, Bertone, Brava, Bravo, Cinquecento, Coupé, Croma, Dino, Doblo, Ducato, Fiorino, Freemont, Fullback, Idea, Marea, Multipla, Palio, Panda, Punto, Qubo, Regata, Ritmo, Scudo, Sedici, Seicento, Spider, Stilo, Strada, Talento, Tempra, Tipo, Topolino, Ulysse, Uno
Marke: Fisker , Model: Karma
Marke: Ford , Model: B-Max, Capri, C-Max, Cortina, Courier, Ecosport, Edge, Escort, Fiesta, Focus, Fusion, Galaxy, Granada, Grand C-MAX, KA, Kuga, Maverick, Mercury, Mondeo, Mustang, Orion, Puma, Ranger, Scorpio, Sierra, S-Max, Streetka, Taunus,Tourneo, Tourneo Connect, Tourneo Courier, Tourneo Custom, Transit, Transit Connect, Transit Courier, Transit Custom
Marke: Ford (USA) , Model: Aerostar, Cougar, Explorer, GT, Mustang, Probe, Taurus, Thunderbird, Windstar
Marke: Giotti Victoria , Model: Gladiator
Marke: Honda , Model: Accord, Acty, Ballade, Civic, Concerto, CR-V, CR-X, CR-Z, FR-V, HR-V, Insight, Integra, Jazz, Joker, Legend, Logo, NSX, Prelude, Quintet,S2000, Shuttle, Stream
Marke: HS , Model: Cobra
Marke: Hummer , Model: H2, H3
Marke: Hyundai , Model: Accent, Atos, Coupé, Elantra, Galloper, Genesis, Getz, Grandeur, H 100, H-1, H350, i10, i20, i30, i40, Ioniq, iX20, iX35, iX55, Kona, Lantra, Matrix, Nexo, Pony, Santa Fe, Santamo, Scoupe, Sonata, Starex, Terracan, Trajet, Tucson, Veloster, XG
Marke: Infiniti , Model: EX, FX, G, M, Q, QX
Marke: Innocenti , Model: Mini
Marke: Isuzu , Model: Campo, D-Max, KB, L, Midi, N, NKR, NLR, NMR, NNR, NPR, Pick Up, WFR, WFS
Marke: Isuzu (GM) , Model: Trooper
Marke: Isuzu (J) , Model: Gemini, Piazza
Marke: Iveco , Model: 50, Daily, Eco-daily, Massif, Turbo-daily
Marke: Jaguar , Model: Concept, E-Pace, E-Type, F-Pace, F-Type, I-Pace, MK, S-Type, XE, XF, XJ, XJR, XJ-S, XK, X-Type
Marke: Jeep , Model: Cherokee, CJ-7, CJ-8, Commander, Compass, Grand Cherokee, Kaiser, Patriot, Renegade, Wagoneer, Willys, Wrangler
Marke: KIA , Model: Carens, Carnival, Cee'd, Cerato, Clarus, K2500, K2900, Leo, Magentis, Niro, Opirus, Optima, Picanto, Pregio, Pride, Rio, Sephia, Shuma, Sorento, Soul, Sportage, Stinger, Stonic, Venga
Marke: KTM , Model: X-Bow
Marke: Lada , Model: 110, 111, 112, 1500, Kalina, Niva, Nova, Priora, Samara
Marke: Lamborghini , Model: Aventador, Countach, Diablo, Gallardo, Huracan, Jalpa, LM, Murciélago, Urus
Marke: Lancia , Model: A 112, Beta, Dedra, Delta, Flaminia, Flavia, Fulvia, Gamma, HPE, Kappa, Lybra, Musa, Phedra, Prisma, Thema, Thesis, Trevi, Voyager, Y, Ypsilon, Zeta
Marke: Land Rover , Model: 109, 110, 88, 90, Defender, Discovery, Evoque, Freelander, Range Rover, Range Rover Sport, Range Rover Velar
Marke: Lexus , Model: CT, GS, IS, LC, LS, NX, RC, RX, SC, UX
Marke: Ligier , Model: Be Up, Nova
Marke: Lotus , Model: 3-Eleven, Elan, Elise, Esprit, Europa, Evora, Exige, Super Seven
Marke: MAN , Model: TGE
Marke: Maserati , Model: 3200, Biturbo, Coupé, Ghibli, Grancabrio, Granturismo, Karif, Levante, Quattroporte, Shamal, Spyder
Marke: Mazda , Model: 121, 2, 3, 323, 5, 6, 626, 929, B 2500, BT-50, CX-3, CX-30, CX-5, CX-7, CX-9, Demio, E, MPV, MX-3, MX-5, MX-6, Premacy, RX-7, RX-8, Tribute, Xedos
Marke: Mclaren , Model: 540c, 570gt, 570s, 600lt, 650S, 675lt, 720S, MP4
Marke: Mega , Model: Mega
Marke: Mercedes-Benz , Model: 100, 190, 207, 208, 209, 210, 220, 230, 250, 280, 307, 308, 309, 310, 350, 400, 407, 409, 410, 450, 508, 510, 609, 611, 612, A-Klasse, B-Klasse, Citan, C-Klasse, CLA-Klasse, CLC-Klasse, CLK-Klasse, CL-Klasse, CLS-Klasse, E-Klasse, EQC, G-Klasse, GLA-Klasse, GLC-Klasse, GLE-Klasse, GLK-Klasse, GL-Klasse, GLS-Klasse, GT, Marco Polo, M-Klasse, R-Klasse, SEC-Klasse, S-Klasse, SLC-Klasse, SLK-Klasse,SL-Klasse, SLS-Klasse, Sprinter, Vaneo, Viano, Vito, V-Klasse, X-Klasse
Marke: MG , Model: F, Maestro, Metro, MGB, Montego, TF, ZR, ZS, ZT
Marke: Minelli , Model: TF
Marke: Mini , Model: Cabriolet, Clubman, Clubvan, Countryman, Coupé, Mini, Paceman, Roadster
Marke: Mitsubishi , Model: 3000 GT, ASX, Attrage, Canter, Carisma, Colt, Cordia, Eclipse, Galant, Grandis, i Miev, L 200, L 300, L 400, Lancer, Outlander, Pajero, Sapporo, Sigma, Space Gear, Space Runner, Space Star, Space Wagon, Starion, Tredia
Marke: Mitsubishi Fuso , Model: Canter
Marke: Morgan , Model: 4/4, Aero 8, Plus
Marke: Nissan , Model: Almera, Atleon, Bluebird, Cabstar, Cabstar E, Cherry, Cube, Double, e-NV200, GT-R, Interstar, Juke, King-/Double, Kubistar, Laurel, Leaf,Maxima, Micra, Murano, Navara, Note, Np300, NT400, Nv200, Nv300, Nv400, NX, Pathfinder, Patrol, Pixo, Prairie, Primastar, Primera, Pulsar, Qashqai, Serena, Silvia, Stanza, Sunny, SX, Terrano, Tiida, Trade, Urvan, Vanette, X-Trail, Z, ZX
Marke: Oldsmobile , Model: Cutlass, Delta 88, Ninety Eight, Toronado
Marke: Opel , Model: Adam, Admiral, Agila, Ampera, Antara, Ascona, Astra, Blitz, Calibra, Campo, Cascada, Combo, Commodore, Corsa, Crossland, Frontera, Grandland, GT, Insignia, Kadett, Karl, Manta, Meriva, Mokka, Monterey, Monza, Movano, Olympia, Omega, Rekord, Senator, Signum, Sintra, Speedster, Tigra, Vectra, Vivaro, Zafira
Marke: Peugeot , Model: 1007, 104, 106, 107, 108, 2008, 205, 206, 207, 208, 3008, 304, 305, 306, 307, 308, 309, 4007, 4008, 404, 405, 406, 407, 5008, 504, 505, 508, 604, 605, 607, 806, 807, Bipper, Boxer, Expert, iON, J 5, J 9, Partner, RCZ, Rifter, Traveler
Marke: Piaggio , Model: Maxxi, Porter, Quargo
Marke: Pontiac , Model: 6000, Bonneville, Firebird, Grand-AM, Grand-Prix, Trans Am, Trans Sport
Marke: Porsche , Model: 356, 911, 924, 928, 944, 968, Boxster, Carrera GT,Cayenne, Cayman, Macan, Panamera
Marke: Puch , Model: G, Haflinger, Pinzgauer
Marke: Qvale , Model: Mangusta
Marke: Reliant , Model: Scimitar
Marke: Renault , Model: 4 F, Alaskan, Alpine, Avantime, Captur, Clio, Espace, Express, Fluence, Fuego, Grand Espace, Grand Scénic, Kadjar, Kangoo, Koleos, Laguna, Latitude, Master, Mégane, Modus, R11, R12, R14, R16, R18, R19, R20, R21, R25, R30, R4, R5, R9, Safrane, Scénic, Spider, Super 5, Talisman, Trafic, Twingo, Twizy, Vel Satis, Wind, Zoe
Marke: Renault Trucks , Model: B 120, B 90, Mascott, Master, Maxity
Marke: Rolls-Royce , Model: Corniche, Cullinan, Dawn, Flying Spur, Ghost, Park Ward, Phantom, Silver Dawn, Silver Seraph, Silver Spirit, Silver Spur, Touring, Wraith
Marke: Rover , Model: 200, 25, 2600, 3500, 400, 45, 600, 75, 800, Mini, Streetwise
Marke: Saab , Model: 90, 900, 9000, 9-3, 9-4X, 9-5, 99
Marke: Seat , Model: Alhambra, Altea, Altea XL, Arona, Arosa, Ateca, Cordoba, Exeo, Ibiza, Inca, Leon, Malaga, Marbella, MII, Ronda, Tarraco, Terra, Toledo
Marke: Secma , Model: F16
Marke: Skoda , Model: 135, 136, Citigo, Fabia, Favorit, Felicia, Forman,Kamiq, Karoq, Kodiaq, Octavia, Praktik, Rapid, Roomster, Scala, Superb, Yeti
Marke: Smart , Model: Forfour, Fortwo, Roadster
Marke: Sokon , Model: Mini Truck
Marke: Ssang Yong , Model: Actyon, Korando, Kyron, Musso, Rexton, Rodius, Tivoli, XLV
Marke: Subaru , Model: 1600, 1800, 700, BRZ, E10, E12, Forester, G3X Justy, Impreza, Justy, Legacy, Levorg, Outback, SVX, Trezia, Tribeca, Vivio, WRX, XT, XV
Marke: Suzuki , Model: Alto, Baleno, Carry, Celerio, Grand Vitara, Ignis, Jimny, Kizashi, Liana, LJ, SA, Samurai, SJ, Splash, Swift, SX4, SX4 S-Cross, Vitara, Wagon R, X-90
Marke: Talbot , Model: Horizon, Matra, Samba, Solara, Tagora
Marke: Tata , Model: Indica, Pick Up, Safari
Marke: Tazzari , Model: Zero
Marke: Tesla , Model: MODEL 3, Model S, Model X, Roadster
Marke: Think , Model: City
Marke: Toyota , Model: 4-Runner, Auris, Auris Touring Sports, Avensis, Avensis Verso, Aygo, Camry, Carina, Celica, C-HR, Corolla, Corolla Verso, Cressida, Crown, Dyna, GT, Hi-ace, Hi-lux, iQ, Land Cruiser, Lite-ace, Mirai, Model - F, MR 2, Paseo, Picnic, Previa, Prius, Prius+, Proace, Proace Verso, RAV-4, Sienna, Starlet, Supra, Tercel, Urban Cruiser, Verso, Verso-S, Yaris, Yaris Verso
Marke: Triumph , Model: Spitfire, TR 6
Marke: TVR , Model: Cerbera, Chimaera, Griffith, S4, T350, Tuscan, V8
Marke: Venturi , Model: Cabriolet, Coupé
Marke: Volvo , Model: 144, 164, 240, 340, 360, 440, 460, 480, 740, 760, 780, 850, 940, 960, C30, C70, PV, S40, S60, S60 CC, S70, S80, S90, V40, V40 CC, V50, V60, V60 CC, V70, V90, V90 CC, XC40, XC60, XC70, XC90
Marke: VW , Model: Amarok, Arteon, Beetle, Bora, Buggy, Caddy, CC, Corrado, Crafter, Cross Caddy, Crossgolf, Crosspolo, Crosstouran, Crossup, Eos, Fox, Golf, Golf Plus, Golf Sportsvan, Grand California, Jetta, Käfer, Karmann, LT - Serie, Lupo,Passat, Phaeton, Polo, Santana, Scirocco, Sharan, T1, T2, T4, T5, T6, Taro - Serie, T-Cross, Tiguan, Touareg, Touran, T-ROC, Typ 2 - Serie, Up, Vento
Marke: Wiesmann , Model: Coupé, Roadster
Marke: Zagato , Model: Hyena