我正在使用Selenium抓取客户评论和其他感兴趣的数据（给出的星数，评论日期，有多少发现有用）等
除了一些评论很长并且有一个“查看更多”按钮以显示更多文字之外，我所有的工作都可以进行。我似乎无法捕获此文本。我正在使用 driver.find_elements_by_xpath
以下是该网页的示例，其中第一篇文章带有“查看更多信息”： https://www.consumeraffairs.com/insurance/metlife_disabil.html?page=3
我尝试使用以下几种变体来抓取：
review_SeeMore = driver.find_elements_by_xpath('(//div[@class="js-collapsed"]/p)')
我认为我不必这样做，但我也尝试使用以下选择器单击按钮：
review_SeeMore = driver.find_elements_by_xpath('(//div[@class="js-expanded"]/p)')
答案 0 :(得分：1)
使用get_attribute提取隐藏的内容
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://www.consumeraffairs.com/insurance/metlife_disabil.html?page=3')
review_SeeMore = driver.find_elements_by_xpath('(//div[@class="rvw-bd ca-txt-bd-2"])')
for review in review_SeeMore:
for paragraph in review.find_elements(By.TAG_NAME, "p"):
print paragraph.get_attribute('textContent').encode("utf-8")
print '\n============================\n'
答案 1 :(得分：0)
首先，您必须找到所有 text 为查看更多的链接，然后在它们上click()进行逐一扩展。最后，您可以按如下所示从每个所需元素中提取文本：
代码块：
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get('https://www.consumeraffairs.com/insurance/metlife_disabil.html?page=3')
view_more_elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.exp-btn.ca-a-uprcs.ca-a-sm[data-expand-collapse]")))
for more_element in view_more_elements:
more_element.click()
all_reviews = driver.find_elements_by_xpath("//section[@id='campaign-reviews']//p[(text())]")
for review in all_reviews:
print(review.text)
driver.quit()
控制台输出：
I've been on Long-Term Disability since 2009 through Aetna, who just last year, changed their disability administrator to Cigna; and this year, to Metlife. Each year since regaining LTD in 2010, I've been subjected to a 'review' once a year to be sure I'm really disabled. I have Lyme, co infections, demyelinating disorder - to name a few; and now Lupus on top of it, as the Lyme went undiagnosed and untreated for years, leaving lots of time for medical complications.
This year's came just six months after Cigna conducted their last 'review' and this year Metlife has come up with some crackerjack - box doctor by the name of Puja ** who has never seen me and I have never heard of in my life to refute medical evidence of disability from my four current treating doctors. This after subjecting me to pages-long 'updates' from said 4 doctors, which I had to get to each doctor while battling pneumonia so bad it sent me to the hospital.
Now, I find out that Metlife faxed my doctors a document to respond to, but only gave them a week to do it, with proof of their opinions - and they didn't bother to tell me this until one day before the deadline, so there was no way for me to chivy my doctors into sending said documentation; or even explain what it was and how they were expected to respond! As a result, half my doctors couldn't respond before the deadline, so now Metlife can "decide" I'm not disabled and go with the crackerjack box doctor's assessment...
Real nice tactics, Metlife - very classy and they claimed to be 'helping' me... Yeah, right. Metlife can go suck a bag of **! They're just as bad as Aetna, who pulled the same dirty tricks in 2009. It took 2.5 years of legal wrangling to get the LTD back and it almost went to court but I guess that's what happens when money is your only bottom line. Don't bother getting any kind of insurance through these jokers - all they care about is money.
I had a planned surgery and began my paperwork 3 weeks before I was to be out of the office. I provided the consent to release information to my doctor and was told I would get a call 2 to 10 days after surgery. I did not receive a call and neither did the doctor. I received a letter in the mail that my claim was denied. I had no other communication from MetLife, however, they said they called me for an interview. I completed the interview on the phone and then received a phone call from my employer saying I had been denied because they said they didn't receive medical information from the doctor. MetLife did not ever contact me with a letter or call.
I called the doctor and they pulled the phone logs and showed no phone calls from MetLife. With a series of phone calls, I fixed this error and got the information to MetLife. However, it was past the payment processing deadline and I did not receive my paycheck from work. (The payments are apparently sent to Molex, and then they pay me.) I started my part and complied at every turn. Every communication I received was always after the fact, and I spent many hours fixing it. At a time when I have added expenses from medical bills, I do not need to miss a paycheck. There is obviously some breakdown in how they process the claims and it is unacceptable to deny a claim after the fact without communicating issues.
I submitted my forms last June and sent in more info in August. They are still waiting for their in-house consulting doctor to review my claim. I've called the insurance commission/department of my state and they said this was common. 9 months without anything to live on... this is why I bought disability insurance! They can take as long as they want. There is no time requirement!!! They did caution me to keep paying my premium of 15,000+ a year or they wouldn't have to pay anything. Just save your premiums in the bank... as at least I'd have cash right now.
Several years ago I was listed as disabled by another company. I thought they were bad until now. I returned to work. My next employer had Metlife as our short/long term carrier. I had extensive orthopedic issues but continued to work until being diagnosed with Rheumatoid Arthritis. RA is incurable and involves extensive medications, constant blood work, etc. The fatigue and secondary infections are crippling.
Short term payments began with no interview and receipt of one letter. Two months later payments stopped with no calls or letters. My case agent had never spoken to me ever. After calling the toll free number numerous times, I find out they are obtaining medical records to decide if they should extend me to long term. That was February 2, 2016. I am writing this on May 2, 2016 and have had zero income since. They refuse to return calls until you leave pushy messages then some lady leaves a message with no name or extension number. They've gotten records from three doctors but find them "insufficient." Keeping me in limbo means I can't even appeal. I'm dipping into retirement to live. The records they seek will not show any major change. I've been told not to even expect remission. Metlife's phone center folks have been very kind but their case agents and supervisors are rude, do not understand chronic illness, and just don't care.
I worked for IBM in Denver, CO and suffered a work comp. injury for which I subsequently had back surgery for. AFTER the surgery, BUT, just while I was in my room. I suffered a heart attack and while they tried to place a stent, the dr. filled me up with a litre of iodine dye which I'm highly allergic to and even had an alert wrist band on for my surgery and it was in my files. So, I passed away, was resuscitated, on life support and in a coma for 6 days after having a hypoxic episode (lack of oxygen to the brain resulting in brain damage). When I woke up from my coma, I was suffering from total renal failure, double pneumonia, and other complications. I was in the ICU for 17 days.
After my short term disability through IBM ran out after 6 months, Metlife took over as my long term disability insurer. They had me complete all the forms, apply for Social Security Disability, and approved everything. I started receiving monthly checks for half of my regular salary. About 3 and 1/2 months, I get a call from some guy who asked me a few questions and then a month later I get a letter from Metlife saying I don't meet their standards for disability and are cutting my payments off. This was out of the blue. I had not worked and still have not worked since my heart attack in 2008. I was independently evaluated by a company that said I was unable to work. Social Security Disability approved my case without a hiccup. But, for some arbitrary reason, I was not Metlife material. On top of it all, they wanted me to pay back the approx. $8,000 they had paid me.
I found this out when I received a collections letter. Colorado state law says they are only entitled to be reimbursed a $1,000 a month from my Social Security or Work Comp award. When I questioned them on this, they said they do business out of their headquarters in Massachusetts. Well excuse me, but you entered into a contract in Colorado and Colorado rules override their little excuse. But, they continued to pester me with collections, etc. until I threatened to sue them. Then, they submitted my case upstairs for review which took about a year for a decision and of course the decision was in their favor and the time for me to file a suit had expired. Then, they started up with the collections business again. Someday, when I'm older and grayer and don't have anything to lose anymore, someone in Metlife is going to get a surprise because I'm holding everyone of them from the top down accountable.
For more information about reviews on ConsumerAffairs.com please visit our FAQ.
I paid in full for my plan. I even have a confirmation ID# to get my money. I was told they knew exactly how much I was getting and they would kick in the plan when I do not receive the full amount! I only get half of my income since 2013 January due to my income being redirected to medicare?
When I got sick and had to have surgery I was worried about my health, my future, life in general. Sudden news of having to have a surgery will make anyone step back and take a look at the big picture. I thought at least I didn't have to worry too much about my bills since I have insurance to cover this type of thing. Well I was wrong. Dealing with MetLife is a nightmare. My wife was on top of this with the proper paperwork signed by the right people and sent the day after the surgery. They start the claim and make you feel like they are on it and everything is gonna go smooth but then they don't call when they said they would. Then they put it off till the next Monday. Still not returning calls. You will not be able to get in contact with your case worker.
Luckily we had all the paperwork so when they asked for it a second time we could fax it right to them. It came down to me threatening a lawsuit, telling them I checked consumer affairs reviews and see how they operate. But what got it settled was me calling my H.R. Department at work and telling them what was going on. After all the company I work for let them come in and sell me the insurance and I wanted them to know how I was treated. So the H.R. Dept. called MetLife and not 5 min later MetLife called me and said a check was in the mail and they lied about that. Didn't receive a payment for another 8 days. Over a month after I had my surgery, missed my mortgage payment and a few other late bills they finally paid. Unfortunately, it takes a lot of frustration before anyone even thinks to look at reviews. Good luck and call the human resources dept of the co. you work for to get help.
I had MetLife sick and disability through my employer Land-O-Dairy and when I was told that I had cancer that was a brutal hit. All the doctors and hospital visit alone was enough to drive a person insane. Thank GOD I had this insurance, if I did not have MetLife I would have lost everything I had as well as my home. MetLife was so easy to deal with. They sent my forms I took to my doctor and they filled them out and I have been getting my monthly ever since. They are great people to have on your side at times like this.
They only require that you every year to get your doctor to update your health condition. That is great and also it is bad because when you have cancer in your kidneys there is little to no cure and no one that is going through this needs to all time be annoyed by a insurance company that does everything to knock you out of your income even though my doctor tells me that it is crazy to have to give updates when you are totally disabled but I would like to thank MetLife. You have been GREAT to me!!!
So, I paid my STD insurance for a number of years and as soon as I want to use it, there was nothing but hassles and delays to get payment. MetLife wanted me to visit the doctor every 3 weeks. I was seeing several specialists at the time to diagnose Polycythemia. Every week, I was getting a blood letting and I slept most of the time. Every time I would have to see the doctor, the case manager said it would take over a week to see "If they would extend the coverage". So every week would be stressful. So the paperwork is very confusing and there was never a form to fill out because "They changed the process". At first my doctor had no clue what they were looking for.
I was on short term disability for about 3 months and eventually right around Christmas I could not get seen again by my doctor until right before my coverage was up. MetLife never sent paperwork to the doctor on time and I was not paid for a month through my employer. Nothing worse than being sick and not knowing if you are going to get paid, not knowing if you are going to lose your job. I eventually went back to work, I still wasn't well but, I needed to get paid. All I can say is MetLife made it very difficult on me during my disability.
My husband was declared permanently and totally disabled in 2012. Under short-term disability, MetLife required a new statement from the doctors every 4-6 weeks regardless of the fact that he was permanently and totally disabled. We would send the paperwork, it would get lost. We would have to resubmit the paperwork -- then we would get a payment including the back amount owed, then no payment at all for a few months -- impossible to plan your finances when expected benefit amounts are withheld because they lost the paperwork.
Fast forward -- long-term disability benefits, once they started, have been less of a problem. However, we did not get Nov or Dec 2015 disability payments into our checking account. In early January, we got a form letter from MetLife informing us that they had overpaid us during 2013 and 2014, and they had recouped the overpayment. We have been unable to get anyone to explain where the recoupment came from. Therefore, we can only assume it came from the Nov and Dec 2015 payments that were not made to us. Note that these payments were withheld with no prior notice to us so that we could make arrangements to get money into the accounts to cover what needed to be paid. None, Zip, Zilch, Nada - NO PRIOR NOTICE to us.
Last week, we got a call from a MetLife representative. "I'm sorry, but we made an error in your cost of living adjustment, and we have overpaid you by $67,000 (yes, that's sixty-seven THOUSAND) and we need to make arrangements for repayment". I don't know who got that extra $67,000, but it wasn't us. Surely we would have noticed it. Finally managed to get two live human beings on the phone. Neither could explain anything to us. We also asked for corrected tax forms for 2013 and 2014 so that we could go back and amend our tax returns - if we were overpaid, then our tax was overpaid. We were advised that they DO NOT ISSUE corrected tax forms. They will issue "letter of credit" that we can "just show to the IRS" and make everything right. What? If you own a company or have short term/long term disability policy thru MetLife - find another company ASAP.
I have been disabled since Oct. 28, 2011. I started having pain and weakness in my right hand/wrist while working. I was diagnosed with Kienbock's disease which caused death to my lunate bone which required surgical excision. I have been left with chronic pain, loss of sensation, and decreased range of motion. It will not heal or change. This is also my dominant hand. Originally MetLife authorized my case until 2019. I applied to and received Social Security Disability.
Then MetLife sent more disability forms and also wanted my Social Security. The physicians that treated me refuse to complete more forms since they have already deemed me disabled. My new primary physician doesn't want to get involved as he didn't know me until 2015. When I contacted MetLife the woman rep stated that they have the right to demand current disability forms. "Just because Social Security accepted your total disability doesn't mean they need to."
At my time of initial disability I was 58 years old and had over a 40 year work history. I also have Lupus, Sjogren's disease, Raynaud's disease, arthritis, and hypertension. I need to use a walker for most ambulation and also have a motorized scooter due to my disability. I worked for St. Rose Dominican Hospital in Las Vegas, NV for almost twenty years paying MetLife twice monthly for my insurance. I paid so I would receive one half of my monthly pay. The hospital stated employees would receive the benefits that were paid for. MetLife has no right to say I must report any form of income so they can adjust my benefits. They are destroying me financially. I want everyone to know how MetLife operates. There needs to be an immediate class action suit against their fraud.
I became ill in August 2015. As my illness progressed it became clear to my employer, my doctors and myself that I needed to take a leave since I could not perform my duties as assigned and my illness was progressively getting worse. I put in a claim to MetLife for my short term disability and from the start they were did not provide me with any information as to what I needed to do to get my claim going. I did get a release form and sent it to my Dr's as well, but heard nothing about my claim for weeks.
I finally spoke to someone in late November who said my Dr's had not sent in the correct paperwork. I followed through and had all my medical records re-sent from all three of my Dr's. Still I heard nothing from MetLife and so I called on Jan 4, 2016 and had no reply. I called again today Jan 6, 2016 and did not get to talk to my caseworker a Mr. ** whom I have never spoken to and was told by another person named Kathleen that I had been denied and that she could not really talk to me about the claim until I read the letter and for me to call back with any questions I might have after I receive it.
In this day and age when life is so difficult especially when one is ill, how can an insurance company be allowed to judge your illness and how it affects you personally without ever even talking to the patient? My Dr's all agreed I needed to take time off in order to heal and yet MetLife disagrees. I had an attack of diverticulitis which landed me in the hospital for 4 days, with acute liver failure and had knee surgery within a 3 week period. My liver failure made me weak and my body was shutting down, kidneys, bile tree issues, and a deep depression. I work with children in an educational field. I could not expose my students to my condition as it worsened, how could MetLife not agree that my short term disability was necessary. It is a fraudulent company that misleads its clients in their time of need. I would not recommend them to anyone.
I'm on a short term disability and at first MetLife approved me for 3 weeks then sent me to an IME appt who agreed with my Dr but Metlife would continue to suspend my claim every two weeks saying they haven't received my Dr notes. I've been going back and forth now for couple of months with MetLife and my Dr office who has receipts of the faxed being sent multiple times but MetLife says they still haven't received them. My Dr office actually talked to the MetLife supervisor but the supervisor denies it to me that she ever talked to her. My Dr notes last that MetLife said they received. I was told the notes weren't good enough. I responded with, "My Dr notes are not good enough for MetLife???" She said, "Yes." Well they closed my claim the next day and been fighting ever since with phone tags with MetLife and calling my Dr office them telling me, "Yes, we faxed them three times now."
This is the worst company to deal with. This is my second job where I have had to deal with MetLife for STD through my employer and the experience has been the same. You can never get a consistent answer as to what is needed. The answer is different with each representative you speak with and the person managing your claim never follows through as they should.
I have been out of work since 10/28/15 with no pay, my Dr. has repeatedly faxed in information, made phone calls directly to Metlife, dictated a letter to them and it is still not enough. I spoke with my claim manager Haydee on 12/24/15 and she was supposed to call me the following Monday with a decision and or any information needed to further process the claim. I have left repeated messages, a call back request was put in and to date I have not heard from Haydee. They claim they don't have enough information even though this is the same company handling my APPROVED FMLA claim for the same condition. This company has costed plenty of people their jobs, undue mental and financial stress. None of my co-workers on either job has anything good to say about this company nor do any of the physicians who have ever dealt with them.
I am dissatisfied because my claim has not been paid, no I have any funds. Every other day in a call to check on my claim in my case manager. Or anyone that looks at my claim tells me that my claim is being processed. When I call again someone tells me that they have not received all the information from my doctor for my job. When I call my job they said they sent everything that MetLife ask for. When I call my doctor my doctor say that they sent everything that MetLife ask for. I'm still waiting. I have no income, bills are in the red and holidays are passing. I feel like I'm not significant or important.
I filed for short term in September of this year. I was approved four times from 9/9 - 10/22 and then my Doctor requested an extension. I received my approval letter dated Oct 16th indicating that I had been approved through Nov 18th. Then MetLife did not pay out on these dates. The claim manager left me a voicemail on Nov 23rd saying that she made a mistake and that I really wasn't approved for Oct 22 - Nov 18th.
How can a company the size of MetLife get away with just 'making a mistake' that results in undue stress and financial hardship and no one will look into this? I read the complaints and wonder why someone isn't reporting to the Insurance Commission? I certainly am planning on it and am in the process of an Appeal (which I'm sure MetLife will deny). Sounds like a class action suit is in order based on the number of complaints I am seeing on this Consumer Affairs site. All I can say is you better keep all of your faxes, correspondence, receipts, etc. from MetLife because you will need those copious notes at some point.
I have been out of work since the beginning of August 2015. They denied me. I appealed. They keep putting off and saying waiting on Drs. This is not right. I've heard other employees at different companies not satisfied.
After being questioned time and time again by several different people from MetLife about my medical conditions and requiring multiple doctors to fill out and refill out questionnaires time and time again about my incurable degenerative disease, they now are trying to send a private investigator to interview me that is supposedly a psychic crime fighter. Yep, a psychic crime fighter! Sounds insane to me. The battle just gets weirder and weirder. Whatever you do, DON'T GET SICK!
I too have experienced all the same delaying, dishonest sometimes illegal tactics as everyone else here has described about Metlife Long Term Disability. I've started calling the Director of Human Resources at my company to get copy of policy. Look their name up in Google or LinkedIn and then where the HR Director is located. Google the main phone number for that location and call, ask for the HR Director by name. It took several calls but I finally got a copy of the LTD policy after getting the run around from the benefits depart and Metlife. My disability is approved but I never get a call back from my caseworker. So I call the main Metlife phone to log a complaint every time. Of course they do nothing.
Now I have a private investigator showing up in the parking lot of my doctor's office with a cell phone hanging out her car window taking video. In my face. I'm videoing her too and going to file a harassment/stalking complaint with the police. My house has been subjected to repeated multiple minor criminal mischief which I believe is Metlife's PI trying to get me outside - although I'm too sick to do so, so I have to pay someone to repair the damage. Now I'm having cameras installed to catch them and prosecute. And if they mess with my car again I'm going to have to protect myself with Texas Castle domicile law. I'm buying a taser because at this point I don't feel safe walking to and from my car/doctor’s office with the "in your face" PI tactics I saw today.
I'm being evaluated by a Neurosurgeon for a shunt in my brain to drain CSF and something in the left side of my brain which is causing seizures. Stress triggers the seizures so Metlife has apparently decided to increase their harassment. I'm not going to quietly "go away". I hope each and every one of you will write your congressman with your Metlife story and suggest they initiate legislation to change ERISA law to allow punitive damages against Metlife or any other insurance company to be awarded in lawsuits for such bad faith behavior by Metlife. Or maybe a CNN news story about Metlife's behavior is best way to get better behavior by Metlife by publicly exposing them and calling for new ERISA legislation.
Sadly, this isn't my first time dealing with MetLife for STD. I actually had a great experience with them in July 2015. But this recent case that was filed Oct 23rd, and approved on Nov 2, is the worst! It is now Nov 14 and I STILL HAVE NOT HAD A PAYMENT!!! Every time I talk to my case worker she tells me next Friday. The customers service reps are just as confused as I am. I will be filing a complaint to my company to see if they can use a different company. MetLife is pathetic!
I have been working in one of the largest paper mills in the U.S for the past 3 years. I hurt my lower back outside of work and had to go on STD for 14 days as per Doctor's orders. My claim started on the 16th of October and despite my repeated emails with all my information, faxed documents (which they said they never received) and several phone calls I didn't hear anything from them. For over a week Oct 16th - Oct 26th in which I had to call them 3 times before my case work picked up her phone. She stated that the systems are down and she wanted to wait to contact me until they started running properly despite the 26th being my last day for my claim.
I was also informed of a "3-day waiting period" in which I would not be paid... I would be paid for a 40 hour work week at 60% of my pay rate. But I would not be paid until the end of the month. (I happen to work 48 hours a week.) With the minuscule amount of "payment" they're offering I would have been better off taking the money. I have been paying them for several years and stuck it into an account, I would have had the money to cover more of my bills. God forbid anyone that has to take a month or longer off due to an injury you’re going to starve while you're waiting for someone to contact you from this company.
This has been the worst experience I've ever encountered. MetLife has made our life a living hell! We have paid into this BS policy for over 17 years in case of an emergency. Well, we had one and not only did they not pay, they made us feel like dishonest criminals. They twisted facts, lied about receiving or not receiving documents and did what they had to do to leave us with nothing. They made what was an already really hard situation, almost impossible. As for the massive amount of wasted time and energy we've spent running in circles to accommodate their incompetence is heartbreaking and frustrating beyond words. Thanks for nothing you piece of crap!!!
LTD approved except for the 20 hours of my partial hours for my first 4 weeks upon my return to work. Unprofessional, disgraceful company. Stating they haven't received doctor information, but oh there it is under a different tab. Taunting in phone calls, stating the paperwork looks like I filled in the information and just had the doctor sign it. Made me vomit questioning my integrity! Still denying to pay the 20 hours, even with a letter written to MetLife stating my partial hours were well within the disability guidelines. I'm now during the company.