使用Nutch我已经抓取了URL,抓取了数据并将输出转储为文本。现在我有文本数据,我想从中提取/删除地址信息。我怎样才能做到这一点 ?
如果我没有错,那么在这种情况下只有注册 - 我不会帮助我,我应该编写一个reg-ex,然后是一些代码逻辑。
任何人都可以帮我解决这个问题吗?
提前致谢。
示例文字的Pastebin网址: http://pastebin.com/n8Eftp2K
示例文字
Recno:: 0
URL:: http://hiltongardeninn3.hilton.com/en/hotels/alabama/hilton-garden-inn-auburn-opelika-AUOAPGI/offers/index.htm
ParseText::
Hotels Auburn, AL - Hilton Garden Inn Auburn Opelika Deals Skip to Content My Reservations My Reservations View Promotions Sign In Join / My Account Sign Out Show Sign In Form View/change a specific reservation: Your Confirmation # Last Name Find Find - OR - Sign in to view all reservations in your account Existing Travel Reservations: Hotel + Air + Car Reservation Air Itinerary Car Rental Details Close Your Next Stay: See all Digital Key Offered View/Edit Go To My Account Page Update your password regularly to keep your account safe. Your Last Stay: See all View Receipt Book Again Join Hilton HHonors™ Upgrade your account and earn points at over 3,600 hotels in 82 countries around the world. Join HHonors | | Close Search GI Skip brand navigation Find a Hotel Offers Meetings and Events About Hilton Garden Inn skip form Where are you going? City, airport, address, attraction, or hotel Arrival You are now focused on a datepicker field. Press the down arrow to enter the calendar table. Once focused on the table, press left or right to navigate days. Press up or down to navigate between weeks. Enter to select. Escape to close datepicker. Your arrival date must be within the next year. Departure You are now focused on a datepicker field. Press the down arrow to enter the calendar table. Once focused on the table, press left or right to navigate days. Press up or down to navigate between weeks. Enter to select. Escape to close datepicker. Your departure date must be within 4 months after your arrival date. Use flexible dates Use HHonors Points Rooms Adults (18+) Children Rooms 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26+ Adults in Room 1 1 2 3 4 Children in Room 1 0 1 2 3 4 Room 2 : Adults in Room 2 1 2 3 4 Children in Room 2 0 1 2 3 4 Room 3 : Adults in Room 3 1 2 3 4 Children in Room 3 0 1 2 3 4 Room 4 : Adults in Room 4 1 2 3 4 Children in Room 4 0 1 2 3 4 Room 5 : Adults in Room 5 1 2 3 4 Children in Room 5 0 1 2 3 4 Room 6 : Adults in Room 6 1 2 3 4 Children in Room 6 0 1 2 3 4 Room 7 : Adults in Room 7 1 2 3 4 Children in Room 7 0 1 2 3 4 Room 8 : Adults in Room 8 1 2 3 4 Children in Room 8 0 1 2 3 4 Room 9 : Adults in Room 9 1 2 3 4 Children in Room 9 0 1 2 3 4 *Best Price Guarantee | More Options Less Options Add special rate codes (AAA, AARP, etc) Find it Find it Check Rooms & Rates Promotion/Offer code: Group code: Corporate account: Travel agent AAA rate * AARP rate * Senior rate * Government / Military rates * * ID required at check-in Close Close close tab panel Offers Discover award-winning service, thoughtful amenities and great deals at HGI hotels. View all Offers Earn 2X Points Book early and save Bed N Breakfast Deal End of tab panel close tab panel Meetings & Events Host your next meeting or special event in one of our newly renovated hotels. Meetings & Events Meetings Weddings Planning Tools Small Meeting Packages End of tab panel close tab panel About Hilton Garden Inn We’re here to help you be successful with great service and complimentary amenities. Learn More about HGI Locations Search more than 640 Hilton Garden Inn hotels worldwide to find the right one for your next trip. See our locations New Hotels See our new hotels and find out where we’re scheduled to open soon. See all new hotels End of tab panel menu_item_property_offers AUOAPGI Hilton Garden Inn Auburn/Opelika 2555 Hilton Garden Drive , Auburn , Alabama , 36830 , USA TEL: +1-334-502-3500 FAX: +1-334-502-3572 Skip secondary navigation Hotel Home Hotel Details Amenities & Services Maps & Directions Rooms & Suites Plan an Event Special Offers Dining Things To Do Not what you're looking for? Find Nearby Hotels Our Promise We promise to do whatever it takes to ensure you're satisfied, or you don't pay. You can count on us. GUARANTEED. The Hilton Garden Inn Promise reflects our focus on hospitality and integrity. We are committed to providing an excellent hotel experience for every guest, every time. Not what you're looking for? Find Nearby Hotels HHonors Reward Category: 7 Find a Special Offer Arrival Departure Find Offers Find Offers Hotel Information Check-in: 3:00 pm Check-out: 12:00 pm Smoking: Non-Smoking A fee of up to 250 USD will be assessed for smoking in a non-smoking room. Please ask the Front Desk for locations of designated outdoor smoking areas. Parking: Self parking: (Complimentary) Valet: Not Available Pets: Service animals allowed: Yes Pets allowed: No Hotel Policies Where we are Find where we are located View the Maps & Directions Page Share Print Special Offers Sort by: Brand Book Date Compare offer Premium Wi-Fi Compare this offer , You can compare up to 4 offers Premium Wi-Fi Boost your speed with Premium Internet access <
代码:
private static Pattern regexPreciseData = Pattern.compile("(?:(no|NO|No|(?:(p|P)?\\.?\\s?(o|O)?\\.?\\s?((B|b)(o|O)(x|X))))?(?:\\s?\\.?\\#?\\/?\\:?\\-?\\s?)[0-9]\\/?\\:?)+(?:\\,?\\s?\\w+\\s?){2,5}(?:\\s?\\w+\\s?)\\-?\\s?(\\d{2,5})",Pattern.CASE_INSENSITIVE);
utils.getText(sb, root); // extract text
text = sb.toString();
sb.setLength(0);
Matcher addressPattern = regexPreciseData.matcher(text);
while (addressPattern.find()) {
int start = addressPattern.start();
int end = addressPattern.end();
sb.append(text.substring(start, end));
}
text = sb.toString();
从上面的代码中你可以看到我正在尝试将我的文本与正则表达式模式匹配,并仅从中删除地址信息。但我的正则表达式也匹配一些不相关的数据。许多人认为只有reg-ex才能准确到足以从文本中删除地址信息。
答案 0 :(得分:1)
使用信息提取库或框架检测地址。对此有很多工作,这总是比在文本上编写正则表达式更好。
例如,您可以利用ANNIE附带的helper,这是一个可以提取地址的简单IE管道。一种方法是使用GATE并在其中运行GATE或导出为GATE格式,以便您可以单独运行GATE。您还可以在GATE模块中为Behemoth搭载代码并为Nutch编写自定义解析器,以便在Nutch中完成提取。
还有其他NLP资源可以做到这一点,检查UIMA等...再次,它是自然语言处理领域的一项着名的运动,你不需要重新发明轮子。
您还应该查看Behemoth并为Nutch编写自定义ParseFilter来处理直接从Nutch注释微数据的页面。
希望这会有所帮助。