您好我已经使用python库来收集主题的数据。例如，我选择了纽约的主题，并使用以下代码检索了内容：
import wikipedia
f2 = open('newyork', 'w')
ny = wikipedia.page("New York")
f2.write(ny.content.encode('utf8')+"\n")
我能够以下面的格式提取信息：
New York is a state in the Northeastern United States and is the 27th-most extensive, fourth-most populous, and seventh-most densely populated U.S. state. New York is bordered by New Jersey and Pennsylvania to the south and Connecticut, Massachusetts, and Vermont to the east. The state has a maritime border in the Atlantic Ocean with Rhode Island, east of Long Island, as well as an international border with the Canadian provinces of Quebec to the north and Ontario to the west and north. The state of New York, with an estimated 19.8 million residents in 2015, is often referred to as New York State to distinguish it from New York City, the state's most populous city and its economic hub.
With an estimated population of 8.55 million in 2015, New York City is the most populous city in the United States and the premier gateway for legal immigration to the United States. The New York City Metropolitan Area is one of the most populous urban agglomerations in the world. New York City is a global city, exerting a significant impact upon commerce, finance, media, art, fashion, research, technology, education, and entertainment, its fast pace defining the term New York minute. The home of the United Nations Headquarters, New York City is an important center for international diplomacy and has been described as the cultural and financial capital of the world, as well as the world's most economically powerful city. New York City makes up over 40% of the population of New York State. Two-thirds of the state's population lives in the New York City Metropolitan Area, and nearly 40% live on Long Island. Both the state and New York City were named for the 17th century Duke of York, future King James II of England. The next four most populous cities in the state are Buffalo, Rochester, Yonkers, and Syracuse, while the state capital is Albany.
The earliest Europeans in New York were French colonists and Jesuit missionaries who arrived southward from settlements at Montreal for trade and proselytizing. New York had been inhabited by tribes of Algonquian and Iroquoian-speaking Native Americans for several hundred years by the time Dutch settlers moved into the region in the early 17th century. In 1609, the region was first claimed by Henry Hudson for the Dutch, who built Fort Nassau in 1614 at the confluence of the Hudson and Mohawk rivers, where the present-day capital of Albany later developed. The Dutch soon also settled New Amsterdam and parts of the Hudson Valley, establishing the colony of New Netherland, a multicultural community from its earliest days and a center of trade and immigration. The British annexed the colony from the Dutch in 1664. The borders of the British colony, the Province of New York, were similar to those of the present-day state.
Many landmarks in New York are well known to both international and domestic visitors, with New York State hosting four of the world's ten most-visited tourist attractions in 2013: Times Square, Central Park, Niagara Falls (shared with Ontario), and Grand Central Terminal. New York is home to the Statue of Liberty, a symbol of the United States and its ideals of freedom, democracy, and opportunity. In the 21st century, New York has emerged as a global node of creativity and entrepreneurship, social tolerance, and environmental sustainability. New York's higher education network comprises approximately 200 colleges and universities, including Columbia University, Cornell University, New York University, and Rockefeller University, which have been ranked among the top 35 in the world.
== History ==
=== 16th century ===
In 1524, Giovanni da Verrazzano, an Italian explorer in the service of the French crown, explored the Atlantic coast of North America between the Carolinas and Newfoundland, including New York Harbor and Narragansett Bay. On April 17, 1524 Verrazanno entered New York Bay, by way of the Strait now called the Narrows into the northern bay which he named Santa Margherita, in honour of the King of France's sister. Verrazzano described it as "a vast coastline with a deep delta in which every kind of ship could pass" and he adds: "that it extends inland for a league and opens up to form a beautiful lake. This vast sheet of water swarmed with native boats". He landed on the tip of Manhattan and perhaps on the furthest point of Long Island. Verrazanno's stay in this place was interrupted by a storm which pushed him north towards Martha's Vineyard.
In 1540 French traders from New France built a chateau on Castle Island, within present-day Albany; due to flooding, it was abandoned the next year. In 1614, the Dutch under the command of Hendrick Corstiaensen, rebuilt the French chateau, which they called Fort Nassau. Fort Nassau was the first Dutch settlement in North America, and was located along the Hudson River, also within present-day Albany. The small fort served as a trading post and warehouse. Located on the Hudson River flood plain, the rudimentary "fort" was washed away by flooding in 1617, and abandoned for good after Fort Orange (New Netherland) was built nearby in 1623.
=== 17th century ===
Henry Hudson's 1609 voyage marked the beginning of European involvement with the area. Sailing for the Dutch East India Company and looking for a passage to Asia, he entered the Upper New York Bay on September 11 of that year. Word of his findings encouraged Dutch merchants to explore the coast in search for profitable fur trading with local Native American tribes.
During the 17th century, Dutch trading posts established for the trade of pelts from the Lenape, Iroquois, and other tribes were founded in the colony of New Netherland. The first of these trading posts were Fort Nassau (1614, near present-day Albany); Fort Orange (1624, on the Hudson River just south of the current city of Albany and created to replace Fort Nassau), developing into settlement Beverwijck (1647), and into what became Albany; Fort Amsterdam (1625, to develop into the town New Amsterdam which is present-day New York City); and Esopus, (1653, now Kingston). The success of the patroonship of Rensselaerswyck (1630), which surrounded Albany and lasted until the mid-19th century, was also a key factor in the early success of the colony. The English captured the colony during the Second Anglo-Dutch War and governed it as the Province of New York. The city of New York was recaptured by the Dutch in 1673 during the Third Anglo-Dutch War (1672–1674) and renamed New Orange. It was returned to the English under the terms of the Treaty of Westminster a year later.
== References ==
== Further reading ==
French, John Homer (1860). Historical and statistical gazetteer of New York State. Syracuse, New York: R. Pearsall Smith. OCLC 224691273. (Full text via Google Books.)
New York State Historical Association (1940). New York: A Guide to the Empire State. New York City: Oxford University Press. ISBN 978-1-60354-031-5. OCLC 504264143. (Full text via Google Books.)
== External links ==
New York at DMOZ
Geographic data related to New York at OpenStreetMap
问题：
问题1：
我在尝试删除“参考和进一步阅读”部分中的所有内容时遇到了麻烦
例如：
== History ==
some text under the section History
=== 17th century ===
some text under the section 17 century
=== 19th century ===
some text under the section 19 century
== References ==
some references
== Further reading ==
some further reading sources
期望的结果：
== History ==
some text under the section History
=== 17th century ===
some text under the section 17 century
=== 19th century ===
some text under the section 19 century
问题1B：
我将获得许多主题的内容，因此会有很多引用删除，我该怎么办？
例如，我想删除所有以“参考”和“进一步阅读”开头的部分：
== New York ==
== References ==
== Further reading ==
== California ==
== References ==
== Further reading ==
== Floria ==
== References ==
== Further reading ==
期望的结果：
== New York ==
== California ==
== Floria ==
对不起，很长的帖子请原谅我，因为我对python知之甚少。
非常感谢所有的建议和帮助。
谢谢。
修改
当前问题
嗨osantana，
我已经尝试了您提供的代码，如下所示：
import wikipedia
import re
f2 = open('osantana', 'w')
ny = wikipedia.page("New York")
section_title_re = re.compile("^=+\s+.*\s+=+$")
raw_content = ny.content
content = []
skip = False
for l in raw_content.splitlines():
line = l.strip()
if "== References ==" in line.lower():
skip = True # replace with break if this is the last section
continue
if "== Further reading ==" in line.lower():
skip = True # replace with break if this is the last section
continue
if "== External links ==" in line.lower():
skip = True # replace with break if this is the last section
continue
if section_title_re.match(line):
skip = False
continue
if skip:
continue
content.append(line)
content = '\n'.join(content) + '\n'
f2.write(content.encode('utf8')+"\n")
除了这3部分外，它适用于所有人：
原始档案：
== References ==
Index of New York-related articles
Outline of New York – organized list of topics about New York
== Further reading ==
French, John Homer (1860). Historical and statistical gazetteer of New York State. Syracuse, New York: R. Pearsall Smith. OCLC 224691273. (Full text via Google Books.)
New York State Historical Association (1940). New York: A Guide to the Empire State. New York City: Oxford University Press. ISBN 978-1-60354-031-5. OCLC 504264143. (Full text via Google Books.)
代码结果：
Index of New York-related articles
Outline of New York – organized list of topics about New York
French, John Homer (1860). Historical and statistical gazetteer of New York State. Syracuse, New York: R. Pearsall Smith. OCLC 224691273. (Full text via Google Books.)
New York State Historical Association (1940). New York: A Guide to the Empire State. New York City: Oxford University Press. ISBN 978-1-60354-031-5. OCLC 504264143. (Full text via Google Books.)
标题被删除但内容仍然完整。
答案 0 :(得分：2)
我会假设参考/进一步阅读不是所有页面的最后部分。如果这些主题是最后一部分，请使用break命令替换下面突出显示的代码。
import re
def parse(raw_content):
section_title_re = re.compile("^=+\s+.*\s+=+$")
content = []
skip = False
for l in raw_content.splitlines():
line = l.strip()
if "= references =" in line.lower():
skip = True # replace with break if this is the last section
continue
if "= further reading =" in line.lower():
skip = True # replace with break if this is the last section
continue
if section_title_re.match(line):
skip = False
continue
if skip:
continue
content.append(line)
return '\n'.join(content) + '\n'
print(parse(ny.content))
答案 1 :(得分：1)
对于问题2 ，你可以做类似的事情
contents = re.sub('=+\s*.+\s*=+', '', contents)
请记住import re，正则表达式模块。
使用的方法是re.sub(pattern, repl, string)。 pattern是正则表达式模式*（re documentation提供了概述）。
repl是您想要替换所有出现的模式的内容。在这种情况下，您要删除模式，因此只需使用空字符串作为替换。
string当然是您正在执行替换的字符串。此方法返回最终结果，因此如果您想覆盖原始字符串，只需将返回的值返回给输入字符串。
以下是我用过的模式，以防万一。 '=+\s*.+\s*=+'表示字符串的任何部分，其中有一个或多个等号（=+），后跟零个或多个空格（\s*），后跟一个或多个任何字符（.+），后面再跟零个或多个空格（\s*），最后以一个或多个等号（=+）结束。
对于问题1 我会说你可能也可以完成你想要使用的正则表达式，而re模块让它变得非常简单。我上面给出的链接应该有所帮助。
答案 2 :(得分：0)
def clean_data(f):
def inner(word):
text=f(word)
text=text.encode("utf-8",errors='ignore').decode("utf-8")
text=re.sub("https?:.*(?=\s)",'',text)
text=re.sub("[’‘\"]","'",text)
text=re.sub("[^\x00-\x7f]+",'',text)
text=re.sub('[#&\\*+/<>@[\]^`{|}~ \t\n\r]',' ',text)
text=re.sub('\(.*?\)','',text)
text=re.sub('\=\=.*?\=\=','',text)
text=re.sub(' , ',',',text)
text=re.sub(' \.','.',text)
text=re.sub(" +",' ',text)
text=re.sub(";",'and',text)
return text.strip()
return inner
@clean_data
def get_data(word):
try:
data = wikipedia.summary("Orange",sentences=300)
except wikipedia.DisambiguationError as e:
print("picking the data from:",e.options[:3])
data=''.join([wikipedia.summary(s,sentences=100) for s in e.options[:3]])
return data
data=get_data("Orange")