Stix,Taxii,Python3,Cabby API - 将数据转换为我可以使用的格式

时间:2017-10-09 20:48:04

标签: python python-3.x

我正在使用cabby API:https://github.com/EclecticIQ/cabby

希望通过taxii客户提取stix信息。

我的python代码从www.hailataxii.com获取数据

数据在容器中..我可以翻阅它...它看起来像,但没有xml解析器会读取或操作数据。我喜欢将每条记录放入字典中,然后将数据放入某种数据库中......但在我找到从下载中访问数据的方法之前,我很茫然。数据或示例很少。

任何建议都将不胜感激。

以下是我测试的基本代码:

import pprint

from cabby import create_client

HailATaxiiFeedList=[
    'guest.Abuse_ch',
    'guest.CyberCrime_Tracker',
    'guest.EmergingThreats_rules',
    'guest.Lehigh_edu',
    'guest.MalwareDomainList_Hostlist',
    'guest.blutmagie_de_torExits',
    'guest.dataForLast_7daysOnly',
    'guest.dshield_BlockList',
    'guest.phishtank_com'
]

client = create_client(
    'hailataxii.com',
    use_https=False,
    discovery_path='/taxii-discovery-service')

print (": Discover_Collections:")
services = client.discover_services()
for service in services:
    print('Service type= {s.type} , address= {s.address}' .format(s=service))

print (": Get_Collections:")
collections = client.get_collections(
    uri='http://hailataxii.com/taxii-data')

for collection_name in HailATaxiiFeedList:
    print ("Polling :", collection_name, ".. could take a while, please be patient..")
    file = open(("./iocs/"+ collection_name + ".xml"), "w")
    content_blocks = client.poll(collection_name=collection_name)

    count =1
    for block in content_blocks:
        taxii_message=block.content.decode('utf-8')
        file.write(taxii_message)
        count+=1
        if count > 20: # just getting the 20 top objects because the lists are huge
            break
    file.close()

输出看起来像xml,但没有xml解析器会触及它。

<stix:STIX_Package xmlns:cyboxCommon="http://cybox.mitre.org/common-2" xmlns:cybox="http://cybox.mitre.org/cybox-2" xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:simpleMarking="http://data-marking.mitre.org/extensions/MarkingStructure#Simple-1" xmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1" xmlns:TOUMarking="http://data-marking.mitre.org/extensions/MarkingStructure#Terms_Of_Use-1" xmlns:opensource="http://hailataxii.com" xmlns:edge="http://soltra.com/" xmlns:indicator="http://stix.mitre.org/Indicator-2" xmlns:ttp="http://stix.mitre.org/TTP-1" xmlns:stixCommon="http://stix.mitre.org/common-1" xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" id="edge:Package-a2c8f8f2-5a4d-4f0e-92be-d3fa482247d0" version="1.1.1" timestamp="2017-10-09T20:39:36.179672+00:00">
    <stix:STIX_Header>
        <stix:Handling>
            <marking:Marking>
                <marking:Controlled_Structure>../../../../descendant-or-self::node()</marking:Controlled_Structure>
                <marking:Marking_Structure xsi:type="tlpMarking:TLPMarkingStructureType" color="WHITE"/>
                <marking:Marking_Structure xsi:type="TOUMarking:TermsOfUseMarkingStructureType">
                    <TOUMarking:Terms_Of_Use>zeustracker.abuse.ch | Abuse source[https://sslbl.abuse.ch/blacklist/] - As for all abuse.ch projects, the use of the SSL Blacklist is free for both commercial and non-commercial usage without any limitation. However, if you are a commercial vendor of security software/services and you want to integrate data from the SSL Blacklist into your products / services, you will have to ask for permission first by contacting me using the contact form [http://www.abuse.ch/?page_id=4727].'
</TOUMarking:Terms_Of_Use>
                </marking:Marking_Structure>
                <marking:Marking_Structure xsi:type="simpleMarking:SimpleMarkingStructureType">
                    <simpleMarking:Statement>Unclassified (Public)</simpleMarking:Statement>
                </marking:Marking_Structure>
            </marking:Marking>
        </stix:Handling>
    </stix:STIX_Header>
    <stix:Indicators>
        <stix:Indicator id="opensource:indicator-00398022-0d9c-474b-b543-31b85a4f22ab" timestamp="2014-10-31T16:44:24.766014+00:00" xsi:type="indicator:IndicatorType" version="2.1.1">
            <indicator:Title>ZeuS Tracker (offline)| s-k.kiev.ua/html/30/config.bin (2014-10-13) | This domain has been identified as malicious by zeustracker.abuse.ch</indicator:Title>
            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">Domain Watchlist</indicator:Type>
            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">URL Watchlist</indicator:Type>
            <indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">File Hash Watchlist</indicator:Type>
            <indicator:Description>This domain s-k.kiev.ua has been identified as malicious by zeustracker.abuse.ch. For more detailed infomation about this indicator go to [CAUTION!!Read-URL-Before-Click] [https://zeustracker.abuse.ch/monitor.php?host=s-k.kiev.ua].</indicator:Description>
            <indicator:Observable idref="opensource:Observable-94ead651-1df5-4cfe-b4bb-e34ce5e60224">
            </indicator:Observable>
            <indicator:Indicated_TTP>
                <stixCommon:TTP idref="opensource:ttp-6055672f-ecfd-40ae-aa84-0b336a5accb6" xsi:type="ttp:TTPType"/>
            </indicator:Indicated_TTP>
            <indicator:Producer>
                <stixCommon:Identity id="opensource:Identity-3066ae12-3db6-44dd-9636-6b083b6479dc">
                    <stixCommon:Name>zeustracker.abuse.ch</stixCommon:Name>
                </stixCommon:Identity>
                <stixCommon:Time>
                    <cyboxCommon:Produced_Time>2014-10-13T00:00:00+00:00</cyboxCommon:Produced_Time>
                    <cyboxCommon:Received_Time>2014-10-20T19:29:30+00:00</cyboxCommon:Received_Time>
                </stixCommon:Time>
            </indicator:Producer>
        </stix:Indicator>
    </stix:Indicators>

任何建议都将不胜感激。

1 个答案:

答案 0 :(得分:2)

您正在查看的XML是STIX。退房:https://www.eclecticiq.com/stix-taxii。然后点击STIX网站的链接,找到(右下角)&#34;工具&#34;部分。您应该找到库和解析工具以使其有用。

或者,有商业平台可用于进行数据处理魔术。 Google&#34;威胁情报平台&#34;。

干杯, 的Joep

方正 EclecticIQ