使用共享端节点对边进行采样

时间：2017-05-29 18:20:37

标签： python graph machine-learning networkx

我正在tensorflow中实现一个NN训练算法，它需要从图形中对神经网络的输入进行采样，特别是我想创建从图形中取出的小批量边缘，从邻域区域中随机选择 - 即该共享一些终端节点。

现在，我有一个生成器，可以从图形中的所有边缘随机返回一批边。你对如何实现这个有什么建议吗？

我认为应该有一种方法可以使用networkx将图形划分为相邻边缘列表，然后从这些分区中随机选择，直到所有边缘都被采样...

提前感谢您对此

的任何意见

1 个答案:

答案 0 :(得分：2)

创建一个简单的图表：

partitions

按节点将图划分为相邻边列表：

import random
for partition in partitions:
    random.shuffle(partition)
    for edge in partition:
        print(edge)

正如您所看到的，$ scrapy version -v Scrapy : 1.4.0 lxml : 3.7.3.0 libxml2 : 2.9.3 cssselect : 1.0.1 parsel : 1.2.0 w3lib : 1.17.0 Twisted : 17.1.0 Python : 2.7.12+ (default, Sep 17 2016, 12:08:02) - [GCC 6.2.0 20160914] pyOpenSSL : 17.0.0 (OpenSSL 1.0.2g 1 Mar 2016) Platform : Linux-4.8.0-53-generic-x86_64-with-Ubuntu-16.10-yakkety $ scrapy shell http://www.instagram.com/nike/ 2017-05-30 10:06:39 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: scrapybot) (...) 2017-05-30 10:06:40 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.instagram.com/nike/> from <GET http://www.instagram.com/nike/> 2017-05-30 10:06:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.instagram.com/nike/> (referer: None) >>> response.xpath('//script[contains(., "sharedData")]') [<Selector xpath='//script[contains(., "sharedData")]' data=u'<script type="text/javascript">window._s'>] >>> response.xpath('string(//script[contains(., "sharedData")])').get() u'window._sharedData = {"activity_counts": null, "config": (...redacted...) "show_app_install": true};' >>> import js2xml >>> # get the JavaScript statements >>> js = response.xpath('string(//script[contains(., "sharedData")])').get() >>> # parse them with js2xml >>> jstree = js2xml.parse(js) >>> # locate the object we're interested in: >>> # here it's the right-part of the following assignement: >>> # |--- this bit >>> # window._sharedData = {"activity_counts":... >>> >>> obj = jstree.xpath('//assign[left//identifier[@name="_sharedData"]]/right/*')[0] >>> # convert this object to a Python dict >>> sharedData = js2xml.jsonlike.make_dict(obj) >>> >>> from pprint import pprint >>> # quite a lot of data... >>> pprint(sharedData) {'activity_counts': None, 'config': {'csrf_token': 'JzQ8ja9YlG1cKqwAbvwacTKhOc4FuBDT', 'viewer': None}, 'country_code': 'FR', 'display_properties_server_guess': {'pixel_ratio': 1.5, 'viewport_width': 360}, 'entry_data': {'ProfilePage': [{'logging_page_id': 'profilePage_13460080', 'user': {'biography': 'Just Do It.', 'blocked_by_viewer': False, 'connected_fb_page': None, 'country_block': False, 'external_url': 'http://nike.com/justdoit', 'external_url_linkshimmed': 'http://l.instagram.com/?u=http%3A%2F%2Fnike.com%2Fjustdoit&e=ATOPqaPHEBPK4ylZ6YZRynSTQdp28oGedfFWw3gLtqoII1ZhtlDhLI0vu9Udbjg', 'followed_by': {'count': 72447697}, 'followed_by_viewer': False, 'follows': {'count': 136}, 'follows_viewer': False, 'full_name': 'nike', 'has_blocked_viewer': False, 'has_requested_viewer': False, 'id': '13460080', 'is_private': False, 'is_verified': True, 'media': {'count': 897, 'nodes': [{'__typename': 'GraphVideo', 'caption': 'Eliud Kipchoge - 2:00:25\nThe barrier just got that much closer.\n#Breaking2 #JustDoIt', 'code': 'BTvxgGEAn-p', 'comments': {'count': 2274}, 'comments_disabled': False, 'date': 1494064175, 'dimensions': {'height': 640, 'width': 640}, 'display_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/18380674_1840678496198968_334757937257906176_n.jpg', 'id': '1508642110004428713', 'is_video': True, 'likes': {'count': 296763}, 'owner': {'id': '13460080'}, 'thumbnail_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/18380674_1840678496198968_334757937257906176_n.jpg', 'video_views': 1193309}, (...redacted...) {'__typename': 'GraphVideo', 'caption': u'We step onto the court as equals. Judged only by our performance. New Orleans sets the stage for All Star Weekend @nikebasketball.\xa0#EQUALITY #nike', 'code': 'BQlZFdugTIG', 'comments': {'count': 1172}, 'comments_disabled': False, 'date': 1487273379, 'dimensions': {'height': 360, 'width': 640}, 'display_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/16789854_771546756335957_8222167172487577600_n.jpg', 'id': '1451676781575746054', 'is_video': True, 'likes': {'count': 184595}, 'owner': {'id': '13460080'}, 'thumbnail_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/e15/c157.0.406.406/16789854_771546756335957_8222167172487577600_n.jpg', 'video_views': 627647}], 'page_info': {'end_cursor': 'AQCseCeUI_vJ_XzpTTIf7AhmwqbkBSf959347Wl03MM_ErpUQqP6RGjuJRdEcH1qJBCWvPGxUhpm6rPv5esAW5GvXHGodwmYA0eT51wnk8_-KA', 'has_next_page': True}}, 'profile_pic_url': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-19/s150x150/17126848_1779368432381854_3589478532054515712_a.jpg', 'profile_pic_url_hd': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-19/s320x320/17126848_1779368432381854_3589478532054515712_a.jpg', 'requested_by_viewer': False, 'username': 'nike'}}]}, 'environment_switcher_visible_server_guess': True, 'gatekeepers': {'bn': True, 'ld': True, 'pl': True}, 'hostname': 'www.instagram.com', 'language_code': 'en', 'platform': 'web', 'probably_has_app': False, 'qe': {'bc3l': {'g': '', 'p': {}}, 'create': {'g': '', 'p': {}}, 'deact': {'g': '', 'p': {}}, 'disc': {'g': '', 'p': {}}, 'ebd': {'g': 'holdout', 'p': {'use_new_styles': 'false'}}, 'feed': {'g': '', 'p': {}}, 'gql': {'g': '', 'p': {}}, 'nav': {'g': '', 'p': {}}, 'nav_lo': {'g': '', 'p': {}}, 'pm': {'g': '', 'p': {}}, 'poe': {'g': '', 'p': {}}, 'profile': {'g': '', 'p': {}}, 'sidecar': {'g': '', 'p': {}}, 'su_universe': {'g': '', 'p': {}}, 'ufi': {'g': '', 'p': {}}, 'ufi_loggedout': {'g': '', 'p': {}}, 'us': {'g': '', 'p': {}}, 'us_li': {'g': '', 'p': {}}, 'video': {'g': '', 'p': {}}}, 'show_app_install': True} >>> # nodes have a 'caption' key that looks like the "alt" attributes >>> pprint(sharedData['entry_data']['ProfilePage'][0]['user']['media']['nodes']) [{'__typename': 'GraphVideo', 'caption': 'Eliud Kipchoge - 2:00:25\nThe barrier just got that much closer.\n#Breaking2 #JustDoIt', 'code': 'BTvxgGEAn-p', 'comments': {'count': 2274}, 'comments_disabled': False, 'date': 1494064175, 'dimensions': {'height': 640, 'width': 640}, 'display_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/18380674_1840678496198968_334757937257906176_n.jpg', 'id': '1508642110004428713', 'is_video': True, 'likes': {'count': 296763}, 'owner': {'id': '13460080'}, 'thumbnail_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/18380674_1840678496198968_334757937257906176_n.jpg', 'video_views': 1193309}, {'__typename': 'GraphImage', 'caption': 'Eliud Kipchoge - 2:00:25\nThe barrier just got that much closer.\n#Breaking2 #JustDoIt', 'code': 'BTvj_-gg6B6', 'comments': {'count': 2269}, 'comments_disabled': False, 'date': 1494057096, 'dimensions': {'height': 1080, 'width': 1080}, 'display_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/e35/18299402_128309274385168_1749644873230712832_n.jpg', 'id': '1508582728264818810', 'is_video': False, 'likes': {'count': 484745}, 'owner': {'id': '13460080'}, 'thumbnail_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/18299402_128309274385168_1749644873230712832_n.jpg'}, (...redacted...) {'__typename': 'GraphVideo', 'caption': u'We step onto the court as equals. Judged only by our performance. New Orleans sets the stage for All Star Weekend @nikebasketball.\xa0#EQUALITY #nike', 'code': 'BQlZFdugTIG', 'comments': {'count': 1172}, 'comments_disabled': False, 'date': 1487273379, 'dimensions': {'height': 360, 'width': 640}, 'display_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/s640x640/e15/16789854_771546756335957_8222167172487577600_n.jpg', 'id': '1451676781575746054', 'is_video': True, 'likes': {'count': 184595}, 'owner': {'id': '13460080'}, 'thumbnail_src': 'https://scontent-cdg2-1.cdninstagram.com/t51.2885-15/e15/c157.0.406.406/16789854_771546756335957_8222167172487577600_n.jpg', 'video_views': 627647}] >>> 是按边缘分区的列表。

  

[[（1,2）]，

     

[（3,2），（3,4）]，

     

[（5,4），（5,6）]，

     

[（7,8），（7,6）]，

     

[（9,8）]]

现在，您想要从这些分区中随机选择，直到所有边缘都用完为止。这种方法将分区混合到位以实现随机性。

if (isset($_POST['username'])){

    $username = stripslashes($_POST['username']);

    $username = mysqli_real_escape_string($con,$username); 
    $email = stripslashes($_POST['email']);
    $email = mysqli_real_escape_string($con,$email);
    $password = stripslashes($_POST['password']);
    $password = mysqli_real_escape_string($con, $password);

    $password = md5($password);

    $geboorte_datum = $_POST['geboorte_datum'];
    $naam = $_POST['naam'];
    $adres = $_POST['adres'];
    $postcode = $_POST['postcode'];
    $woonplaats = $_POST['woonplaats'];

    $query = "
        INSERT INTO 
            `users` (id, username, password, email, geboortedatum, medewerker, volledige_naam, volledige_adres, postcode, woonplaats)
        VALUES 
            (NULL, '$username', '$password', '$email', '$geboorte_datum', '0', '$naam', '$adres', '$postcode', '$woonplaats')
    ";

    $result = mysqli_query($con,$query);
    if($result){
        echo "
            <div class='form'>
                <h3>Je bent succesvol geregistreerd</h3>
                <br/>Klik hier om <a href='inloggen.php'>in te loggen</a>
            </div>
        ";
    } else {
        echo "
            <div class='form'>
                <h3>Fout opgetreden</h3>
                <br/>Klik hier om <a href='registreren.php'>opnieuw te proberen</a>
            </div>
        ";
    }
}else{
?>
相关问题
最新问题