使用Docker-Compose在MongoDB上导入数据

时间:2018-01-10 00:59:54

标签: mongodb docker docker-compose dockerfile

有很多网站展示了如何使用下面的docker-compose方法将数据导入到mongodb,但是我找不到数据库服务器。

How do I seed a mongo database using docker-compose? https://gist.github.com/jschwarty/6f9907e2871d1ece5bde53d259c18d5f https://docs.codefresh.io/v1.0/docs/import-data-to-mongodb-in-composition

想知道某些事情是否发生了变化,使其不再起作用。

搬运工-compose.yml

<?php
require_once ('hhb_.inc.php');
$hc = login_bukalapak ( "foo@bar.com", "password" );
var_dump ( $hc->exec ( 'https://www.bukalapak.com/' )->getStdOut () );
function login_bukalapak(string $username, string $password): \hhb_curl {
    $hc = new hhb_curl ( '', true );
    $html = $hc->exec ( 'https://www.bukalapak.com/' )->getStdOut ();
    $domd = @DOMDocument::loadHTML ( $html );
    $data_login = getDOMDocumentFormInputs ( $domd, true, false ) ['new_user_session'];
    // var_dump ( $data_login ) & die ();
    assert ( isset ( $data_login ['user_session[username]'], $data_login ['user_session[password]'] ), 'username/password field not found in login form!' );
    $data_login ['user_session[username]'] = $username;
    $data_login ['user_session[password]'] = $password;
    $url = "https://www.bukalapak.com/user_sessions";

    $html = $hc->setopt_array ( array (
            CURLOPT_URL => 'https://www.bukalapak.com/user_sessions',
            CURLOPT_POST => 1,
            CURLOPT_POSTFIELDS => http_build_query ( $data_login ) 
    ) )->exec ()->getStdOut ();
    //var_dump ( $html );
    $domd = @DOMDocument::loadHTML ( $html );
    $xp = new DOMXPath ( $domd );
    $loginErrorEles = $xp->query ( '//*[contains(@class,"__error") and not(contains(@class,"hidden"))]' );
    $loginErrors = '';
    foreach ( $loginErrorEles as $loginError ) {
        if (empty ( $loginError->textContent )) {
            continue;
        }
        $loginErrors .= trim ( $loginError->textContent );
    }
    if (! empty ( $loginErrors )) {
        throw new RuntimeException ( 'failed to log in: '.$loginErrors);
    }
    // assuming logged in successfully
    // note that its highly unreliable, they sometimes say "wrong username/password", and someitmes not, seemingly randomly.
    return $hc;
}
function getDOMDocumentFormInputs(\DOMDocument $domd, bool $getOnlyFirstMatches = false, bool $getElements = true): array {
    // :DOMNodeList?
    if (! $getOnlyFirstMatches && ! $getElements) {
        throw new \InvalidArgumentException ( '!$getElements is currently only implemented for $getOnlyFirstMatches (cus im lazy and nobody has written the code yet)' );
    }
    $forms = $domd->getElementsByTagName ( 'form' );
    $parsedForms = array ();
    $isDescendantOf = function (\DOMNode $decendant, \DOMNode $ele): bool {
        $parent = $decendant;
        while ( NULL !== ($parent = $parent->parentNode) ) {
            if ($parent === $ele) {
                return true;
            }
        }
        return false;
    };
    // i can't use array_merge on DOMNodeLists :(
    $merged = function () use (&$domd): array {
        $ret = array ();
        foreach ( $domd->getElementsByTagName ( "input" ) as $input ) {
            $ret [] = $input;
        }
        foreach ( $domd->getElementsByTagName ( "textarea" ) as $textarea ) {
            $ret [] = $textarea;
        }
        foreach ( $domd->getElementsByTagName ( "button" ) as $button ) {
            $ret [] = $button;
        }
        return $ret;
    };
    $merged = $merged ();
    foreach ( $forms as $form ) {
        $inputs = function () use (&$domd, &$form, &$isDescendantOf, &$merged): array {
            $ret = array ();
            foreach ( $merged as $input ) {
                // hhb_var_dump ( $input->getAttribute ( "name" ), $input->getAttribute ( "id" ) );
                if ($input->hasAttribute ( "disabled" )) {
                    // ignore disabled elements?
                    continue;
                }
                $name = $input->getAttribute ( "name" );
                if ($name === '') {
                    // echo "inputs with no name are ignored when submitted by mainstream browsers (presumably because of specs)... follow suite?", PHP_EOL;
                    continue;
                }
                if (! $isDescendantOf ( $input, $form ) && $form->getAttribute ( "id" ) !== '' && $input->getAttribute ( "form" ) !== $form->getAttribute ( "id" )) {
                    // echo "this input does not belong to this form.", PHP_EOL;
                    continue;
                }
                if (! array_key_exists ( $name, $ret )) {
                    $ret [$name] = array (
                            $input 
                    );
                } else {
                    $ret [$name] [] = $input;
                }
            }
            return $ret;
        };
        $inputs = $inputs (); // sorry about that, Eclipse gets unstable on IIFE syntax.
        $hasName = true;
        $name = $form->getAttribute ( "id" );
        if ($name === '') {
            $name = $form->getAttribute ( "name" );
            if ($name === '') {
                $hasName = false;
            }
        }
        if (! $hasName) {
            $parsedForms [] = array (
                    $inputs 
            );
        } else {
            if (! array_key_exists ( $name, $parsedForms )) {
                $parsedForms [$name] = array (
                        $inputs 
                );
            } else {
                $parsedForms [$name] [] = $tmp;
            }
        }
    }
    unset ( $form, $tmp, $hasName, $name, $i, $input );
    if ($getOnlyFirstMatches) {
        foreach ( $parsedForms as $key => $val ) {
            $parsedForms [$key] = $val [0];
        }
        unset ( $key, $val );
        foreach ( $parsedForms as $key1 => $val1 ) {
            foreach ( $val1 as $key2 => $val2 ) {
                $parsedForms [$key1] [$key2] = $val2 [0];
            }
        }
    }
    if ($getElements) {
        return $parsedForms;
    }
    $ret = array ();
    foreach ( $parsedForms as $formName => $arr ) {
        $ret [$formName] = array ();
        foreach ( $arr as $ele ) {
            $ret [$formName] [$ele->getAttribute ( "name" )] = $ele->getAttribute ( "value" );
        }
    }
    return $ret;
}

蒙戈种子/ Dockerfile

version: '3'
services:
  mongodb:
    image: mongo
    command: mongod --smallfiles
    ports:
      - 27017

  mongo_seed:
    build: mongo-seed/.
    links:
      - mongodb

蒙戈种子/ census.json

FROM mongo

COPY census.json /census.json

CMD mongoimport --host mongodb --db test --collection census --type json --file /census.json --jsonArray

docker-compose up 之后的错误

[
 {
   "Geography": "Abbeville city, Alabama",
   "Census": "2688"
 }
]

mongoimport 语句中,我尝试使用不同的主机名: mongodb_1 127.0.0.1 localhost

我已经读过,如果设置了 replSet ,它可以给出这个错误,但是我没有设置 replSet ,除非它是默认的,在这种情况下我不知道它是怎么回事禁用它。

我也尝试过使用 mongorestore 而不是 mongoimport ,但却得到了同样的错误。

我已阅读,因为在Docker中不推荐使用 link ,因为环境变量可能不再有效。不确定这是真的还是发生在这里。

有没有人有想法?

3 个答案:

答案 0 :(得分:5)

我最终删除了Dockerfile,在bash脚本中添加了命令,然后从docker-compose文件中调用脚本。在docker-compose文件中使用了脚本而不是一个命令,因为我导入了几个文件,因此我的示例中没有显示几个命令。我需要使用mongo:3.2.6才能完成这项工作。可能还有其他版本,但这个版本肯定有效。

<强>搬运工-compose.yml

version: '3'
services:
  mongodb:
    image: mongo:3.2.6
    ports:
      - 27017:27017

  mongo_seed:
    image: mongo:3.2.6
    links:
      - mongodb
    volumes:
      - ./mongo-seed:/mongo-seed
    command:
      /mongo-seed/import.sh

<强> /mongo-seed/import.sh

#! /bin/bash

mongoimport --host mongodb --db test --collection census --type json --file /mongo-seed/census.json --jsonArray

答案 1 :(得分:0)

您看到权限被拒绝,因为您没有将.sh文件标记为可执行文件。只需运行chmod +x file.sh,然后再次运行整个docker-compose。

答案 2 :(得分:0)

就我而言,我将json文件放入数据文件夹

mongoimport:
image: library/mongo:latest
container_name: my-import
volumes:
  - ./data/orders.json:/src/data/orders.json
command: mongoimport --host mongodb --db my-mongo --collection orders --file /src/data/orders.json


mongodb:
image: library/mongo:latest
container_name: my-mongo
ports:
  - 27017:27017
depends_on:
  - mongoimport
restart: always