大家早上好!
当前风景
我创建了一个API,该API使用selenium(Chrome驱动程序)从特定网站获取数据并将其保存到数据库中。我一直在尝试创建一个适合我的Docker。硒脚本可以在Docker外部完美运行。
Dockerinzing
因为我需要连接一个Mysql数据库(以在api中获取数据)并连接到Postgresql(以插入从硒中接收到的数据),所以我创建了一个docker-compose来连接它们。
这是我的docker-compose配置:
version: '2'
services:
postgres:
image: postgres:9.3
volumes:
- ./data/postgres:/var/lib/postgresql/data
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: db
POSTGRES_USER: user
POSTGRES_PASSWORD: pass
mysql:
image: mysql:5.7
volumes:
- /home/user/db/mysql/data:/var/lib/mysql
- /home/user/db/mysql/init:/docker-entrypoint-initdb.d/:ro
restart: always
environment:
MYSQL_ROOT_PASSWORD: passroot
MYSQL_USER: user
MYSQL_HOST: host
MYSQL_PASSWORD: pass
MYSQL_DATABASE: db
ports:
- "3307:3306"
web:
build: .
ports:
- "8080:8080"
并创建Docker,我在下面创建了Dockerfile:
FROM ubuntu:18.04
COPY pom.xml /usr/local/service/pom.xml
COPY src /usr/local/service/src
# Install required dependencies
RUN apt-get update
RUN apt-get install -y wget unzip curl dpkg
#Install Open JDK 8
RUN apt-get -y install software-properties-common
RUN add-apt-repository -y ppa:openjdk-r/ppa
RUN apt-get update
RUN apt-get -y install openjdk-8-jdk
# Install Chrome for Selenium
RUN curl https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb -o /chrome.deb
RUN dpkg -i /chrome.deb || apt-get install -yf
RUN rm /chrome.deb
# Install chromedriver for Selenium
RUN curl https://chromedriver.storage.googleapis.com/2.31/chromedriver_linux64.zip -o /usr/local/bin/chromedriver
RUN chmod +x /usr/local/bin/chromedriver
# Setup JAVA_HOME, this is useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV PATH $JAVA_HOME/bin:$PATH
# Install Maven
RUN java -version && ls -l /usr/bin/java
RUN apt-get update -y && apt-get install maven -y
RUN java -version && ls -l /usr/bin/java
WORKDIR .
COPY pom.xml /tmp/
COPY src /tmp/src/
WORKDIR /tmp/
RUN mvn package
WORKDIR .
RUN cp -a target/*.jar /app.jar
EXPOSE 8080
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
docker-compose已构建并正常运行,但是当我在api中进行请求时,收到以下消息错误:
{
"timestamp": "2019-04-24T12:36:19.023+0000",
"status": "500 INTERNAL_SERVER_ERROR",
"mensagem": "unknown error: Chrome failed to start: exited abnormally\n (unknown error: DevToolsActivePort file doesn't exist)\n (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\n (Driver info: chromedriver=73.0.3683.68 (47787ec04b6e38e22703e856e101e840b65afe72),platform=Linux 4.4.0-131-generic x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 1.47 seconds\nBuild info: version: 'unknown', revision: 'unknown', time: 'unknown'\nSystem info: host: '41cd92b21873', ip: '172.18.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-131-generic', java.version: '1.8.0_191'\nDriver info: driver.version: ChromeDriver"
}
我尝试了其他许多Chrome和Chrome驱动程序的版本,但这不起作用。
我在许多其他论坛中进行了搜索,他们向我建议了以下配置(我已经使用了chrome headless配置):
ChromeOptions options = new ChromeOptions();
options.addArguments("--headless");
options.addArguments("--no-sandbox");
options.addArguments("--disable-dev-shm-usage");
options.addArguments("--disable-extensions");
options.addArguments("--disable-gpu");
options.addArguments("disable-infobars");
但这对我不起作用!
有人可以帮助我解决这个问题或建议我做点更好的事情吗?
非常感谢!