我需要运行一个需要GUI界面来启动和配置的应用程序。我还需要能够在亚马逊的EC2服务和EMR服务上运行此应用程序。 EMR要求意味着它必须在亚马逊的Linux AMI上运行。
经过广泛搜索后，我一直无法找到任何现成的解决方案，特别是要求在亚马逊的AMI上运行。最接近的匹配和最常引用的解决方案是here。不幸的是，它是在RHEL6实例上开发的，该实例与亚马逊的AMI有很大不同，解决方案不起作用。
我在下面发布我的解决方案。希望它可以节省一些其他人用于提出正确配方所需的数小时实验。
答案 0 :(得分：8)
这是我在亚马逊的AMI上运行GUI的解决方案。我使用此post作为起点，但必须进行许多更改才能使其在Amazon的AMI上运行。我还添加了额外的信息，以便以合理的自动化方式完成这项工作，因此需要不止一次启动此环境的个人可以毫不费力地完成此任务。
注意：我在这篇文章中包含了很多评论。我提前道歉，但我认为如果他们能够理解为什么会在路上做出各种选择，那么需要制作修改的人可能会有所帮助。
下面包含的脚本会沿途安装一些文件。有关这些脚本使用的文件和目录结构的列表，请参阅第4节。
执行“yum update”后，大多数解决方案都包含
这样的行sudo yum groupinstall -y "Desktop"
这个极其简单的步骤需要在Amazon AMI上付出更多努力。此组未在Amazon AMI中配置（AAMI从此处开始）。 AAMI默认安装并启用了亚马逊自己的存储库。还安装了epel repo，但默认情况下禁用它。启用epel后，我找到了桌面组，但没有填充包。我还发现了填充的Xfce（另一种桌面替代品）。最终我决定安装Xfce而不是Desktop。尽管如此，这并不是直截了当的，但它最终导致了解决方案。
值得注意的是，我尝试的第一件事就是安装centos存储库并从那里安装Desktop组。最初这看起来很有希望。该小组完全填充了包裹。但是，经过一番努力，我最终确定AAMI上已经安装的依赖项和软件包之间存在太多的版本冲突。
这导致我从epel repo中选择Xfce。由于epel repo已经安装在AAMI上，我认为与Amazon repos的依赖版本协调会更好。这通常是正确的。在epel repo或Amazon repos中发现了许多依赖项。对于那些没有的，我能够在centos repo中找到它们，并且在大多数情况下那些是叶子依赖。因此，大多数问题来自于centos repo中的少数依赖项，这些依赖项具有与亚马逊或epel repo冲突的子依赖项。最后，需要一些黑客来绕过依赖冲突。我试图尽可能地减少这些。这是安装Xfce的脚本
installGui.sh
#!/bin/bash
# echo each command
set -x
# assumes RSRC_DIR and IS_EMR set by parent script
YUM_RSRC_DIR=$RSRC_DIR/yum
sudo yum -y update
# Most info I've found on installing a GUI on AWS suggests to install using
#> sudo yum groupinstall -y "Desktop"
# This group is not available by default on the Amazon Linux AMI. The group
# is listed if the epel repo is enabled, but it is empty. I tried installing
# the centos repo, which does have support for this group, but it simply end
# up having to many dependency version conflicts with packages already installed
# by the Amazon repos.
#
# I found the path of least resistance to be installing the group Xfce from
# the epel repo. The epel repo is already included in amazon image, just not enabled.
# So I'm guessing there was at least some consideration by Amazon to align
# the dependency versions of this repo with the Amazon repos.
#
# My general approach to this problem was to start with the last command:
#> sudo yum groupinstall -y Xfce
# which will generate a list of missing dependencies. The script below
# essentially works backwards through that list to eliminate all the
# missing dependencies.
#
# In general, many of the dependencies required by Xfce are found in either
# the epel repo or the Amazon repos. Most of the remaining dependencies can be
# found in the centos repo, and either don't have any further dependencies, or if they
# do those dependencies are satisfied with the centos repo with no collisions
# in the epel or amazon repo. Then there are a couple of oddball dependencies
# to clean up.
# if yum-config-manager is not found then install yum-utils
#> sudo yum install yum-utils
sudo yum-config-manager --enable epel
# install centos repo
# place the repo config @ /etc/yum.repos.d/centos.repo
sudo cp $YUM_RSRC_DIR/yum.repos.d/centos.repo /etc/yum.repos.d/
# The config centos.repo specifies the key with a URL. If for some reason the key
# must be in a local file, it can be found here: https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# It can be installed to the right location in one step:
#> wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# Note, a key file must also be installed in the system key ring. The docs are a bit confusing
# on this, I found that I needed to run both gpg AND then followed by rpm, eg:
#> sudo gpg --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
#> sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
# I found there are a lot of version conflicts between the centos, Amazon and epel repos.
# So I did not enable the centos repo generally. Instead I used the --enablerepo switch
# enable it explicitly for each yum command that required it. This only works for yum. If
# rpm must be used, then yum-config-manager must be used to enable/disable repos as a
# separate step.
#
# Another problem I ran into was yum installing the 32-bit (*.i686) package rather than
# the 64-bit (*.x86_64) verision of the package. I never figured out why. So I had
# to specify the *.x86_64 package explicitly. The search tools (eg. 'whatprovides')
# did not list the 64 bit package either even though a manual search through the
# package showed the 64 bit components were present.
#
# Sometimes it is difficult to determine which package must be in installed to satisfy
# a particular dependency. 'whatprovides' is a very useful tool for this
#> yum --enablerepo centos whatprovides libgdk_pixbuf-2.0.so.0
#> rpm -q --whatprovides libgdk_pixbuf
sudo yum --enablerepo centos install -y gdk-pixbuf2.x86_64
sudo yum --enablerepo centos install -y gtk2.x86_64
sudo yum --enablerepo centos install -y libnotify.x86_64
sudo yum --enablerepo centos install -y gnome-icon-theme
sudo yum --enablerepo centos install -y redhat-menus
sudo yum --enablerepo centos install -y gstreamer-plugins-base.x86_64
# problem when we get to libvte, installing libvte requires expat, which conflicts with amazon lib
# the centos package version was older and did not install right lib version
# but … the expat dependency was coming from a dependency on python-libs.
# the easiest workaround was to install python using the amazon repo, that in turn
# installs a version of python libs that is compatible with the version of libexpat on the system.
sudo yum install -y python
sudo yum --enablerepo centos install -y vte.x86_64
sudo yum --enablerepo centos install -y libical.x86_64
sudo yum --enablerepo centos install -y gnome-keyring.x86_64
# another sticky point, xfdesktop requires desktop-backgrounds-basic, but ‘whatprovides’ does not
# provide any packages for this query (not sure why). It turns out this is provided by the centos
# repo, installing ‘desktop-backgrounds-basic’ will try to install the package redhat-logos, but
# unfortunately this is obsoleted by Amazon’s generic-logos package
# The only way I could find to get around this was to erase the generic logos package.
# This doesn't seem too risky since this is just images for the desktop and menus.
#
sudo yum erase -y generic-logos
# Amazon repo must be disabled to prevent interference with the install
# of redhat-logos
sudo yum --disablerepo amzn-main --enablerepo centos install -y redhat-logos
# next problem is a dependency on dbus. The dependency comes from dbus-x11 in
# centos repo. It requires dbus version 1.2.24, the amazon image already has
# version 1.6.12 installed. Since the dbus-x11 is only used by the GUI package,
# easiest way around this is to install dbus-x11 with no dependency checks.
# So it will use the newer version of dbus (should be OK). The main thing that could be a problem
# here is if it skips some other dependency. When doing manually, its possible to run the install until
# the only error left is the dbus dependency. It’s a bit risky running in a script since, basically it’s assuming
# all the dependencies are already in place.
yumdownloader --enablerepo centos dbus-x11.x86_64
sudo rpm -ivh --nodeps dbus-x11-1.2.24-8.el6_6.x86_64.rpm
rm dbus-x11-1.2.24-8.el6_6.x86_64.rpm
sudo yum install -y xfdesktop.x86_64
# We need the version of poppler-glib from centos repo, but it is found in several repos.
# Disable the other repos for this step.
# On EMR systems a newer version of poppler is already installed. So move up 1 level
# in dependency chain and force install of tumbler.
if [ $IS_EMR -eq 1 ]
then
yumdownloader --enablerepo centos tumbler.x86_64
sudo rpm -ivh --nodeps tumbler-0.1.21-1.el6.x86_64.rpm
else
sudo yum --disablerepo amzn-main --disablerepo amzn-updates --disablerepo epel --enablerepo centos install -y poppler-glib
fi
sudo yum install --enablerepo centos -y polkit-gnome.x86_64
sudo yum install --enablerepo centos -y control-center-filesystem.x86_64
sudo yum groupinstall -y Xfce
以下是centos存储库配置文件的内容：
centos.repo
[centos]
name=CentOS mirror
baseurl=http://repo1.ash.innoscale.net/centos/6/os/x86_64/
failovermethod=priority
enabled=0
gpgcheck=1
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
如果您需要的只是在Amazon AMI上安装桌面软件包的方法，那么您就完成了。本文的其余部分介绍了如何配置VNC以通过SSH隧道访问桌面，以及如何打包所有这些以便可以从脚本轻松启动实例。
以下是我安装GUI的顶级脚本。配置几个变量后，它首先要做的是从上面的步骤1调用脚本。这个脚本有一些额外的包袱，因为我已经将它构建为在常规ec2实例上工作，或者是emr，以root身份或ec2-user工作。基本步骤是
需要注意的几个要点：
这假设您将通过SSH隧道访问VNC服务器。最后，这似乎是最简单，最可靠的安全方式。由于您可能在安全组规范中打开了SSH端口，因此您不必对其进行任何更改。此外，VNC客户端/服务器的加密配置并不简单。错误似乎很容易让你的通讯不加密。其设置位于vncservers文件中。 -localhost开关告诉vnc只接受本地连接。 ＆＃39; -nolisten tcp＆＃39;告知关联的xserver模块也不接受来自网络的连接。最后，＆＃39; -SecurityTypes无＆＃39; switch允许你打开你的VNC会话而不输入passwd，因为进入机器的唯一方法是通过ssh，额外的密码检查似乎是多余的。
xstartup文件确定第一次启动VNC会话时将启动的内容。我注意到很多关于这个主题的帖子都跳过了这一点。如果您没有告诉它启动Xfce桌面，那么启动VNC时只会出现一个空白窗口。我在这里的配置非常简单。
尽管我上面提到VNC服务器配置为不提示输入密码，但它仍然需要.vnc目录中的passwd文件才能启动服务器。第一次运行脚本时，它会在尝试启动服务器时失败。通过ssh登录计算机并运行＆＃39; vncpasswd＆＃39;。它将在.vnc目录中创建一个passwd文件，您可以将其保存为在安装期间作为这些脚本的一部分使用。请注意，我已经读过VNC没有做任何复杂的事情来保护passwd文件。因此，我不建议使用您用于其他更重要帐户的密码。
installGui.sh
#!/bin/bash
# echo each command
set -x
BIN_DIR="${BASH_SOURCE%/*}"
ROOT_DIR=$(dirname $BIN_DIR)
RSRC_DIR=$ROOT_DIR/rsrc
VNC_DIR=$RSRC_DIR/vnc
# Install user config files into ec2-user home directory
# if it is available. In practice, this should always
# be true
if [ -d "/home/ec2-user" ]
then
USER_ACCT=ec2-user
else
USER_ACCT=hadoop
fi
HOME_DIR="/home"
# Use existence of hadoop home directory as proxy to determine if
# this is an EMR system. Can be used later to differentiate
# steps on EC2 system vs EMR.
if [ -d "/home/hadoop" ]
then
IS_EMR=1
else
IS_EMR=0
fi
# execute Xfce desktop install
. "$BIN_DIR/installXfce.sh"
# now roughly follow the following from step 3: https://devopscube.com/setup-gui-for-amazon-ec2-linux/
sudo yum install -y pixman pixman-devel libXfont
sudo yum -y install tigervnc-server
# install the user account configuration file.
# This setup assumes the user will always connect to the VNC server
# through an SSH tunnel. This is generally more secure, easier to
# configure and easier to get correct than trying to allow direct
# connections via TCP.
# Therefore, config VNC server to only accept local connections, and
# no password required.
sudo cp $VNC_DIR/vncservers-$USER_ACCT /etc/sysconfig/vncservers
# install the user account, vnc config files
sudo mkdir $HOME_DIR/$USER_ACCT/.vnc
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc
# need xstartup file to tell vncserver to start the window manager
sudo cp $VNC_DIR/xstartup $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/xstartup
# Even though the VNC server is config'd to not require a passwd, the
# server still looks for the passwd file when it starts the session.
# It will fail if the passwd file is not found.
# The first time these scripts are run, the final step will fail.
# Then manually run
#> vncpasswd
# It will create the file ~/.vnc/passwd. Then save this file to persistent
# storage so that it can be installed to the user account during
# server initialization.
sudo cp $ROOT_DIR/home/user/.vnc/passwd $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/passwd
# This script will be running as root if called from the EC2 launch
# command. VNC server needs to be started as the user that
# you will connect to the server as (eg. ec2-user, hadoop, etc.)
sudo su -c "sudo service vncserver start" -s /bin/sh $USER_ACCT
# how to stop vncserver
# vncserver -kill :1
# On the remote client
# 1. start the ssh tunner
#> ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>
# for debugging connection use -vvv switch
# 2. connect to the vnc server using client on the remote machine. When
# prompted for the IP address, use 'localhost:5901'
# This connects to port 5901 on your local machine, which is where the ssh
# tunnel is listening.
VNCSERVERS
# The VNCSERVERS variable is a list of display:user pairs.
#
# Uncomment the lines below to start a VNC server on display :2
# as my 'myusername' (adjust this to your own). You will also
# need to set a VNC password; run 'man vncpasswd' to see how
# to do that.
#
# DO NOT RUN THIS SERVICE if your local area network is
# untrusted! For a secure way of using VNC, see this URL:
# http://kbase.redhat.com/faq/docs/DOC-7028
# Use "-nolisten tcp" to prevent X connections to your VNC server via TCP.
# Use "-localhost" to prevent remote VNC clients connecting except when
# doing so through a secure tunnel. See the "-via" option in the
# `man vncviewer' manual page.
# Use "-SecurityTypes None" to allow session login without a password.
# This should only be used in combination with "-localhost"
# Note: VNC server still looks for the passwd file in ~/.vnc directory
# when the session starts regardless of whether the user is
# required to enter a passwd.
# VNCSERVERS="2:myusername"
# VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"
VNCSERVERS="1:ec2-user"
VNCSERVERARGS[1]="-geometry 1280x1024 -nolisten tcp -localhost -SecurityTypes None"
的xstartup
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
# exec /etc/X11/xinit/xinitrc
/usr/share/vte/termcap/xterm &
/usr/bin/startxfce4 &
一旦您在EC2上运行VNC服务器，您就可以尝试连接它。首先打开到您的实例的SSH隧道。 5901是VNC服务器从vncservers文件侦听显示1的端口。它将侦听端口5902等上的显示2.此命令将创建从本地计算机上的端口5901到实例上的端口5901的隧道。
ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>
现在打开您首选的VNC客户端。在提示输入服务器IP地址的地方输入：
本地主机：5901
如果没有任何反应，那么启动vnc服务器时出现问题，或者存在阻止客户端到达服务器的连接问题，或者vncservers配置文件中可能存在问题
如果出现一个窗口，但它只是空白，请检查Xfce安装是否已成功完成，以及是否已安装xstartup文件。
如果您只需要执行此操作，那么将脚本sftp到您的实例并手动运行就可以了。否则，当您需要使用GUI启动实例时，您将尽可能地自动执行此操作，以使其更快，更不容易出错。
自动化的第一步是创建一个EFS卷，其中包含可在启动实例时挂载的脚本和配置文件。亚马逊在创建网络文件系统方面有很多info。创建音量时，有几点需要注意。如果您不希望您的卷向全世界开放，您可能需要创建一个用于EFS卷的自定义安全组。我为我的EFS卷创建了安全组（称之为NFS_Mount），它只允许来自我的其他安全组的端口2049上的入站TCP流量，称之为MasterVNC。然后，在创建实例时，请确保将MasterVNC安全组与该实例相关联。否则，EFS卷不允许您的实例与其连接。
现在安装EFS卷：
sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR
现在使用以下目录结构使用步骤1和2中提到的6个文件填充/ mnt / YOUR_MOUNT_POINT_DIR。回想一下，您必须首次使用命令＆＃39; vncpasswd＆＃39;创建passwd文件。它将在〜/ .vnc / passwd。
创建文件/mnt/YOUR_MOUNT_POINT_DIR/bin/installGui.sh /mnt/YOUR_MOUNT_POINT_DIR/bin/installXfce.sh
的/ mnt / YOUR_MOUNT_POINT_DIR / RSRC / VNC / VNCSERVERS-EC2用户 的/ mnt / YOUR_MOUNT_POINT_DIR / RSRC / VNC / xstartup中 的/ mnt / YOUR_MOUNT_POINT_DIR / RSRC / VNC / passwd的
/mnt/YOUR_MOUNT_POINT_DIR/rsrc/yum/yum.repos.d/centos.repo
此时，使用GUI设置实例应该非常简单。像往常一样创建实例（确保包含MasterVNC安全组），ssh到实例，安装EFS卷，然后运行installGui.sh脚本。
您可以更进一步，使用本地计算机上的AWS CLI工具在一步中启动实例。为此，您需要安装EFS卷并使用AWS CLI命令的参数运行installGui.sh脚本。这只需要创建一个顶级脚本并将其传递给CLI命令。
当然有几个并发症。 EC2和EMR使用不同的开关和机制来附加脚本。此外，在EMR上我只希望GUI安装在主节点（而不是核心节点或任务节点）上。
启动EC2实例需要使用--user-data开关将脚本嵌入到命令中。通过在本地计算机上指定脚本文件的绝对路径，可以轻松完成此操作。
aws ec2 run-instances --user-data file:///PATH_TO_YOUR_SCRIPT/top.sh ... other options
EMR启动不支持从本地文件嵌入脚本。相反，您可以在引导操作中指定S3 URI。
aws emr create-cluster --bootstrap-actions '[{"Path":"s3://YOUR_BUCKET/YOUR_DIR/top.sh","Name":"Custom action"}]' ... other options
最后，您将在下面的top.sh中看到，大多数脚本都是一个函数，用于确定计算机是基本EC2实例还是EMR主机。如果不是这样，脚本可能是3行。你可能想知道为什么不使用内置的＆run; if＆＃39;引导动作而不是编写我自己的函数。内置＆＃39; run-if＆＃39;脚本有一个错误，并没有正确运行位于S3的脚本。
将它们放入init序列后进行调试可能是一个挑战。有一点可以帮助的是日志文件：/var/log/cloud-init-output.log。这将捕获在引导程序初始化期间运行的脚本的所有控制台输出。
top.sh
#!/bin/bash
# note: conditional bootstrap function run-if has a bug, workaround ...
# this function adapted from https://forums.aws.amazon.com/thread.jspa?threadID=222418
# Determine if we are running on the master node.
# 0 - running on master, or non EMR node
# 1 - running on a task or core node
check_if_master_or_non_emr() {
python - <<'__SCRIPT__'
import sys
import json
instance_file = "/mnt/var/lib/info/instance.json"
try:
with open(instance_file) as f:
props = json.load(f)
is_master_or_non_emr = props.get('isMaster', False)
except IOError as ex:
is_master_or_non_emr = True # file will not exist when testing on a non-emr machine
if is_master_or_non_emr:
sys.exit(1)
else:
sys.exit(0)
__SCRIPT__
}
check_if_master_or_non_emr
IS_MASTER_OR_NON_EMR=$?
# If this machine is part of EMR cluster, then ONLY install on the MASTER node
if [ $IS_MASTER_OR_NON_EMR -eq 1 ]
then
sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR
. /mnt/YOUR_MOUNT_POINT_DIR/bin/installGui.sh
fi
exit 0
答案 1 :(得分：0)
@蒂姆·瑞安（Tim Ryan） 不幸的是，我无法发表评论，但是installGui.sh中步骤1的这一行似乎引起了问题：
YUM_RSRC_DIR=$RSRC_DIR/yum
到达时
sudo cp $YUM_RSRC_DIR/yum.repos.d/centos.repo /etc/yum.repos.d/
有一个错误，但是我想知道我是否也正确地解释了这一行。我猜$ RSRC_DIR通常是/ etc，但是两行的第一行似乎会使$ YUM_RSRC_DIR类似于/ etc / yum。当您到达第二条发布的行$ YUM_RSRC_DIR / yum.repos.d / centos.repo时，它似乎正在尝试复制不存在的 /etc/yum/yum.repos.d/centos.repo（授予，无论文件/目录是什么，我都会收到“找不到文件”错误）。是的，那是在我将您提到的centos.repo文件放置在installGUI.sh下面的多个位置之后。
我仍在尝试，但是我认为应该解决