是否可以仅使用 NAT 从私有子网访问 Internet，而没有 Internet 网关与其 VPC 关联？
具有公共和私有子网 (NAT) 的 VPC如下所述。仅凭这句话，就可以得出一个公平的结论，即它只需要 NAT 即可到达 Internet，而无需 Internet Gateway 连接其 VPC。
具有自己的弹性 IPv4 地址的 NAT 网关。私有子网中的实例可以通过 IPv4 上的 NAT 网关向 Internet 发送请求（例如，用于软件更新）。
我相信对于从 VPC 到达 Internet 的任何流量，VPC 都需要 Internet 网关，但文档没有明确说明，因此感到困惑。
想了解为什么使用成为ansible 用户导致“权限被拒绝（公钥）”的机制。
用户ansible运行 ansible-playbook 以签出 Github 存储库。SSH 密钥已使用 ssh-copy-id 与用户一起复制。
没有成为，剧本运行。
[ansible@ip-172-31-39-108 playbooks]$ whoami
ansible
[ansible@ip-172-31-39-108 playbooks]$ ansible-playbook git.yml
PLAY [Git example] *************************************************************
TASK [setup] *******************************************************************
ok: [ub01]
TASK [check out the repository on the host] ************************************
changed: [ub01]
PLAY RECAP *********************************************************************
ub01 : ok=2 changed=1 unreachable=0 failed=0
但是，使用“成为：是”会导致错误。
[ansible@ip-172-31-39-108 playbooks]$ ansible-playbook git.yml
PLAY [Git example] *************************************************************
TASK [setup] *******************************************************************
ok: [ub01]
TASK [check out the repository on the host] ************************************
fatal: [ub01]: FAILED! => {"changed": false, "cmd": "/usr/bin/git clone --origin origin '' /home/ansible/project/mezzanine-example", "failed": true, "msg": "Cloning into '/home/ansible/project/mezzanine-example'...\nPermission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.", "rc": 128, "stderr": "Cloning into '/home/ansible/project/mezzanine-example'...\nPermission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n", "stdout": "", "stdout_lines": []}
to retry, use: --limit @/home/ansible/playbooks/git.retry
PLAY RECAP *********************************************************************
ub01 : ok=1 changed=0 unreachable=0 failed=1
ssh-agent 正在运行并且已添加私钥。
[ansible@ip-172-31-39-108 playbooks]$ eval $(ssh-agent -s)
Agent pid 1513
[ansible@ip-172-31-39-108 playbooks]$ ssh-add ~/.ssh/id_rsa
Identity added: /home/ansible/.ssh/id_rsa (/home/ansible/.ssh/id_rsa)
请解释为什么会发生这种情况或指向要调查的资源。
此外，我只在运行 ansible-playbook 的服务器上启动了 ssh-agent，而不是在目标服务器上启动。Github SSH 身份验证如何在目标服务器上发生？
- name: Git example
hosts: webservers
become: no # <----- Changing to yes cause the issue
become_user: ansible
become_method: sudo
vars:
repo_url: [email protected]:lorin/mezzanine-example.git
proj_dirname: /home/ansible/project
proj_name: mezzanine-example
proj_path: "{{ proj_dirname }}/{{ proj_name }}"
tasks:
- name: check out the repository on the host
git: repo={{ repo_url }} dest={{ proj_path }} accept_hostkey=yes
主机
[webservers]
ub01
#rh01
ansible.cfg
[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes
Ansible playbook 在 RedHat 上运行。
NAME="Red Hat Enterprise Linux Server"
VERSION="7.3 (Maipo)"
目标主机是 Ubuntu。
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
TASK [check out the repository on the host] ************************************
task path: /home/ansible/playbooks/git.yml:12
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/source_control/git.py
<ub01> ESTABLISH SSH CONNECTION FOR USER: None
<ub01> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ForwardAgent=yes)
<ub01> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ub01> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ub01> SSH: PlayContext set ssh_common_args: ()
<ub01> SSH: PlayContext set ssh_extra_args: ()
<ub01> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r)
<ub01> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r ub01 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /tmp/ansible-tmp-1485919043.94-240537002849590 `" && echo ansible-tmp-1485919043.94-240537002849590="` echo /tmp/ansible-tmp-1485919043.94-240537002849590 `" ) && sleep 0'"'"''
<ub01> PUT /tmp/tmpAjaOMc TO /tmp/ansible-tmp-1485919043.94-240537002849590/git.py
<ub01> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ForwardAgent=yes)
<ub01> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ub01> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ub01> SSH: PlayContext set ssh_common_args: ()
<ub01> SSH: PlayContext set sftp_extra_args: ()
<ub01> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r)
<ub01> SSH: EXEC sftp -b - -vvv -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r '[ub01]'
<ub01> ESTABLISH SSH CONNECTION FOR USER: None
<ub01> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ForwardAgent=yes)
<ub01> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ub01> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ub01> SSH: PlayContext set ssh_common_args: ()
<ub01> SSH: PlayContext set ssh_extra_args: ()
<ub01> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r)
<ub01> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r ub01 '/bin/sh -c '"'"'setfacl -m u:ansible:r-x /tmp/ansible-tmp-1485919043.94-240537002849590/ /tmp/ansible-tmp-1485919043.94-240537002849590/git.py && sleep 0'"'"''
<ub01> ESTABLISH SSH CONNECTION FOR USER: None
<ub01> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ForwardAgent=yes)
<ub01> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ub01> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ub01> SSH: PlayContext set ssh_common_args: ()
<ub01> SSH: PlayContext set ssh_extra_args: ()
<ub01> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r)
<ub01> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r -tt ub01 '/bin/sh -c '"'"'sudo -H -S -n -u ansible /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-cxuzmrsbxdvydelfnjrsmgvocgkeptxd; /usr/bin/python /tmp/ansible-tmp-1485919043.94-240537002849590/git.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
<ub01> ESTABLISH SSH CONNECTION FOR USER: None
<ub01> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ForwardAgent=yes)
<ub01> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ub01> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ub01> SSH: PlayContext set ssh_common_args: ()
<ub01> SSH: PlayContext set ssh_extra_args: ()
<ub01> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r)
<ub01> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ForwardAgent=yes -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ansible/.ansible/cp/ansible-ssh-%h-%p-%r ub01 '/bin/sh -c '"'"'rm -f -r /tmp/ansible-tmp-1485919043.94-240537002849590/ > /dev/null 2>&1 && sleep 0'"'"''
fatal: [ub01]: FAILED! => {
"changed": false,
"cmd": "/usr/bin/git clone --origin origin '' /home/ansible/project/mezzanine-example",
"failed": true,
"invocation": {
"module_args": {
"accept_hostkey": true,
"bare": false,
"clone": true,
"depth": null,
"dest": "/home/ansible/project/mezzanine-example",
"executable": null,
"force": false,
"key_file": null,
"recursive": true,
"reference": null,
"refspec": null,
"remote": "origin",
"repo": "[email protected]:lorin/mezzanine-example.git",
"ssh_opts": null,
"track_submodules": false,
"umask": null,
"update": true,
"verify_commit": false,
"version": "HEAD"
},
"module_name": "git"
},
"msg": "Cloning into '/home/ansible/project/mezzanine-example'...\nPermission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.",
"rc": 128,
"stderr": "Cloning into '/home/ansible/project/mezzanine-example'...\nPermission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n",
"stdout": "",
"stdout_lines": []
}
to retry, use: --limit @/home/ansible/playbooks/git.retry
PLAY RECAP *********************************************************************
ub01 : ok=1 changed=0 unreachable=0 failed=1
感谢@Jakuje 和其他文章的回答，了解到 SSH 代理侦听 UNIX 套接字文件。文件名存储在 SSH_AUTH_SOCK 环境变量中。但是，SUDO 消除了环境变量，因此 sudo-ed SSH 客户端不知道如何与 SSH 代理通信。因此，无法通过 SSH 认证。
解决方案在@Jakuje 提供的文章中。
在环境中安装了 Spark 集群，没有更改 spark-env.sh、spark-defaults.conf 文件和程序中的 SparkConf 对象。
对于 N Queen 程序，分区数为 2，仅分配一个节点任务。对于字数统计程序，分区数为 22，任务分配给所有节点。对这两个程序都使用了 spark-submit。
N皇后
val sparkConf = new SparkConf().setAppName("NQueen").set("spark.files.overwrite", "true")
val sc = new SparkContext(sparkConf)
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
def isSafe(column: Int, placement: List[Int]): Boolean = { ... }
def placeQueensAt(row: Int, placements: Set[List[Int]]): Set[List[Int]] = { ... }
val initial = sc.parallelize(queensAtFirst)
//val initial = sc.parallelize(queensAtFirst, 12)
println("Partitions = %d".format(initial.partitions.size))
val result = initial.flatMap(x => placeQueensAt(1, Set(x))).collect()
字数
val sparkConf = new SparkConf().setAppName("WordCount").set("spark.files.overwrite", "true")
val sc = new SparkContext(sparkConf)
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val lines = sc.textFile("hdfs:/user/wynadmin/sfpd.csv")
println("Patitions = %d".format(lines.partitions.size))
val words = for (line <- lines; word <- line.split(",") if word.toLowerCase.matches("[a-z]+")) yield (word, 1)
val counts = words.reduceByKey(_ + _)
Ubuntu 14.04 上的 Spark 2.0.1（3 个节点，每个节点有 4 个 CPU）。
独立部署（不是 YARN 也不是 Mesos）