我使用Apache camel FTP和AWS模块(v2.18)在SFTP和AWS S3之间创建路由。通过ssh jump-host建立与SFTP位置的连接。
能够通过Unix命令连接:
sftp -o UserKnownHostsFile=/dev/null
-o StrictHostKeyChecking=no
-i /path/to/host/private-key-file
-o 'ProxyCommand=ssh
-o UserKnownHostsFile=/dev/null
-o StrictHostKeyChecking=no
-i /path/to/jumphost/private-key-file
-l jumphostuser jump.host.com nc sftp.host.com 22' sftp-user@sftp.host.com
但是在使用Apache camel进行连接时出现错误:
Cannot connect/login to: sftp://sftp-user@sftp.host.com:22
出于测试目的,我尝试使用Spring -Integration连接到SFTP,并且我能够使用下面提到的相同代理实现(JumpHostProxyCommand)成功完成。
以下是我一直使用的Spring boot + Apache Camel代码:
Jsch代理:
import com.jcraft.jsch.*;
class JumpHostProxyCommand implements Proxy {
String command;
Process p = null;
InputStream in = null;
OutputStream out = null;
public JumpHostProxyCommand(String command) {
this.command = command;
}
public void connect(SocketFactory socket_factory, String host, int port, int timeout) throws Exception {
String cmd = command.replace("%h", host);
cmd = cmd.replace("%p", new Integer(port).toString());
p = Runtime.getRuntime().exec(cmd);
log.debug("Process returned by proxy command {} , {}", command, p);
in = p.getInputStream();
log.debug("Input stream returned by proxy {}", in);
out = p.getOutputStream();
log.debug("Output stream returned by proxy {}", out);
}
public Socket getSocket() {
return null;
}
public InputStream getInputStream() {
return in;
}
public OutputStream getOutputStream() {
return out;
}
public void close() {
try {
if (p != null) {
p.getErrorStream().close();
p.getOutputStream().close();
p.getInputStream().close();
p.destroy();
p = null;
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Spring boot camel配置:
@Configuration
public class CamelConfig {
@Autowired
DataSource dataSource;
@Bean(name = "jdbcMsgIdRepo")
public JdbcMessageIdRepository JdbcMessageIdRepository() {
return new JdbcMessageIdRepository(dataSource,"jdbc-repo");
}
@Bean(name = "s3Client")
public AmazonS3 s3Client() {
return new AmazonS3Client();
}
@Bean(name="jumpHostProxyCommand")
JumpHostProxyCommand jumpHostProxyCommand()
{
String proxykeyFilePath = "/path/to/jumphost/private-key-file";
String command = "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /proxy/host/key/path -l jumphostuser jump.host.com nc %h %p";
log.debug("JumpHostProxyCommand : " + command);
return new JumpHostProxyCommand(command);
}
}
Camel Route建设者:
@Component
public class FtpRouteInitializer extends RouteBuilder {
@Value("${s3.bucket.name}")
private String s3Bucket;
@Autowired
private JdbcMessageIdRepository repo;
@Override
public void configure() throws Exception {
String ftpRoute = "sftp://sftp-user@sftp.host.com:22/?"
+ "delay=300s"
+ "&noop=true"
+ "&idempotentRepository=#jdbcMsgIdRepo"
+ "&idempotentKey=${file:name}-${file:modified}"
+ "&proxy=#jumpHostProxyCommand"
+ "&privateKeyUri=file:/path/to/host/private-key-file"
+ "&jschLoggingLevel=DEBUG"
+ "&knownHostsFile=/dev/null"
+ "&initialDelay=60s"
+ "&autoCreate=false"
+ "&preferredAuthentications=publickey";
from(ftpRoute)
.routeId("FTP-S3")
.setHeader(S3Constants.KEY, simple("${file:name}"))
.to("aws-s3://" + s3ucket + "?amazonS3Client=#s3Client")
.log("Uploaded ${file:name} complete.");
}
}
build.gradle文件:
task wrapper(type: Wrapper) {
gradleVersion = '2.5'
}
ext {
springBootVersion = "1.4.1.RELEASE"
awsJavaSdkVersion = "1.10.36"
postgresVersion = "11.2.0.3.0"
jacksonVersion = "2.8.4"
sl4jVersion = "1.7.21"
junitVersion = "4.12"
camelVersion ="2.18.0"
}
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.4.1.RELEASE")
}
}
repositories {
mavenCentral()
}
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'spring-boot'
sourceCompatibility = 1.8
targetCompatibility = 1.8
springBoot {
executable = true
}
dependencies {
//logging
compile("ch.qos.logback:logback-classic:1.1.3")
compile("ch.qos.logback:logback-core:1.1.3")
compile("org.slf4j:slf4j-api:$sl4jVersion")
//Spring boot
compile("org.springframework.boot:spring-boot-starter-web:$springBootVersion")
compile("org.springframework.boot:spring-boot-starter-jdbc:$springBootVersion")
compile("org.apache.camel:camel-spring-boot-starter:$camelVersion")
//Jdbc
compile("postgresql:postgresql:9.0-801.jdbc4")
//Camel
compile("org.apache.camel:camel-ftp:$camelVersion")
compile("org.apache.camel:camel-aws:$camelVersion")
compile("org.apache.camel:camel-core:$camelVersion")
compile("org.apache.camel:camel-spring-boot:$camelVersion")
compile("org.apache.camel:camel-sql:$camelVersion")
//Aws sdk
compile("com.amazonaws:aws-java-sdk:$awsJavaSdkVersion")
//Json
compile("com.fasterxml.jackson.core:jackson-core:$jacksonVersion")
compile("com.fasterxml.jackson.core:jackson-annotations:$jacksonVersion")
compile("com.fasterxml.jackson.core:jackson-databind:$jacksonVersion")
compile("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:$jacksonVersion")
//Swagger
compile("io.springfox:springfox-swagger2:2.0.2")
compile("io.springfox:springfox-swagger-ui:2.0.2")
//utilities
compile('org.projectlombok:lombok:1.16.6')
compile("org.apache.commons:commons-collections4:4.1")
compile("org.apache.commons:commons-lang3:3.4")
//Junit
testCompile("junit:junit:$junitVersion")
testCompile("org.springframework.boot:spring-boot-starter-test:$springBootVersion")
testCompile("org.mockito:mockito-all:1.10.19")
}
过去两天我一直在努力找出错误的根本原因,对此问题的任何帮助都非常感谢。谢谢!
答案 0 :(得分:0)
尝试在运行此代码的计算机上的ssh配置文件中添加跳转主机配置。您应该能够使用配置文件中指定的主机的跳转主机透明地连接到服务器,而无需在sftp命令中指定任何代理或跳转主机。
设置动态跳转主机的示例配置如下:
Host sftp.host.com
user sftp-user
IdentityFile /home/sftp-user/.ssh/id_rsa
ProxyCommand ssh sftp-user@jump.host.com nc %h %p 2> /dev/null
ForwardAgent yes
您可以在主机行中添加多个主机或正则表达式模式。此条目需要在〜/ .ssh / config文件中完成(如果尚未存在,则创建此文件)。