ITPub博客
首页 博客 专栏 学院 社区 IT新媒体评选
发博

首页 > 数据库 > 数据库开发技术 > Spring Boot集成SFTP快速入门Demo

Spring Boot集成SFTP快速入门Demo

原创 数据库开发技术 作者：hblog 时间：2024-07-19 22:31:52 0 删除 编辑

1.什么是SFTP？

SFTP（SSH File Transfer Protocol，也称 Secret File Transfer Protocol），是一种基于SSH（安全外壳）的安全的文件传输协议。使用SFTP协议可以在文件传输过程中提供一种安全的加密算法，从而保证数据的安全传输，所以SFTP是非常安全的。但是，由于这种传输方式使用了加密/解密技术，所以传输效率比普通的FTP要低。SFTP是SSH的一部分，SFTP没有单独的守护进程，它必须使用SSHD守护进程（端口号默认是22）来完成相应的连接操作，sftp服务作为ssh的一个子服务，是通过 /etc/ssh/sshd_config 配置文件中的 Subsystem 实现的，如果没有配置 Subsystem 参数，则系统是不能进行sftp访问的。所以，要分离ssh和sftp服务的话，基本的思路是创建两个sshd进程，分别监听在不同的端口，一个作为ssh服务的deamon，另一个作为sftp服务的deamon。

Spring Integration核心组件

  • SftpSessionFactory: sftp 客户端与服务端的会话工厂。客户端每次访问服务器时都会创建一个 session 对象，且可以通过 SftpSessionCaching 将 session 对象缓存起来，支持 session 共享，即可以在一个会话上进行多个 channel 的操作。如果 session 被重置，则在最后一次 channel 关闭之后，将断开连接。isSharedSession 为 true 时 session 将共享。

  • SftpSessionCaching: sftp 会话缓存工厂。通过 poolSize 和 sessionWaiteTimeout 来设置缓存池大小和会话等待超时时间。缓存池默认是无限大，超时时间默认是 Integer.MAX_VALUE。

  • SftpRemoteFileTemplate: 基于 SftpSessionFactory 创建的 sftp 文件操作模板类。其父类是 RemoteFileTemplate。支持上传、下载、追加、删除、重命名、列表、是否存在。基于输入输出流实现。

  • SftpInboundChannelAdapter: sftp 入站通道适配器。可同步远程目录到本地，且可监听远程文件的操作，可实现下载。

  • SftpOutboundChannelAdapter: sftp 出站通道适配器。实际是一个 sftp 消息处理器，将在服务器与客户端之间创建一个消息传输通道。此处的消息指的是 Message 的 payload，其支持 File、byte[]、String。其支持 ls、nlst、get、rm、mget、mv、put、mput 操作。

  • Channel Adapter: 通道适配器，实际上是适配消息在客户端和服务器之间的传输。inbound adapter 是接收其它系统的消息，outbound adapter 是发送消息到其它系统。

  • @ServiceActivator: 将注解作用的方法注册为处理消息的站点，inputChannel 表示接收消息的通道。

2.环境搭建

docker run -p 22:22 -d atmoz/sftp foo:pass:::upload

验证环境

说明已经可以连接上去了

3.代码工程

实验目标

实现文件上传和下载

pom.xml

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <parent>        <artifactId>springboot-demo</artifactId>        <groupId>com.et</groupId>        <version>1.0-SNAPSHOT</version>    </parent>    <modelVersion>4.0.0</modelVersion>
    <artifactId>sftp</artifactId>
    <properties>        <maven.compiler.source>8</maven.compiler.source>        <maven.compiler.target>8</maven.compiler.target>    </properties>    <dependencies>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-web</artifactId>        </dependency>
        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-autoconfigure</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-test</artifactId>            <scope>test</scope>        </dependency>        <dependency>            <groupId>org.springframework.integration</groupId>            <artifactId>spring-integration-sftp</artifactId>            <!--            <version>5.4.1</version>-->            <version>5.2.8.RELEASE</version>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-configuration-processor</artifactId>            <optional>true</optional>        </dependency>        <dependency>            <groupId>org.projectlombok</groupId>            <artifactId>lombok</artifactId>            <optional>true</optional>        </dependency>    </dependencies></project>

service

package com.et.sftp.service.impl;
import com.et.sftp.config.SftpConfiguration;import com.et.sftp.service.SftpService;import lombok.extern.slf4j.Slf4j;import org.springframework.integration.file.remote.FileInfo;import org.springframework.integration.sftp.session.SftpRemoteFileTemplate;import org.springframework.stereotype.Component;import org.springframework.web.multipart.MultipartFile;
import javax.annotation.Resource;import java.io.*;import java.util.List;

@Slf4j@Componentpublic class SftpServiceImpl implements SftpService {
    @Resource    private SftpRemoteFileTemplate remoteFileTemplate;
    @Resource    private SftpConfiguration.SftpGateway gateway;
    /**     * single file upload     *     * @param file File     */    @Override    public void uploadFile(File file) {        gateway.upload(file);    }
    /**     * single file upload by byte[]     *     * @param bytes bytes     */    @Override    public void uploadFile(byte[] bytes, String name) {        try {            gateway.upload(bytes, name);        } catch (Exception e) {            log.error("error:", e);        }    }
    /**     * uopload by path     *     * @param bytes     * @param filename     * @param path     */    @Override    public void upload(byte[] bytes, String filename, String path) {        try {            gateway.upload(bytes, filename, path);        } catch (Exception e) {            log.error("error:", e);        }    }
    /**     * list files by path     *     * @param path     * @return List<String>     */    @Override    public String[] listFile(String path) {        try {            return remoteFileTemplate.execute(session -> {                return session.listNames(path);            });        } catch (Exception e) {            log.error("error:", e);        }        return null;    }

    /**     * list file and directory by path     *     * @param path     * @return List<String>     */    @Override    public List<FileInfo> listALLFile(String path) {        return gateway.listFile(path);    }
    /**     * download     *     * @param fileName      * @param savePath      * @return File     */    @Override    public File downloadFile(String fileName, String savePath) {        try {            return remoteFileTemplate.execute(session -> {                remoteFileTemplate.setAutoCreateDirectory(true);                boolean existFile = session.exists(fileName);                if (existFile) {                    InputStream is = session.readRaw(fileName);                    return convertInputStreamToFile(is, savePath);                } else {                    return null;                }            });        } catch (Exception e) {            log.error("error:", e);        }        return null;    }
    /**     * read file     *     * @param fileName     * @return InputStream     */
    @Override    public InputStream readFile(String fileName) {        return remoteFileTemplate.execute(session -> {            return session.readRaw(fileName);        });    }
    /**     * files is exists     *     * @param filePath      * @return boolean     */    @Override    public boolean existFile(String filePath) {        try {            return remoteFileTemplate.execute(session ->                    session.exists(filePath));        } catch (Exception e) {            log.error("error:", e);        }        return false;    }
    public void renameFile(String file1, String file2) {        try {            remoteFileTemplate.execute(session -> {                session.rename(file1, file2);                return true;            });        } catch (Exception e) {            log.error("error:", e);        }    }
    /**     * create directory     *     * @param dirName     * @return     */
    @Override    public boolean mkdir(String dirName) {        return remoteFileTemplate.execute(session -> {            if (!existFile(dirName)) {
                return session.mkdir(dirName);            } else {                return false;            }        });    }
    /**     * delete file     *     * @param fileName      * @return boolean     */    @Override    public boolean deleteFile(String fileName) {        return remoteFileTemplate.execute(session -> {            boolean existFile = session.exists(fileName);            if (existFile) {                return session.remove(fileName);            } else {                log.info("file : {} not exist", fileName);                return false;            }        });    }
    /**     * batch upload (MultipartFile)     *     * @param files List<MultipartFile>     * @throws IOException     */    @Override    public void uploadFiles(List<MultipartFile> files, boolean deleteSource) throws IOException {        try {            for (MultipartFile multipartFile : files) {                if (multipartFile.isEmpty()) {                    continue;                }                File file = convert(multipartFile);                gateway.upload(file);                if (deleteSource) {                    file.delete();                }            }        } catch (Exception e) {            log.error("error:", e);        }    }
    /**     * batch upload (MultipartFile)     *     * @param files List<MultipartFile>     * @throws IOException     */    @Override    public void uploadFiles(List<MultipartFile> files) throws IOException {        uploadFiles(files, true);    }
    /**     * single file upload (MultipartFile)     *     * @param multipartFile MultipartFile     * @throws IOException     */    @Override    public void uploadFile(MultipartFile multipartFile) throws IOException {        gateway.upload(convert(multipartFile));    }
    @Override    public String listFileNames(String dir) {        return gateway.nlstFile(dir);    }
    @Override    public File getFile(String dir) {        return null;    }
    @Override    public List<File> mgetFile(String dir) {        return null;    }
    @Override    public boolean rmFile(String file) {        return false;    }
    @Override    public boolean mv(String sourceFile, String targetFile) {        return false;    }
    @Override    public File putFile(String dir) {        return null;    }
    @Override    public List<File> mputFile(String dir) {        return null;    }
    @Override    public String nlstFile(String dir) {        return gateway.nlstFile(dir);    }
    private static File convertInputStreamToFile(InputStream inputStream, String savePath) {        OutputStream outputStream = null;        File file = new File(savePath);        try {            outputStream = new FileOutputStream(file);            int read;            byte[] bytes = new byte[1024];            while ((read = inputStream.read(bytes)) != -1) {                outputStream.write(bytes, 0, read);            }            log.info("convert InputStream to file done, savePath is : {}", savePath);        } catch (IOException e) {            log.error("error:", e);        } finally {            if (inputStream != null) {                try {                    inputStream.close();                } catch (IOException e) {                    e.printStackTrace();                }            }            if (outputStream != null) {                try {                    outputStream.close();                } catch (IOException e) {                    log.error("error:", e);                }            }        }        return file;    }
    private static File convert(MultipartFile file) throws IOException {        File convertFile = new File(file.getOriginalFilename());        convertFile.createNewFile();        FileOutputStream fos = new FileOutputStream(convertFile);        fos.write(file.getBytes());        fos.close();        return convertFile;    }}
package com.et.sftp.service;
import org.springframework.integration.file.remote.FileInfo;import org.springframework.web.multipart.MultipartFile;
import java.io.File;import java.io.IOException;import java.io.InputStream;import java.util.List;

public interface SftpService {

    void uploadFile(File file);
    void uploadFile(byte[] bytes, String name);
        void upload(byte[] bytes, String filename, String path);
        String[] listFile(String path);
    List<FileInfo> listALLFile(String path);
        File downloadFile(String fileName, String savePath);

    InputStream readFile(String fileName);
        boolean existFile(String filePath);

    boolean mkdir(String dirName);
    boolean deleteFile(String fileName);
        void uploadFiles(List<MultipartFile> files, boolean deleteSource) throws IOException;
        void uploadFiles(List<MultipartFile> files) throws IOException;
        void uploadFile(MultipartFile multipartFile) throws IOException;

       String listFileNames(String dir);

    File getFile(String dir);
       List<File> mgetFile(String dir);
      boolean rmFile(String file);
        boolean mv(String sourceFile, String targetFile);
       File putFile(String dir);
        List<File> mputFile(String dir);

    //void upload(File file);
    //void upload(byte[] inputStream, String name);
    //List<File> downloadFiles(String dir);

    String nlstFile(String dir);}

config

在配置SFTP adapters之前，需要配置SFTP Session Factory；Spring Integration提供了如下xml和spring boot的定义方式。每次使用 SFTP adapter，都需要Session Factory会话对象，一般情况，都会创建一个新的SFTP会话。同时还提供了Session的缓存功能。Spring integration中的Session Factory是依赖于JSch库来提供。JSch支持在一个连接配置上多个channel的操作。原生的JSch技术开发，在打开一个channel操作之前，需要建立Session的连接。同样的，默认情况，Spring Integration为每一个channel操作使用单独的物理连接。在3.0版本发布之后，Cache Session Factory 出现 (CachingSessionFactory)，将Session Factory包装在缓存中，支持Session共享，可以在一个连接上支持多个JSch Channel的操作。如果缓存被重置，在最后一次channel关闭之后，才会断开连接。

package com.et.sftp.config;
import com.jcraft.jsch.ChannelSftp;import org.springframework.beans.factory.BeanFactory;import org.springframework.boot.context.properties.EnableConfigurationProperties;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.expression.common.LiteralExpression;import org.springframework.expression.spel.standard.SpelExpressionParser;import org.springframework.integration.annotation.Gateway;import org.springframework.integration.annotation.MessagingGateway;import org.springframework.integration.annotation.ServiceActivator;import org.springframework.integration.core.MessagingTemplate;import org.springframework.integration.file.FileNameGenerator;import org.springframework.integration.file.remote.FileInfo;import org.springframework.integration.file.remote.session.CachingSessionFactory;import org.springframework.integration.file.remote.session.SessionFactory;import org.springframework.integration.file.support.FileExistsMode;import org.springframework.integration.sftp.gateway.SftpOutboundGateway;import org.springframework.integration.sftp.outbound.SftpMessageHandler;import org.springframework.integration.sftp.session.DefaultSftpSessionFactory;import org.springframework.integration.sftp.session.SftpRemoteFileTemplate;import org.springframework.messaging.Message;import org.springframework.messaging.MessageHandler;import org.springframework.messaging.handler.annotation.Header;import org.springframework.messaging.handler.annotation.Payload;
import javax.annotation.Resource;import java.io.File;import java.util.List;


@Configuration@EnableConfigurationProperties(SftpProperties.class)public class SftpConfiguration {
    @Resource    private SftpProperties properties;

    @Bean    public MessagingTemplate messagingTemplate(BeanFactory beanFactory) {        MessagingTemplate messagingTemplate = new MessagingTemplate();        messagingTemplate.setBeanFactory(beanFactory);        return messagingTemplate;    }
    @Bean    public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory() {        DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);        factory.setHost(properties.getHost());        factory.setPort(properties.getPort());        factory.setUser(properties.getUsername());        factory.setPassword(properties.getPassword());        factory.setAllowUnknownKeys(true);//        factory.setTestSession(true);//        return factory;        return new CachingSessionFactory<ChannelSftp.LsEntry>(factory);    }
    @Bean    public SftpRemoteFileTemplate remoteFileTemplate(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        return new SftpRemoteFileTemplate(sftpSessionFactory);    }

    @Bean    @ServiceActivator(inputChannel = "downloadChannel")    public MessageHandler downloadHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory, "mget", "payload");        sftpOutboundGateway.setOptions("-R");        sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);        sftpOutboundGateway.setLocalDirectory(new File(properties.getLocalDir()));        sftpOutboundGateway.setAutoCreateLocalDirectory(true);        return sftpOutboundGateway;    }

    @Bean    @ServiceActivator(inputChannel = "uploadChannel", outputChannel = "testChannel")    public MessageHandler uploadHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory);        handler.setRemoteDirectoryExpression(new LiteralExpression(properties.getRemoteDir()));//        handler.setChmod();        handler.setFileNameGenerator(message -> {            if (message.getPayload() instanceof File) {                return ((File) message.getPayload()).getName();            } else {                throw new IllegalArgumentException("File expected as payload.");            }        });        return handler;    }
    @Bean    @ServiceActivator(inputChannel = "uploadByteChannel")    public MessageHandler multiTypeHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory);        handler.setRemoteDirectoryExpression(new LiteralExpression(properties.getRemoteDir()));        handler.setFileNameGenerator(message -> {            if (message.getPayload() instanceof byte[]) {                return (String) message.getHeaders().get("name");            } else {                throw new IllegalArgumentException("byte[] expected as payload.");            }        });        return handler;    }

    @Bean    @ServiceActivator(inputChannel = "lsChannel")    public MessageHandler lsHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory, "ls", "payload");        sftpOutboundGateway.setOptions("-R");         return sftpOutboundGateway;    }

    @Bean    @ServiceActivator(inputChannel = "nlstChannel")    public MessageHandler listFileNamesHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory, "nlst", "payload");        return sftpOutboundGateway;    }
    @Bean    @ServiceActivator(inputChannel = "getChannel")    public MessageHandler getFileHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory, "get", "payload");        sftpOutboundGateway.setOptions("-R");        sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);        sftpOutboundGateway.setLocalDirectory(new File(properties.getLocalDir()));        sftpOutboundGateway.setAutoCreateLocalDirectory(true);        return sftpOutboundGateway;    }
    /**     * create by: qiushicai     * create time: 2020/11/20     *     * @return     */    @Bean    @ServiceActivator(inputChannel = "abc")    public MessageHandler abcHandler(SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory) {        SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory);        handler.setRemoteDirectoryExpression(new LiteralExpression(properties.getRemoteDir()));        handler.setFileNameGenerator(message -> {            if (message.getPayload() instanceof byte[]) {                System.out.println("receive message:" + new String((byte[]) message.getPayload()));                message.getHeaders().forEach((k, v) -> System.out.println("\t\t|---" + k + "=" + v));                return "ok";            } else {                throw new IllegalArgumentException("byte[] expected as payload.");            }        });        return handler;    }
    /**     *     *  the #root object is the Message, which has two properties (headers and payload) that allow such expressions as payload, payload.thing, headers['my.header'], and so on     *     *  link{ https://stackoverflow.com/questions/46650004/spring-integration-ftp-create-dynamic-directory-with-remote-directory-expressi}     *  link{ https://docs.spring.io/spring-integration/reference/html/spel.html}     * @return     */    @Bean    @ServiceActivator(inputChannel = "toPathChannel")    public MessageHandler pathHandler() {        SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory());        // automatically create the remote directory        handler.setAutoCreateDirectory(true);        handler.setRemoteDirectoryExpression(new SpelExpressionParser().parseExpression("headers[path]"));        handler.setFileNameGenerator(new FileNameGenerator() {            @Override            public String generateFileName(Message<?> message) {                return (String) message.getHeaders().get("filename");            }        });        return handler;    }

    /**     * <ul>     * <li>ls (list files)     * <li> nlst (list file names)     * <li> get (retrieve a file)     * <li> mget (retrieve multiple files)     * <li> rm (remove file(s))     * <li> mv (move and rename file)     * <li> put (send a file)     * <li> mput (send multiple files)     * </ul>     *     * @author ：qiushicai     * @date ：Created in 2020/11/20     * @description：outbound gateway API     * @version:     */    @MessagingGateway    public interface SftpGateway {
        //ls (list files)        @Gateway(requestChannel = "lsChannel")        List<FileInfo> listFile(String dir);

        @Gateway(requestChannel = "nlstChannel")        String nlstFile(String dir);


        @Gateway(requestChannel = "getChannel")        File getFile(String dir);
        @Gateway(requestChannel = "mgetChannel")        List<File> mgetFile(String dir);
        @Gateway(replyChannel = "rmChannel")        boolean rmFile(String file);
        @Gateway(replyChannel = "mvChannel")        boolean mv(String sourceFile, String targetFile);
        @Gateway(requestChannel = "putChannel")        File putFile(String dir);
        @Gateway(requestChannel = "mputChannel")        List<File> mputFile(String dir);
        @Gateway(requestChannel = "uploadChannel")        void upload(File file);
        @Gateway(requestChannel = "uploadByteChannel")        void upload(byte[] inputStream, String name);
        @Gateway(requestChannel = "toPathChannel")        void upload(@Payload byte[] file, @Header("filename") String filename, @Header("path") String path);
        @Gateway(requestChannel = "downloadChannel")        List<File> downloadFiles(String dir);
    }}

从Spring Integration 3.0开始，通过SftpSession对象提供了一个新类Remote File Template。提供了Sftp文件发送、检索、删除和重命名文件的方法。此外，还提供了一个执行方法，允许调用者在会话上执行多个操作。在所有情况下，模板负责可靠地关闭会话。SftpRemoteFileTemplate继承Remote File Template,可以很容易的实现对SFTP文件的发送(包括文件追加，替换等)，检索，删除和重命名。

package com.et.sftp.config;

import lombok.Data;import org.springframework.boot.context.properties.ConfigurationProperties;
@Data@ConfigurationProperties(prefix = "sftp")public class SftpProperties {    private String host;    private Integer port;    private String password;    private String username;    private String remoteDir;    private String localDir;}

application.properties

##sftp propertiessftp.host=127.0.0.1sftp.port=22sftp.username=foosftp.password=passsftp.remoteDir=/uploadsftp.localDir=D:\\tmp\\sync-files

以上只是一些关键代码，所有代码请参见下面代码仓库

代码仓库

  • https://github.com/Harries/springboot-demo

4.测试

在测试包里面有专门的测试类，具体可以查看源代码

文件是否存在

@Testvoid testExistFile() {    boolean existFile = sftpService.existFile("/upload/home222.js");    System.out.println(existFile);}

列出目录下的文件

@Testvoid listFileTest() {    sftpService.listALLFile("/upload").stream().forEach(System.out::println);}

下载文件

 @Test    void testDownLoad() throws Exception {        sftpService.downloadFile("/upload/home.js", "D:\\tmp\\c222c.js");////        sftpService.uploadFile(new File("D:\\tmp\\cc.js"));

//        InputStream inputStream = sftpService.readFile("/upload/cc.js");////        IOUtils.copy(inputStream, new FileOutputStream(new File("D:\\tmp\\" + UUID.randomUUID() + ".js")));
    }

上传文件

@Testvoid uploadFile() {    sftpService.uploadFile(new File("D:\\tmp\\cc.js"));}

5.引用

  • https://docs.spring.io/spring-integration/docs/5.2.0.M4/reference/html/sftp.html

  • http://www.liuhaihua.cn/archives/710920.html

原文作者：HBLOG；如有侵权，请联系管理员删除。

0
0
上一篇： Spring Boot集成Spring Batch快速入门Demo
下一篇： Spring Boot集成syslog快速入门Demo
请登录后发表评论 登录
全部评论
hblog

注册时间：2022-05-04

  • 博文量
    210
  • 访问量
    111759

最新文章

支持我们 作者招募 用户协议 FAQ Contact Us

北京盛拓优讯信息技术有限公司. 版权所有  京ICP备16024965号-8 北京市公安局海淀分局网监中心备案编号：11010802021510 niuxiaotong@pcpop.com 17352615567 未成年人举报专区

广播电视节目制作经营许可证(京) 字第1234号 中国互联网协会会员

违法和不良信息举报电话:18101376593